The easiest way to get that level of distortion is to have a weak machine, especially in one axis.

This can be difficult to resolve because even if you use a very low feedrate (which is not great long term anyway) you can get pulling of the bit in the material if the machine frame stiffness is low.

If you lean your weight on the spindle can you deflect it?

I don't think it is lost steps etc.

That would be my first guess anyway. What machine is it? Can you post pictures of the machine?