Unless those straight line moves cause an abrupt direction change that creates a corner angle larger than the set corner break angle/corner rounding settings which would force an exact stop, then they shouldn't cause any problems.

Has exact stop mode (G61) been enabled in your controller?
I don't see it in your G-code file, but you could try adding a G64 to the beginning to see if things improve.