You'll be fine to leave the stepper motor engaged when machining manually. Don't leave the handles engaged when driving with the stepper motor, as their extra inertia will greatly increase the torque requirement.