If you gear it down with a worm gear drive then far less torque is required from the stepper motor since (the majority of) worm drives do not drive backwards - if you apply a significant force to the worm-wheel the motor will not spin. Backlash might be a problem, although there are ways to reduce it to a negligible magnitude.

Also, since you only require 180° of movement, you could use some sort of lever arrangement with a ballscrew which can have very low backlash.