I would expect that connector has a small hidden tab of plastic which needs to be pushed back by a thin screw driver before it can be removed.

Once removed you can try a multimeter across different pairs of terminal pins and check the resistance across the coils. On my motor (completely different Nema23) this was 8 ohms ? can't quite remember. If you look at the colour coding on the wires at the connector and trace them back to the stepper drive you can see A+ and A-, and B+ and B-. So there should be resistance across the motor pins which where connected to RED and BLUE (A+ and A-), and across WHITE and YELLOW (B+ and B-). Infinite resistance across anything A and anything B.
Since it works, but with some vibration I doubt the above is the issue.

If you can't remove the motor connector then remove the green terminal block from the stepper driver and put the multimeter probes in there as this is a direct read of the motor windings.

I don't know much (!) about small Nema 17 motors used for lightweight laser printers but the MS on my router is more like 800 or 1600 (it was a long time ago). At x32 MS (3200) sounds like the driver would be working pretty hard? Is there a manual which states it needs to have this setting? You could try changing the DIP switches to drop to 1600 MS. Be aware the laser will move the wrong distance but you may be able to jog it around at see if the roughness is improved. Then change the 'steps-per' configuration in the laser software (assuming this is configurable).