As Doddy says, the opto may not give enough current to switch the relay. I used the 'arduino' opto isolated type of relay module that uses 5v at very little current for switching and I just drove it off two of the outputs (A or B step/Dir), one for the spindle and the other for coolant. I had an onboard relay for the spindle power or coolant pump, whatever, but never used it. It seems the simplest solution for you to use the onboard relay.