That diagram is correct. Think of the Alarm output as a basic single throw switch, that is either open, or closed.

The opto input needs to see 24V. So 24V from the PSU goes to the opto positive, then the opto negative goes to the Alarm positive, before finally completing the circuit by connecting the Alarm negative to the PSU negative.

There a couple different ways you can connect to an e-stop/safety relay, however it depends on what you're using.