Not quite sure what the issue is here. If the output of a psu - any psu - is left floating them you can guarantee what volts you will see between the terminals but volts wrt ground is undetermined. It will be somewhere between earth and full mains depending on leakage, transformer internal coupling, EMI filler leakage and so on. Take one terminal to ground and everything is now properly referenced to that - no more tingles!

I don't see that the EMI filter effectiveness or anything else will be affected. In fact, electrically, you are in a much better position because at the moment I would be worrying about the effect of stray voltages on sensitive components. If that "tingle" happens to occur around any high-impedance inputs...

The reason for leaving the outputs floating is that someone might want a -24V supply so in that case they would ground the +'ve output pin. There might be other reasons but that's one good one.