To me its strange that there seems to be no individual setting for X and Y inverts etc.

re
$3=5 (Step direction invert) ... Is 5 the correct number ? I would have thought 0 or 1 !!
$4=0 (Invert step enable pin)
$5=0 (Invert limit pins)

$20=0 (Soft limits enable) ... Is this enabled or disabled ?
$21=0 (Hard limits enable)

Sorry I don't know much about GRBL