This is the first time I've thought about it, but this is what I'd find useful.

3 Buttons:
E-stop
Pause
Play / Unpause

2 Buttons:
Jog +
Jog -

Rotary switch:
Off
Select x
Select y
Select z

Rotary switch:
Jog scale 0.01
Jog scale 0.1
Jog scale 1.0
etc

A headphone jack to connect my touchoff plate (and an LED to check it's working before it drives the cutter through the table because of a faulty connection)

2 more buttons:
Feed override - up
Feed override - down

Possibly 2 more buttons (but only if I can work out how to acknowledge the tool change screens in EMC2 from the pendant, without going back to the PC:
G92.2, and raise Z to zero to do a tool change
G92 to reset the co-ordinates, and re-start after tool change.

Hmm, yep, not so small and dinky either.
Have you seen this post on Anders Wallin's blog?
http://www.anderswallin.net/2006/11/jogging-emc2/