Although I have limited CNC, I am generally considered fairly expert when it comes to wood...., my thoughts on the above

rule of thumb for cutting wood... depth of cut never greater than cutter diameter, to a maximum of shaft diameter... so 4mm = max 4mm cut, for say 10mm with 1/4(6mm) shank = 1/4 etc.... obviously this is a catch all and can be pushed, but with this you will generally not break tooling, burn material or blunt tooling with rubbing, damage collets, and allows for better chip clearance.... although at less than 6mm this will often be an issue with cuts more than 2x dia. and up to 1/2" at more than 3x dia deep

cutting speed is crucial with small cutters, at 4mm I would be running nearer 30000rmp and adjust depth and feed to suit... ie shallow & fast