Cropwell - I'm not explaining myself well and I don't particularly want to drag out this discussion as it's a distraction to the real OP post. However, if you'll bear with me for a contrived example.

Workpiece surface = Z0, clearance place = Z3.

Pocket #1 to Z-10. Controller moves to Z0, then proceeds to pocket to Z-10, but either motor stalls or - something not yet discussed (but something I have experienced) - tool slips in collet - controller believes tool cuts to Z-10, but actually loses e.g. 2mm due to either stall or slip. Tool position is, in this case Z-8, controller believes tool is at Z-10. Controller then retracts 10mm plus clearance- to a total of 13mm. Controller believes tool is at Z3, but having started the retract at tool position Z-8 the 13mm retract results in a tool Z5, controller Z3. Essentially the controller has lost the true reference to the tool Z.

Pocket #2 is depth Z-1; controller moves by clearance height to Z0 (tool Z2), then cuts to what it believes is Z-1, but is actually Z1 according to the tool cutting face. And on it goes. Contrived example perhaps, but if the OP is 2.5d carving then feasible. Of course this is just one example of one possible problem.

OP - this considers another possibility of tooling slipping in the collet. All things discussed here are possibilities to investigate.