Still having problems with accuracy.

I have my custom vice jaws that hold my case parts. I use the probe to touch off for origin.
I then run a small test cut to double check and it is usually 0.1-0.2mm out in Y.
I then test again and looks fine.

Remove part from vice and replace with same part, and for some reason the X is now out by quite a bit. 0.3-0.5mm from the looks of it.
I know the vice clamping down on the part does make it bend slightly. But I am only going finger tight to reduce this.

Any ideas?

Literally the last operation on the part to chamfer it. I would do it by hand if I could but I also have to drill holes for bolt heads.