I had (still have) a similar problem with broadly similar numbers. However, in my case it was always undercutting - external sizes were about 0.3mm oversize and holes about 0.3mm undersize. After a bit of discussion on the forum, I concluded that it was machine weakness - bend, bearing slop, backlash, or whatever - and for the parts I was cutting I found that defining the 3mm cutter to be 2.3mm instead and generating gcode accordingly gave nicely fitting parts. Over the weekend I was cutting some birch ply instead of the 3mm MDF I had been working with, using a 6mm cutter instead of the 3mm cutter I had used, but the error was about the same. Cutter speed around 12K, 500mm/min speed, 2-3mm DOC (=depth of cut). Missing steps would probably show up quite clearly with the size of error as the cuts would not line up at the end of cutting out a component.
Given that you are seeing everything coming out slightly undersize (both holes and outside profiles) it does sound like a calibration problem.
I know that it's not a fix and a really crude workaround, but there's a useful box in the toolpath generation window in VCarve labelled "allowance". That lets you specify an amount to overcut/undercut a given part (although really it's intended for something like cutting male and female parts to the same nominal dimension but allowing you to create a bit of clearance so that they fit nicely). Saves faking the cutter diameter, which was my first workaround!