it's actually 3mm rigid PVC which I machined on the CNC so that each cutout was a snug

Looking at the picture it looks very much like foamex which they may call a rigid PVC but because of the way is is made tends not to be great for sealing. If your using a vacuum cleaner to try & maintain a vacuum then an accumulation of very small leaks can cause you problems. One option would be more suck by using a bigger hoover, more hoovers or something else to pull a vacuum & the other option would be less leaks by sealing it better or reducing the area your sucking from. What sort of condition is the holey rubber mat in ?

There are a couple of threads on the forum about making vacuum tables so they might be worth a read.