Hole saws have a bad habit of starting in the wrong place even if the centre drill is in the right location, Always start slowly just touching the work at first and slowly easing the cutter into the work to allow it to find its centre, Even then the hole can be a mm out.

Regards
John