I'd very much doubt that the housings are deforming and causing the bearings to bind.

I would think that when you're tightening the bearings onto the MDF, they are twisting some way causing the problems.
How are you tightening the bearings onto the MDF?
Have you tried tightening the bolts up in an alternating pattern (ie. top right, bottom left, bottom right, top left), and only tightening each bolt up a fraction (say an eight of a turn) at a time?