I think you're over thinking this.

Provided the retaining plate actually presses against the bearing and holds it in the housing, then there will be enough give in the plate for what is a very minimal amount of expansion.