Strength is not the issue.
Rigidity is.

All machine tools are loaded, at max, to == 2% of their strength ie yield point.
Machine tools today are lightly loaded stressed-skin structures.

You can make a router with the ball bearings, and it will work. Yes.

It will not be productive vs others, and cannot use modern carbide cutters, due to flex/vibration/chatter, effectively.
Your proposed solution will work fine at 5-10 ipm, feed, where real machines run 100-200-800 ipm.
So about 20x less productivity.