Yes, I'd start swapping over the motor / encoder etc. Sounds like a buggered component or a wiring fault, crossed over wiring etc. Time to go for a process of elimination etc.

IIRC, you fitted your own encoders. Getting those wired up correctly would be helpful. That must be a fairly likely candidate.