The problem is I can't even find a single point lathe tool that will work in the smaller diameters.

One option I had forgotten about was a tapping head mounted in a drill press with the work piece mounted high up to give enough room for the tap to pass through, although I'd have to make sure the drill has got enough travel.