Yes, you need one like this 5mm head on a 6mm shank (top one on page). If your normal lathe tool is bigger than 6mm then you might want to make up a boring bar holder (or buy one) to hold the cutting edge at the right height.

Boring is not as accurate as reaming, but within 0.05mm is possible with care and light cuts and thats easily good enough for this need.