That's normally because you're using an unsuitable tool type (i.e. a drill, and not an endmill), a toll with too big a diameter, and/or the toolpath settings aren't letting it run at tight enough of an arc to fit in the hole.

You might want to have a look at this video for various ways to bore a hole - https://www.youtube.com/watch?v=nKbEzSleF14
(NYCNC vids are usually pretty good, and their Fusion Friday videos have a good few tips in them)