i'd imagine that in small bursts on easy material your ok but if you do anything longer it will get warmer and warmer..

when you say it's not warm you have to consider you are touching the bit on the outside, you have no way of knowing what the temp is inside..

i'd imagine even if you have some kind of syphon system running where you can fill the spindle up with some water and then syphon it off and put fresh in mid job it will help more than just doing nothing