Chuck the part facing down in the mill and put the centre drill facing up in the vice?
You can also do some small basic turning in a mill with a tool held in a vice ;-)

- Nick