Perhaps I'm missing something but it looks to me more like a job for the lathe (to make the fundamentally cylindrical body), followed by the mill (to position and drill 4 holes), then bang some pins in the holes. How were you planning to use the end mills?

304 isn't actually very hard (typically HRB 70 ie it doesn't even make the bottom of the HRC scale) - and being stainless you can't through harden it either. Machineability is different to hardness. Some form of carbon steel would be a better, allowing you to machine it easily and then harden it with a torch and a bucket of water. Then you'll get something nearer HRC60.

If it's an emergency and you are short of carbon steel, how about using an old cold chisel, crowbar etc as a donor. Cut a bit off with an angle grinder, heat it up to a cherry red and let it cool. Then you should find it easy to machine. It's easy enough to harden up again using the bucket of water etc. You can get tools like this at Aldi etc for peanuts if you don't have any.

While you are figuring this out, you could be letting some penetrating oil slowly do its business....