No idea what 6061 is, HE20 is the best seller for extrusions.

I heard that HSS takes a better edge than carbide, could be a myth.

I think my spindle speed is about right. Increasing the rpm means increasing the feed rate, which increases the power going through the tool which increases the bending. As soom as I drop below 8mm tooling I find bending to be a factor.

I just measured an outline on this part, the drawing says 28.25mm, Mitutoyo calipers say 28.26mm. The error is 0.0004", could be the mill, could be the calipers, works for me