Several factors come into play here I think. For me, the main reason I would fit a high-speed spindle onto a Mill is so could use modern high-feed adaptive tool paths which can drastically increase MRR.

Next would be the use of small tooling, sub 3mm, where 24K rpm is considered the minimum.

However, the quality and power of the High-speed spindle make a big difference. For a long time, I've fit what would be considered the default 2.2Kw ER20 24K water-cooled spindle coming out of China to just about every Router/MiniMill serious hobby level machine I've ever built and they do an OK job for the money they cost but I wouldn't fit one to a milling machine.
However, I've recently come across a Chinese supplier of Spindles whose spindles are in another class in every way and can supply electro spindles up to 47Kw, but even their 2.2Kw ER20 spindles are out of this world and will quite happily cut steel as well as aluminum with Sub 12mm tooling, these I would fit onto a milling machine and could see them being far more useful than the main spindle for 12mm or less tooling as the feeds and MRR would be so much higher, esp with Adaptive tool-paths.