You need to look at proper spindles and use a stack of conical spring washers to retain the tool via pull studs and a readily available, off the shelf puller (petal/clamp/claw are useful search terms)
You don't have space in that design for a good pull system.