The first cut is flaring the alu outwards
sounds like that would be in some part down to the one of the qualities if the material your cutting (it can be a bitch)
im not sure if its down to it being rolled as cast seems to machine better (more brittle because of the way it crystallizes I think but dont hold me to it)
the chip clings to the tool, gets dragged back in re-cut and before you know it its cutting more material than it can clear boggs down and snaps

i think your being a tad ambitious with your depth of cut for a router... id stick with the recomended feed n speed and reduce the depth of cut even if it means going to 0.5mm... slow and painful but will get you there in the end :)

im assuming you use WD40 ?
if not dabbing plenty WD40 or maybe paraffin and brushing the chips clear as you go will improve things markedly