That looks about 30um / 0.03mm runout and presumably worse at the cutter tip. That means the cutter will be doing most of the cutting on only one or two edges. You should really use a better quality toolholder, or you will be unable to do any serious cutting without breaking tools and / or poor surface finish.

Rather than use an ER collet toolholder, you should get some (Chinese) sidelock holders that are inherently much better due to the lack of collets - it's the collets that are the biggest contributor to runout. Then you can see how good your spindle is - and make best use of it without breaking cutters.

Very interesting work - please let us know how you are getting on!