From what I know, it is not allowed to put a bigger tool diameter in the collet. You can put the 3.175 tool in the 4-3 mm collet (even if it is not recommended) but not in the 3-2 mm one.

I never tried it myself as I have a set of metric collets with 1 mm increment plus the 1/2, 1/4, 1/8 inch ones.