Because it's not part of the program. It's a little bit like you going for a walk in dense forest, you know where you are because you have a map, but then someone blind folds you and dumps you in a different position with no reference where you have come from.... you are then lost.

Some controllers will let you move out of position and others wont, it's highly controller dependent.

The normal thing to do is program a tool change position.

Example

Z 50.
X-100. Y100. (or were you want to tool to go for a safe tool change position)
Then a program suspend or stop.

Change tool... resume program.

But as we don't know what you procedure is, its hard to know exactly what you are doing.

Again it's not working as you think it should, but the controller is not the problem in this case.