And from it's stump is growing some fresh branches (life springs eternal). I would like an apple tree in another place. Do you think I could saw this stump even with the ground and transplant the whole thing to another place without damaging the new little baby apple trees.

No, The tree stump will have roots that are to big to transplant. Leave the tree that is growing from the stump, and plant another sapling instead. We had 3 apple orchards on our farm. When you cut down a tree, it usually always sprouts another one from the roots. This goes for most trees. We cut down a huge pine tree and two yrs later another tree popped up from the stump. It's a beautiful tree now. If you try to cut the new growth off this stump, you will more than likely kill it.

You would have to dig some of the roots out before you transplant in order for it to survive.