Hey Mike,
Since it sounds like you are currently doing all of the polishing and waxing by hand, I thought I would add a few additional helpful comments.
The most important thing to remember when polishing and waxing by hand is that all cloths must be clean and remain clean at all times. With regard to an applicator cloth, you could use a small folded light weight, white terry cloth towel to apply the polish and wax, instead of the foam type applicator.
When applying either polish or wax, it is a good idea to dampen your applicator cloth, and put a small amount of the product directly on the cloth. Do not put product directly on the surface of the vehicle. Starting at the passenger side of the hood, apply the cloth to the surface and begin to use either a gentle circular polishing motion or back and forth motion, applying an even pressure. If you are right handed you may want to work around the car in a counter-clockwise direction. Do not apply too much pressure. You should begin to see a change in surface texture as a result of your efforts. This change in surface texture is an indication that road film and surface contamination are being removed and that the surface is in fact becoming clean. The applicator cloth should have just enough dampness to allow the cloth to glide smoothly. You will need to add polish or wax to the cloth, as necessary, to insure even coverage. Remember, do only one section at a time, until you see how easily the polish or wax comes off. Then proceed accordingly.
To remove either polish or wax residue after drying, you could use two clean towels aprox. 16� x 16�. One towel for your right hand and one for your left. Place one towel in the palm of each hand. Proceed to apply the towels to the dried polish or wax. Begin removing the polish or wax with either a gentle circular motion or a back and forth motion. Again, do not apply too much pressure. You should change towel surfaces by flipping from one side to the other. If the polish or wax comes off relatively easy, then you can do a larger section. However, if the polish or wax is hard to remove, then you will want to stay with doing small sections at a time.
Mike, the key to reducing the chance of causing those swirl marks and very fine scratches is to simply not apply too much pressure during either the application or removal process. That's really the key.