I do my polishing in a carport walled on two sides. Airborne stuff settles on my red paint very quickly, even when it is not windy. However, this stuff is very light, more like small particles of lint (pollen?) than dirt. I don't think it scratches paint easily. If it does, they are VERY shallow.
Having said that, I ALWAYS wipe down a panel with a damp MF immediately before working on it using David B's's Quick Detailing method. I do this wipedown between each application of fresh polish too. I check my pad for bad stuff frequently. I also avoid working on paint when it is windy.
I do mar the paint occasionally while polishing, but it doesn't happen often enough, and the marring is not severe enough to be a problem. I attribute these incidents to excessive pressure on the pad and/or that stray piece of grit.
Mikeman