I wipe the wheels with my waffle weave after I'm done drying the car. The wheels are spotless so there's no concern about contaminating the towels with filth.
I don't really towel dry mine much, rather I blow 'em off with the AirWand and the air compressor. Any leftover water gets wiped off when I FK425 them at the end of the wash.