I've yet to find a good polish that actually removes swirls to any extent by hand.
For your efforts, you are better off to use a great glaze to fill them (so long as they are minor) and follow with a Carnauba based wax.
I've tried a LOT of products by hand through the years and finally bought a PC, it is WELL worth the initial investment.
Meg's Deep Crystal #2, 3M Imperial Hand Glaze, Clearkote Vanilla Moose and Red Machine Glaze all work really well by hand for filling minor imperfections. 3M seems to fill the most and last the longest, RMG would be a close second and is a bit easier to use.