Hey Tony!
I certainly understand your concern. Scratch-X is actually a fairly abrasive product that can remove the flaws that you speak of but, it may be a bit too abrasive IMO.
A very common practice within the detailing world is to use the least aggressive product first to yield results. Often times, you may have to move 'up' a product in order to achieve those results.
Here's my recommendation...
From what you've described, your swirls are very light and not very noticable. You're looking to achieve perfection with your paint and will be doing it by hand. This isn't a lofty goal and you should be able to obtain it.
I'd recommend going with a product like
Meguiar's #9 Swirl Remover 2.0. Premiumautocare.com is a very reputable online vendor and can supply this product to you if you are unable to locate it locally (or, just do what I do and forget about the local suppliers and go straight to PAC!).
#9 (as it's commonly called) is specifically designed with this type of surface imperfection in mind and features diminishing microscopic abrasives that break down as you 'work' the product. Now, when I say 'work', I don't mean that you simply wipe it on and wipe it off. What I mean is that you continually rub this product within a smaller area until it is almost dry or has a very slightly wet nap to it. What you're doing is removing a very small, thin layer of clearcoat to bring the swirls (which are actually microscopic scratches) out of the paint. It's a VERY safe process and is VERY easy to do by hand in my opinion. #9 is one of my STAND BY products as I've achieved great results oodles of times with it.
As a hint, it may take one, two, maybe three applications based upon the severity of your swirls. Don't loose faith in the product. I can assure you that, if you're working it correctly, taking your time, and paying attention to the area you're working upon, you will achieve results. Be patient.
Feel free to fire any other questions that you may have...
Good luck!!!
RP