This is why pictures would help with a diagnosis.
I've worked on a very similarly colored M6 and found the "light swirls" were fairly tricky to remove.
In all honesty, I don't think Pinnacle's swirl remover carries enough weight to remove those defects. I could be wrong, but even by Pinnacle's own description of their product, I'd doubt it to be strong enough to help you very much.
Although the technique you're using could be the issue, I'd track down a bottle of Megs 205, which should be easy, and see if that gets you the finish you're looking for.
This is a quick 50/50 I did with my cell phone, results were achieved with two passes of 205 and a white CCS pad on my Flex.
205 is easy to work with, and it should play nice with your orange pad, but it may leave some fine hazing depending on how you use it. That Pinnacle swirl remover would finish the job, no doubt.