All purpose cleaner. (you were close
)
Something like
Meguiars APC, Simple Green, etc.
Alcohol might work. Another idea might be to try a plastic razor blade to scrape it off, if it really is that hard.
Be careful with the WD40. Im not sure how it might affect the finish on your wheels.