I bought Yoshimi Battles the Pink Robots http://en.wikipedia.org/wiki/Yoshimi_battles_the_pink_robots CD in 2002. It isn't Japanese music, but Japan inspired.
 
There was some Japanese text on the CD, and I think that it made me want to learn more about Japanese language and culture.
 
A week ago I found out I could read what the cover says in Japanese (It had been a while since I listened to this music).
It made me happy.
 
I highly recommend The Flaming Lips.
Yoshimi albums cover art is by the lead man.

I have been too lazy to listen more of them.