My guide will be divided into two entries: A. shops that let you purchase directly and B. middleman services. This one deals with the former.
CDJapan is my personal favorite because they offer numerous payment and shipment options and have a handy point system. Their catalogue is very extensive, which can only be beaten by Amazon Japan.
Yesasia is not as popular as it used to be and their Japanese music selection is relatively limited but if you're interested in Chinese and Korean stuff, they're the place to stop by. They also offer free shipping for orders over $39 USD, but from what I know, the CD prices are already a little inflated to include a bit of the shipping charge.
I can't say anything for HMV Japan because I've found everything I looked for elsewhere. Like Amazon, they only take credit cards. Check their help guide.
eBay is self-explanatory. You may be surprised at what you can find there. My only warning about eBay is when you buy from stores based in China and sometimes Japan, please make sure that the CD is not a promo or sample. Those basically have limited resell value in case you ever want to sell them away. They were meant to be played on radio and in shops on the street, yet those stores got them for free and are planning to sell them to you for a profit. I'm saying this as someone who mistakenly bought a few promos myself. You may dismiss this warning if you don't care.
I haven't used Amazon much outside of Japan because their shipping charges are steep and the couple times I bought from them, the local post office has slapped a huge customs charge on top of it all. And all I ordered were a few CDs for each order. J-Canuck apparently didn't have such unpleasant experience, and we both live in Ontario, Canada, so go figure. What I do love about Amazon is their Marketplace, which sells a ton of new and used CDs. Shipping for those is cheap but be prepared for the slowness of SAL delivery because the sellers don't let you choose. Basically, just search for the CD you want in regular search (copy/paste the name in Japanese after googling or just type if you can), and then look for links to the right of the cover image that say something like this: