When I went to Akihabara it felt like pretty much every computer related shop had keyboards, so they aren't that hard to find really.
I do remember Sofmap (ソフマップ) had quite a lot of keyboards in some of their stores (these are everywhere). There were also some sort of super large store (not Yodobashi) that also had some. If you're looking to buy a white HHKB Pro 2 specifically, like I were, I'd recommend Dospara, since they were the only store that had it in stock when I went there in June. Was 2000¥ more expensive than the black version at some other places, but for me it was worth it. :>
Something I've been wondering about is if there are any stores in Tokyo (or the rest of Japan really) that sell used computer accessories? I'd love to find something more or less rare (there aren't a lot of rare keyboard stuff in Sweden), and I think those kind of stores would be the best place to look.