All in all, the HHKB layout keeps your hands closer to the home row at all times, and I think that's the point of it.
Initially, the FN-layer arrow keys were the hardest to get used to. It took a few weeks of occasional use to get used to that, and I'm sure bouncing back and forth between HHKB and my work or laptop keyboards didn't help the matter. I'd estimate 3-5 days of more intensive use and I'd have gotten accustomed way faster.
Now (a year later) it's all completely second nature. Because my right hand has much less distance to travel, I find the FN-layer arrows more convenient and easier to reach than traditional arrow keys. Seems trivial, but over the dozens, hundreds (?) of times a day you make that reach, it adds up. Your hands are just closer to the home row with the HHKB most of the time, and it keeps you more on target for typing.
I edit a lot of text, and very frequently use arrow keys with modifiers on OS X to keep my hands off the mouse (option-arrow or option-shift-arrow to move or select a word at a time, for instance), and that too has become totally reflexive and does not necessitate looking down at all. I still find myself looking down to find PgUp/Down and Home/End, though. And using the HHKB more intensively over the past month, the proximity of the Delete key to the home row has also become quite welcome.
I will admit to missing the Caps Lock every once in a great while, and while I'm sure the closer Control key placement is a boon on Windows, it's much less relevant on OS X where that key usually sees infrequent use.
The only situations in which I find the HHKB layout a bit sub-optimal are when entering a lot of numbers (as with any compact or TKL), and when trying to casually use the keyboard one-handed, like to activate Dashboard or change the system volume. Casually scrolling through a website with one finger on an arrow- or page-down key is a bit less convenient as well, but definitely not a huge deal.