I should mention that I recently remapped caps-lock to control on both machines, so I'm already training myself on that little quirk.
The majority of my time on the computer is spent in vim and Firefox with the vimperator plugin, so most of my scrolling is done with h/j/k/l, but I do still find myself using arrow keys in other programs. I suppose my main concern is that I'll be on a normal board and find myself hitting ctrl+/ or something expecting a down arrow, or be on the HHKB and find myself hitting esc expecting a `. Little things like that could add up to a decrease in productivity, which would defeat the purpose of the HHKB layout.
Of course, If the Topre switches are all they're cracked up to be, I may have trouble switching between the HHKB and another board regardless of layout!