Yeah it's kind of funny...
The default action of the "Delete" key is a forward delete (the Delete that's in the nav cluster above the arrows). Function+"Delete" gives you the backspace.
That's the way almost everyone uses it since you're more likely correcting a typo that you just made. Strangely enough, when you have it set to Backspace, Function+Delete doesn't give you a forward delete.
Function+Tilde (just above the Delete) gives you the numpad forward delete, which I believe registers differently to the computer as it's from the numberpad rather than the nav cluster.
The HHKB layout is quite interesting, but it isn't how I would have made it, personally!