I had some trouble with this too when I first started to use Vim. At first I tried to remap everything, but in the end, I found something simpler. Surely you must know about WASD cursor movement on Qwerty? Another variation of that is ESDF, one set of keys to the right. This leaves your index finger on the homing bar on the left side of the keyboard. On Colemak, this would be FRST.
In Vim, R and S are quite similar, and both are similar to C. F and T are again both similar. Also, FTRS are somewhat less critical than other Vim functions (C can often do what R and S can do or you can use X or D paired with I; F and T can be accomplished with other movement techniques or a general search), which is good because we don't want to cause too much disruption when reassigning keys. We then have HJKL as remappable, and I've done the following:
//change FRST to up, left, down, right respectively
R --> H
S --> J
T --> L
F --> K
H --> S (s stands for substitute and in my mind is now hotSwap)
K --> R (r stands for replace and in my mind is now Kill)
J --> F (f goes before the next instance of a character, maybe it was Forward (?) but now I call it Jump)
L --> T (t goes To the next instance of a character which is similar to Forward. Now I call it Leap, which is similar to Jump but a leap is further)
The rest I keep the same, so you can still find most commands by pressing the letter that goes along with their mnemonic. As a bonus, H and K on Colemak are still next to each other as are J and L, so it makes a lot of sense physically.
In general though, using FRST simply avoids worse problems when trying to find 4 keys that are next to each other to take the place of arrows. If you did a direct HJKL to HNEI on Colemak then you lose Next and Insert which are quite notable to lose.