Layout Analysis: Dvorak/Colemak/MTGAP/Carpalx/Workman/my own

[Steven Tammen]

Snarfangel, did you run that layout through an optimizer with the E on a thumb key? I never was able to find much good discussion of E on thumb keys and would be curious to know why you chose that particular layout (I'm planning on getting an ergodox or Kinesis Advantage, depending on the merits of the new iterations of both products and how Keyboardio and Axios turn out).

I think in practicality that it would probably be best to leave the letters on the 30 key layout as is and use the thumb keys to take some of the load off the pinkies with respect to shift, enter, backspace, etc. Especially if you keep alternation high, my guess is that the reduced load on the pinkies would be more beneficial than removing e. Especially since it still allows you to use laptop keyboards.

[Snarfangel]

Snarfangel, did you run that layout through an optimizer with the E on a thumb key? I never was able to find much good discussion of E on thumb keys and would be curious to know why you chose that particular layout (I'm planning on getting an ergodox or Kinesis Advantage, depending on the merits of the new iterations of both products and how Keyboardio and Axios turn out).

I think in practicality that it would probably be best to leave the letters on the 30 key layout as is and use the thumb keys to take some of the load off the pinkies with respect to shift, enter, backspace, etc. Especially if you keep alternation high, my guess is that the reduced load on the pinkies would be more beneficial than removing e. Especially since it still allows you to use laptop keyboards.

Actually, I took the letter E completely out of the stats in MTGAP and put it under the left thumb manually. I used files like AllDigraphsNoE.txt (I never claimed to be creative!), and had listings like:
th 100272945963
in 68595215308
an 55974567611
on 49570981965
at 41920838452
nd 38129777631
ti 37856196209
or 35994097756
of 33130341561
is 31817918249
(etc.)
without any e (notice the lack of he, er, re, and the like).

I left the letter E in the program so I would have the right number of characters, but because it had no statistics, it would stick it in the keyboard equivalent of Siberia. I also tried it with and without forcing number keys at the top of the keyboard, and got things like:

Hands: 49% 50%
Fingers: 8.0% 10% 14% 17% 0.00% 0.00% 16% 16% 10% 7.0%

 5  E  }  #  ^  (   Z  Q  \  {  9  4  3
    <  |  U  P  >   V  D  L  B  J  2  [  0
    I  O  A  N  Y   F  H  T  S  R  +
    1  $  ;  C  %   K  M  G  W  X

 8  e  @  '  _  *   z  q  =  ~  `  ]  )
    !  ,  u  p  -   v  d  l  b  j  &  6  7
    i  o  a  n  y   f  h  t  s  r  :
    ?  /  .  c  "   k  m  g  w  x

Fitness:       842845944581
Distance:      694524426001
Finger work:   11764900580
Inward rolls:  10.39%
Outward rolls: 1.36%
Same hand:     41.06%
Same finger:   2.20%
Row change:    14.83%
Home jump:     1.20%
Ring jump:     2.52%
To center:     2.65%
To outside:    0.22%

Time elapsed: 0 hours, 41 minutes, 38 seconds

Shift-8 is 5 is a weird result, but I wasn't looking at the numbers as much as getting an idea of what settings made sense. I kept tweaking settings, re-running it, then test possible candidates on they keyboard layout analyzer here:http://patorjk.com/keyboard-layout-analyzer/#/config using the Ergodox layout and a few huge text files (collected works of Mark Twain, stuff like that, plus examples of programming code, a few years of my emails, and random stuff).

Here are some of the lower-case versions of keyboards that MTGAP came up with:

+  !  @  "  ?  #   ~  q  w  z  &  `  e
    '  y  u  ,  _   j  v  d  f  =  *  \  4
    r  i  n  o  .   m  h  t  s  c  g
    :  /  -  a  ;   k  l  p  x  b
   
 `  +  !  =  @  ?   ~  q  k  x  &  [  e
    '  y  .  ,  ;   j  m  p  b  *  z  \  >
    r  i  n  a  -   f  h  t  s  c  g
    /  _  u  o  :   "  l  d  v  w
   
 @  1  2  3  4  5   6  7  8  9  0  q  z
    "  y  .  ,  -   v  m  p  b  :  j  e  \
    r  i  n  a  /   k  h  t  s  c  =
    x  _  u  o  '   f  l  d  w  g

   
 `  +  !  =  @  ?   ~  q  k  x  &  [  e
    '  y  .  ,  ;   j  m  p  b  *  z  \  >
    r  i  n  a  -   f  h  t  s  c  g
    /  _  u  o  :   "  l  d  v  w
   

 e  8  9  :  /  _   4  1  0  2  5  6  z
    3  y  u  o  7   v  d  l  b  g  =  j  q
    r  i  n  a  .   f  h  t  s  c  '
    "  )  -  ,  (   k  p  m  w  x
   

 e  8  9  :  /  _   6  1  0  2  4  7  z
    3  y  u  o  5   b  d  l  g  w  =  j  q
    r  i  n  a  .   f  h  t  s  c  '
    "  )  -  ,  (   k  p  m  v  x

 @  1  2  3  4  5   6  7  8  9  0  q  ~
    "  y  .  ,  =   k  m  p  b  x  j  z  e
    r  i  n  a  -   f  h  t  s  c  _
    :  /  u  o  '   v  l  d  w  g
   

 `  @  !  ?  &  e   +  x  q  ~  *  ^  ]
    "  y  .  ,  =   k  m  p  b  j  z  #  6
    r  i  n  a  -   f  h  t  s  c  _
    :  /  u  o  '   v  l  d  w  g
   

 `  &  !  "  ?  #   *  q  g  =  z  \  )
    '  y  u  .  _   j  v  d  f  w  ~  +  e
    r  i  n  o  ,   m  h  t  s  c  ;
    /  :  -  a  @   k  l  p  x  b
   

   ?  =  _  ,  '  @   ~  +  q  x  %  |  e
    .  y  a  l  "   b  p  m  w  j  z  *  #
    u  i  o  r  -   f  h  t  s  c  v
    ;  &  :  /  $   !  n  d  k  g

(etc. for lots and lots of keyboards).

Actually, I think the layout I tried that got the best results on the patorjk analyzer was something like

But the difference was tiny, and it didn't score as well with the MTGAP. Plus, my last name starts with R, and it's easier to hit hit with my left index finger than my right pinky, even if it's on the home row.

So anyway, to answer your initial question, I essentially took out the letter E, put it where I wanted, then had the program calculate the rest of the keys. Then I ran it a bunch of times and tweaked things until it seemed reasonable. And the result seems like it will be better for me than either the Dvorak or Maltron layout, though I still need to program the converter.

[Steven Tammen]

Thanks for taking the time to write all that up.

This seems like really the only way you could do it with the optimizers at this time (pity really, since ADNW, Carpalx, etc. would give you a broader range of testing metrics). Your procedure does leave one wondering how removing e would change digram rolls and alternation between hands in a more statistically rigorous sort of way (noting, of course, that all this layout analysis is ultimately grounded in subjective heuristics and weight values assigned to the parameters). Do you think you could just assign a very positive value to the left thumb key (in terms of layout weighting) and see if MTGAP would place a high frequency letter there as part of the algorithm itself?

I'll be interested to see if anyone else has done any work with this issue. I know Oneproduct mentioned it earlier in this thread as well (but never followed up on it), and I've seen some other discussion about it at GH, DT, and on the Colemak forum. One would think with the explosion of thumb clusters on ergo boards people might think about putting those new keys to good use outside of macros or ctrl.

[Pro XKB]

This seems like really the only way you could do it with the
optimizers at this time (pity really, since ADNW, Carpalx,
etc. would give you a broader range of testing metrics).

The AdNW optimizer has support for thumb keys, including allowing for letters
on thumb keys.  There is some analysis of Shift on thumbs:

  http://adnw.de/index.php?n=Main.OptimierungF%C3%BCrDieGeradeTastaturMitDaumen-Shift

and with letters on thumbs:

  http://adnw.de/index.php?n=Main.FehlerfreihheitUndDaumentasten

[MajorMajor]

Awesome write up, are you still using your custom keyboard layout now?

[Snarfangel]

This seems like really the only way you could do it with the
optimizers at this time (pity really, since ADNW, Carpalx,
etc. would give you a broader range of testing metrics).

The AdNW optimizer has support for thumb keys, including allowing for letters
on thumb keys.  There is some analysis of Shift on thumbs:

  http://adnw.de/index.php?n=Main.OptimierungF%C3%BCrDieGeradeTastaturMitDaumen-Shift

and with letters on thumbs:

  http://adnw.de/index.php?n=Main.FehlerfreihheitUndDaumentasten

Do you know if they did an English version with thumbs?


Just for fun, I made a heat map of the YOR UIAN keyboard using the Collected Works of Mark Twain (courtesy of Project Gutenberg)


Not too bad, really. 

[Pro XKB]

Do you know if they did an English version with thumbs?

I found this is the mailing list archive.  It says the layout with the name 4287 was done for English:

   http://permalink.gmane.org/gmane.comp.hardware.keyboards.layout.nordtast/700

There is a long thread surrounding this, so it is hard to tell what exactly the optimization criteria were.  But it clearly tries to stick with Malt-like criteria, not the default AdNW ones.

[Steven Tammen]

Do you know if they did an English version with thumbs?

I found this is the mailing list archive.  It says the layout with the name 4287 was done for English:

   http://permalink.gmane.org/gmane.comp.hardware.keyboards.layout.nordtast/700

There is a long thread surrounding this, so it is hard to tell what exactly the optimization criteria were.  But it clearly tries to stick with Malt-like criteria, not the default AdNW ones.

Unfortunately I can't read German, and google translate only takes you so far. I was not aware that AdNW supported this (I don't think it used to?), and I can't run the algorithms myself because I'm primarily a writer interested in keyboards (technical things like compiling C code and running from the command line are a bit above me presently). This is more or less why I'm at GH pestering people to help me figure out which layout is best  .

I would need to see what criteria that layout was generated by before I would get too far into trying it out. If you go back a little bit you can see some of the heuristics I proposed when I necro'd this thread (such as same finger, alternation and inward rolls, low travel distance, etc.). I've skimmed a couple of the papers by Lillian Malt and it is my opinion that there are more important things to optimize for than some of the things she aimed to (such as error reduction). I was kind of hoping for more of an analysis of what putting a high frequency letter (such as E) on a thumb cluster did to other optimization parameters.

[Pro XKB]

I can't run the algorithms myself because I'm primarily a writer interested in keyboards (technical things like compiling C code and running from the command line are a bit above me presently)

I guess you overestimate the difficulty.  The most difficult thing for Windows and Mac users appears to be to get a decent compiler installed.

This is more or less why I'm at GH pestering people to help me figure out which layout is best

In my observation, one always needs to play with the optimization parameters get a feel how they influence the layout.  To do this playing over GH will require to many back and forth iterations.

I've skimmed a couple of the papers by Lillian Malt and it is my opinion that there are more important things to optimize for than some of the things she aimed to (such as error reduction).

I tend to agree with you.  Here is a layout optimized with nearly AdNW default criteria (attached), using the 1/2-grams shipped with the optimizer:

2313             194.703 total effort   156.063 positional effort    left right
                   0.545 same finger rp   9.584 shift same finger top  7.7  9.4
  ;.ugf vmlpx     53.761 hand alternat.  53.206 shift hand alter. mid 31.2 29.7
  oaisc dtrnh      1.168 inward/outward  --.--- indir same finger bot  4.8  5.6
  ',yjz qkw-b     16.897 adjacent        13.899 shift adjacent    sum 43.7 56.3
        e         8.8 10.2 11.1 13.6 --.- 11.6 16.5 11.5  9.1  7.7 Sh  1.5  1.3

You would probably prefer not punishing digrams typed by adjacent finger (but I cannot change this, for ethical reasons :-), or to punish outward motions more.

I was kind of hoping for more of an analysis of what putting a high frequency letter (such as E) on a thumb cluster did to other optimization parameters.

And this is kind of analysis that needs playing with the optimization parameters.