Yet another keyboard layout: eNNe

[iandoug]

Hi, I somehow lost my login stuff, so made a new account. I have been around here much longer 
I don't agree with Carpalx assumptions, so high ranking layouts in Carpalx I don't find very good in real life. I myself like the ADNW analyzer best. Can also be tweaked to your liking, that is by setting penalities for all sorts of things.

What was your previous handle?

FWIW, Den (shenafu.com) has made some changes to Patrick's code: (two versions at least, with different emphasis).
first version includes adding vertical distance as well as horizontal:
http://shenafu.com/code/keyboard/Keyboard%20Layout%20Analyzer.html#/main

Current version has much stronger focus on avoiding pinky use, and avoiding horizontal index movement.
http://shenafu.com/code/keyboard/klatest/#/main

So different layouts do better at the different versions. Version 1 scores are largely in line with Patrick's original, except for the AltGr style layouts (Arensito, AdNW, etc).

From what I've read of CarpalX his methodology is largely in line with Patrick's although I don't think Patrick/KLA worries about rolls or row-jumps.

[Sc0tTy]

Wow.. Haven't checked back in a while. I see my scoring is totally off now with all the new rules in KLA.


This is my current layout:

http://configure.ergodox-ez.com/keyboard_layouts/kljbmn/edit

Have the eNNe home-row pegged down now (no speedtest yet) and I must say I really like the pinky mod!
Been working on positioning the keyboard shortcut layer and stuff.
The Q has been moved to another layer, thinking of doing the same for the Z and Y.
The punctuation's all have their own layer but they might have to move when i do the Z and Y move.
I have shortcuts for Ctrl +Z, X, C, V , Y on the base layer which is great. Saves a lot of pinky strain and bad positioning of fingers.

Was gonna decide what keys I would move next for typing but now with new scoring I think I need to redo some tests. Also I already know I need 4 layer toggles and that won't work how eNNe is now.
I'm also still not sure if I want to use the thumb keys for layer switching, I might move them all to the pinky.

[Sc0tTy]

First evening of hunting for a score, coming close X7.1 layout:


Shift keys are now under Pinky home key and X, Z, Y, Q have been moved to the "home cluster" on the Alt-Layer:


@Ian: Whats the main Idea behind the "move the pinky home key to the middle finger upper row" concept ?
I really like the "home cluster" idea, but what is the research that this is faster/better?
Or is the only upside to lower usage of the pinky? Because now the middle finger gets a bigger load.

I'm trying to put all modifiers and layer-switch keys on the pinky, as its already used to it and its better at it then the thumb (down movement vs sideways movement).
Also with the pinky home letters gone it should be able to take that load no problem

[iandoug]

First evening of hunting for a score, coming close X7.1 layout:

@Ian: Whats the main Idea behind the "move the pinky home key to the middle finger upper row" concept ?
I really like the "home cluster" idea, but what is the research that this is faster/better?
Or is the only upside to lower usage of the pinky? Because now the middle finger gets a bigger load.

1. It's illegal to beat X7.1H ... :-) ... but you're getting very close so I'll have to see where I can improve. But see note below.

2. You lost me here : "move the pinky home key to the middle finger upper row" .... where did you see that? First I've heard of the idea. So I thought maybe there was an error in one of my layouts, but 7.1 looks okay on Klatest.

3. The 'home cluster' aka home block is Den's idea... he is very focused on reducing pinky effort. I think because his model is based on 'weighted effort' where using the pinky gets punished because it is weak. Which means Dvorak and Colemak get smacked because they have common vowels on pinkies.

Been reading some research papers, saw one that says index is actually stronger and more flexible than middle, so in theory it should have lowest effort score, so that it can do more work. Which makes sense because most layouts give the index two columns to worry about. However Klatest also punishes horizontal motion, so inner index column is also punished. All scoring algos that I know of assume middle finger is strongest.

All of which makes finding a winning layout a bit of a challenge :-). Hence my "note" comment above: X7.1H is designed to win on Klatest, but it does not do so well on Den1 scoring, and presumably also not on Patrick's original scoring. Best layout on Den1 is still X6.3H (for English and probably Code).

I've just run a fresh batch of tests using over 300 layouts, and Den1 scoring. Will upload results in next few days, need to fix the database browser program to handle some things differently.
Will then repeat the process using Klatest scoring, and Patrick's original.

I don't think I'll be able to add your latest eNNe versions to the tests... KLA does not handle layers apart from Shift, AltGr, and Shift-AltGr. Which may mean that your scores above are not accurate... did you check the Javascript console output for error messages?

Must figure out how to subscribe to these threads...

Cheers, Ian

[Sc0tTy]

1. It's illegal to beat X7.1H ... :-) ... but you're getting very close so I'll have to see where I can improve. But see note below.

Guess, it's a good thing I don't like laws

2. You lost me here : "move the pinky home key to the middle finger upper row" .... where did you see that? First I've heard of the idea. So I thought maybe there was an error in one of my layouts, but 7.1 looks okay on Klatest.

3. The 'home cluster' aka home block is Den's idea... he is very focused on reducing pinky effort. I think because his model is based on 'weighted effort' where using the pinky gets punished because it is weak. Which means Dvorak and Colemak get smacked because they have common vowels on pinkies.

Well when I started doing eNNe I wanted to do home-block but would get a lower score so I like that its not a big of an issue now. But what I mean is that now your middle finger will do a lot more work (distance) with two of the main letters. Wheres when its behind the pinky you spread the load a bit. Just wondering if there was more info on it

Been reading some research papers, saw one that says index is actually stronger and more flexible than middle, so in theory it should have lowest effort score, so that it can do more work. Which makes sense because most layouts give the index two columns to worry about. However Klatest also punishes horizontal motion, so inner index column is also punished. All scoring algos that I know of assume middle finger is strongest.

Guess my thoughts agree with that research then
With the pinky mod I'm using the upper row is actually a bit harder for the index and ring finger to reach, the bottom row is easyer now. Without the pinky mod the upper row is easier, just so you know.

I've just run a fresh batch of tests using over 300 layouts, and Den1 scoring. Will upload results in next few days, need to fix the database browser program to handle some things differently.
Will then repeat the process using Klatest scoring, and Patrick's original.

I don't think I'll be able to add your latest eNNe versions to the tests... KLA does not handle layers apart from Shift, AltGr, and Shift-AltGr. Which may mean that your scores above are not accurate... did you check the Javascript console output for error messages?

No need to do it, this eNNe is still WIP

Must figure out how to subscribe to these threads...

Cheers, Ian

Have you tried the "Notify" button at the bottom? You should be able to get email notifications.

[iandoug]

Have you tried the "Notify" button at the bottom? You should be able to get email notifications.

From what I can see (both here and in my Notification settings) I'm supposed to get notifications for this topic, but don't seem to... maybe the backend does not like my postgrey filter on my mail server.

[iandoug]

Have you tried the "Notify" button at the bottom? You should be able to get email notifications.

From what I can see (both here and in my Notification settings) I'm supposed to get notifications for this topic, but don't seem to... maybe the backend does not like my postgrey filter on my mail server.

Notification finally arrived. Guess the greylisting slowed it down.

[Sc0tTy]

Ah okay great


Getting closer


For some wierd reason the right pinky has a to high count, not sure where that's coming from.
And even though Enter is way higher then AltGr, if I switch them my score drops and I have no idea why.

Switching spacebar with AltGr also lowers the score even though it (i know from experience) would be better to have the layer toggle key on the other hand.

[iandoug]

For some wierd reason the right pinky has a to high count, not sure where that's coming from.
And even though Enter is way higher then AltGr, if I switch them my score drops and I have no idea why.

Switching spacebar with AltGr also lowers the score even though it (i know from experience) would be better to have the layer toggle key on the other hand.

Um, think your left pinky has higher usage than right pinky.

Interesting idea putting the letters on altgr/shift-altgr like that... not something I have thought of before ...

[iandoug]

You could try this for Dutch:

[Sc0tTy]

Nope only makes it worse.

This is what I mean with the pinky:



Count says 11200 but only has the J key there now and a way lower keycount

[iandoug]

I moved the right pinky home onto the j key. I know this breaks your setup, but it seemed better than moving the j down to where home is, and leaving a large gap to the top keys.

[Sc0tTy]

Well I want the shift modifier under the pinky home as this key is used a lot more then the J key.

But that doen't answer my question, where is that high pinky count coming from?
Is finger keycount displaying the keycount with a penalty ?
Don't know what the pinky penalty is but 11120/2240=5

[iandoug]

Well I want the shift modifier under the pinky home as this key is used a lot more then the J key.

But that doen't answer my question, where is that high pinky count coming from?
Is finger keycount displaying the keycount with a penalty ?
Don't know what the pinky penalty is but 11120/2240=5

:-) That's why I put the shift on the right thumb. And j home on the right pinky. That layout got better scores than X7.1H on Ugly Duckling in Dutch. Which is why I suggested it ;-)


I didn't see such high usage on right pinky as what you are seeing ... might be Input Text dependant? Where you have a lot of characters on left hand that need to be uppercased?

[Sc0tTy]

Well I want the shift modifier under the pinky home as this key is used a lot more then the J key.

But that doen't answer my question, where is that high pinky count coming from?
Is finger keycount displaying the keycount with a penalty ?
Don't know what the pinky penalty is but 11120/2240=5

:-) That's why I put the shift on the right thumb. And j home on the right pinky. That layout got better scores than X7.1H on Ugly Duckling in Dutch. Which is why I suggested it ;-)


I didn't see such high usage on right pinky as what you are seeing ... might be Input Text dependant? Where you have a lot of characters on left hand that need to be uppercased?

As you can see in the images, that score is only with the J on the upper pink row, the shift is still on the thumb. Hence my questioning.

[iandoug]

As you can see in the images, that score is only with the J on the upper pink row, the shift is still on the thumb. Hence my questioning.

You need to move pinky home to the j .

[Sc0tTy]

As you can see in the images, that score is only with the J on the upper pink row, the shift is still on the thumb. Hence my questioning.

You need to move pinky home to the j .

So the high keycount is 2210*5 to calculate for the penalty ?
I just want to confirm that

[Sc0tTy]

Well I did it


Big difference compared to original to the "original final eNNe layout"



Getting pretty excited about it too, this is the reduction in pure finger stress according to my typing statistics:

Stats for when the layout is flipped btw and includes arrow cluster (not in KLA stats)
Good to see how putting those QYZX letters on the homeblock helps a lot on the layout

This is on the dutch/english word list I've used previously.

[iandoug]

Good to see how putting those QYZX letters on the homeblock helps a lot on the layout

This is on the dutch/english word list I've used previously.

Played around a bit... attached :-)

[Sc0tTy]

Good to see how putting those QYZX letters on the homeblock helps a lot on the layout

This is on the dutch/english word list I've used previously.

Played around a bit... attached :-)

Not bad

Must say I don't think its a real world improvement. The J is quite common in Dutch so I'd prefer to have it on the primary layer.
J is 0.67% compared to F at 0.63% and quite a few hits will be CTRL+F and CTRL+SHIFT+F which ill be putting on a separate key.

Also the shift next to the most used key is also not that great imho. Its also a modifier and sideways action with thumb is not great imho.
And I use that key to switch to a layout too.

But you have me an idea and shaved another point off


I also switched the P and W which lowers the score with .02 but in my personal stats W is 1.13% vs P at 0.78% usage. So that should be a lot better in real world (though I'm not sure how much of that is me hitting CTRL+W )
EDIT: Guess that got revert somehow lol, see now its back. Anyways score with P/W switch is 108.29

I also switched the H and U because i think it will make it a little easier to type stuff like "choose".

[Sc0tTy]

So after a while of life getting in the way, I started working on my punctuation and numpad layers when I suddenly got another idea.
That change ended up taking off more then a whole point: 107.12. Then I noticed that having the dash instead of the coma in the normal grid was making it artificially lower (also on the previous 108.29 layout) so I fixed that and this is the result:




I moved the X to the 2nd layer, moved Y to the index finger and then moved the J to the other side, should be okay enough to type "jij" and "je" (though it's not my preference). I was now able to move the CTRL above the SHIFT.

I was then checking the key data in KLA with my own WhatPulse data and decided to fix the inconsistency of the T and Z key's. Their frequency was to low, Z is actually used more frequently than F so I updated my data set and added more of both keys.

I then moved the Z to the middle and F to the index finger and got a little lower score



When looking at the heatmap in KLA v2 you'll see that the biggest load is now on the index finger (corresponding with my data its a 0.36% difference)


So obviously I tried switching the T and N but that gave me quite a bigger score, almost 6pt increase.

The only reason I can think of is that having R, T and L keys would be a lot of keys on just one finger (only the same finger is significantly higher).
I have not yet gotten around that issue to see if I can get done but decided to post an update as it has been a while
Turns out the index sees a big encrease in same finger usage. I guess even though the T is used a lot, its not used as much as N with other vowels:


Attached you'll find the wordlist with extra TZ's and both KLA exports.
I'll be using that wordlist in the future but somewhere in the future I want to change it so that it uses actually words based on the stats and the top word list. Because I do feel that removing and adding random characters does skew KLA's results. Although this layout does look very good in my WhatPulse data.

@Ian: I'm now getting a 4% gap on X7.1H Ergolinear and a 17% gap on BEAKL 9 (pretty significant I would say) on my wordlist, I wanted to know what is your main wordlist for working on a layout and if you know what layout Den uses. So that I can test it against those wordlists and see how it does.

On a side note:
I've been using the pinky mod for quite a while now and I must say that it makes the ErgoDox thumb cluster a lot better to use but also makes everything a little easyer on the pinky. I'll write a longer post in a separate thread for that one of these days.

[Snarfangel]

Just a quick note, since it's time for me to head off to sleep. I went back to some of the old layouts I generated with the MTGap software, and found one that seemed to work unusually well with the current round of keyboard layout analyzers -- lower in rolls than the one I was using, but quite a bit better in hand alternation and lower in same finger use for consecutive letters, and significantly better scores. After trying a variety of inputs and tweaking things in the keyboard layout analyzers at Shenafu's site (https://shenafu.com/code/keyboard/klatest/#/main) as well as http://kla.keyboard-design.com/ I came up with this somewhat unusual layout for my Kinesis Advantage2 that I might as well call KFOU DHAI Y (the letters on the left hand):



Comparing to other layouts with the wordlist and layout provided by Sc0tTy, here is the heatmap I get for the collected works of Mark Twain:


And the results of both sets of tests -- Shenafu's ver. 3 on the left, KLA on the right:


And here is the layout you can plop into any of the above keyboard layout analyzers:
 KFOU_DHAI_Y_Kinesis.txt (14.71 kB - downloaded 165 times.)

I have another one with a bunch of foreign characters mapped to AltGR, but I still need to see if I can improve on the high ASCII side. I might post it in the next few days for completeness, though.

[Sc0tTy]

That's an interesting layout!
That location of the right shift modifier is quite interesting, have you checked the scores when putting the left shift modifier in the same place?
Because I do think its on the wrong place currently for a key that's hit over 4%.

What's the reason for the Enter on the right pinky's second layer?
The CTRL is on a bad place though, what finger do you use for it, the middle?
Do you really use the middle and index finger to hit those delete and backspace keys?

I also assume that you have another layer modifier on the other side otherwise it would make hitting some of the keys on the same grid very hard.

I guess you win the most for having the E on the thumb cluster. This layout wouldn't work for me because my right thumb cluster needs a low key count.
Though that shift placement I might use that somehow, I need to update my dataset to represent the actual shift count, its way to low currently and it might effect eNNe.

You might need to check the Enter usage, because it doesn't show on the heatmap and that's very strange. You can see on my eNNe heatmap it should show up on at least my wordlist.

[Snarfangel]

That's an interesting layout!
That location of the right shift modifier is quite interesting, have you checked the scores when putting the left shift modifier in the same place?

Not really, I like to hit shift with one hand and the key I want shifted with the other.

Unless you mean put the left shift on the standard QWERTY location for "B"  ) -- then yes. And everywhere around there. The left thumb is where the left shift likes to be. I even looked into changing <left shift><space> to Enter, freeing up the right thumb for a shift key, but haven't found a way that the analyzers like.

Because I do think its on the wrong place currently for a key that's hit over 4%.

It does look wonky. I think it's mostly the asymmetry. But the stats got better when I moved from the normal shift position. Using my Kinesis Advantage2 layout, I first put it on the right pinky single quote-double quote key, then to the up-arrow under the right index finger, then the down arrow under the right forefinger (mirroring backspace on the left side), and then finally where N is on the standard Kinesis. Here is a picture, which also helps with the next question:

What's the reason for the Enter on the right pinky's second layer?
The CTRL is on a bad place though, what finger do you use for it, the middle?
Do you really use the middle and index finger to hit those delete and backspace keys?

The Enter key is part of the embedded keypad -- you just hit the button on the upper right with the blue letters. I just wanted to account for it, since I do use it when typing in long strings of numbers. I'm going to set up a set of non-standard characters there too -- with the AltGr key -- but the keyboard layout analyzer doesn't handle four characters plus numbers.

The Control keys work pretty well, actually. The are under the pinky, and I just have to curl my hand a bit. It's a little easier for me to hit than the normal shift keys, and quite a bit easier than the top of the thumb cluster, which is where Kinesis puts them.

I also assume that you have another layer modifier on the other side otherwise it would make hitting some of the keys on the same grid very hard.

I guess you win the most for having the E on the thumb cluster. This layout wouldn't work for me because my right thumb cluster needs a low key count.
Though that shift placement I might use that somehow, I need to update my dataset to represent the actual shift count, its way to low currently and it might effect eNNe.

You might need to check the Enter usage, because it doesn't show on the heatmap and that's very strange. You can see on my eNNe heatmap it should show up on at least my wordlist.

I'll check the Enter usage. That does look odd.

[Sc0tTy]

That's an interesting layout!
That location of the right shift modifier is quite interesting, have you checked the scores when putting the left shift modifier in the same place?

Not really, I like to hit shift with one hand and the key I want shifted with the other.

Unless you mean put the left shift on the standard QWERTY location for "B"  ) -- then yes. And everywhere around there. The left thumb is where the left shift likes to be. I even looked into changing <left shift><space> to Enter, freeing up the right thumb for a shift key, but haven't found a way that the analyzers like.

I guess the thumbcluster might be positioned differently and/or your hands are bigger. For me the thumb where your 'e' is placed is the only natural place for the thumb and thats only after doing the pinky mod.

What's the reason for the Enter on the right pinky's second layer?
The CTRL is on a bad place though, what finger do you use for it, the middle?
Do you really use the middle and index finger to hit those delete and backspace keys?

The Enter key is part of the embedded keypad -- you just hit the button on the upper right with the blue letters. I just wanted to account for it, since I do use it when typing in long strings of numbers. I'm going to set up a set of non-standard characters there too -- with the AltGr key -- but the keyboard layout analyzer doesn't handle four characters plus numbers.

Ahh the enter under the pinky makes sense for the numpad layer indeed.

The Control keys work pretty well, actually. The are under the pinky, and I just have to curl my hand a bit. It's a little easier for me to hit than the normal shift keys, and quite a bit easier than the top of the thumb cluster, which is where Kinesis puts them.

Looks like its a little to the right from where the CTRL usually is on a normal keyboard. And I already have a bad pinky from that position, your CTRL would probably be worse for me.

[Snarfangel]

I think the previous heat map was from my "Big bunch o' text" corpus I got by combining all of the inputs on the page (from "English: ANSI characters" to "Tech: Currency Modern..."). Not sure why it didn't show the return key. Here are our two layouts for comparison:


When I re-did Mark Twain's Project Gutenberg corpus with the High ASCII/Foreign character version (had to remove the number pad, which dropped the score a couple of points, even though it's still there), I got the following. The character count on the bottom of each picture is while I had the mouse over the "Enter" key:


No clue why some blank keys have color.

[Sc0tTy]

Yeah eNNe has been optimized for lower pinky usage so it wont get a top score on V2.

I guess there is something iffy going on with Ian's version of KLA. You can find V2 of Den's KLA here: http://shenafu.com/code/keyboard/Keyboard%20Layout%20Analyzer%202.html

I see you also use non-standard alphabet characters in your layout as well.
Dutch has some of those characters but they're not used much so I don't need to take that into account.

AltGr and Enter are not going to end up on those places but KLA gives me a massive penalty if I move them.
It doesn't understand that if I place it on the left and right side it will need to put half on each side

[Sc0tTy]

I updated my wordlist (attached) with the right shift modifier count. And it confirms my choice for putting it directly under the pinky:




What I noticed is that CTRL and SHIFT need to be placed quite close to each other because their combined usage is also quite high.
This is another reason for me not the put the shift under the right thumb hitting CTRL+SHIFT+LEFT (my most used combo, according to data) a pain.
But the main issue is that my right thumb can't handle the load and I also hold my pen there which is my input device. Plus the holding the thumb down like that is not its most natural motion: side-ways vs inwards.

I'm sad KLA can't handle CTRL, UP, DN, LEFT and RIGHT because it would show that eNNe should be even better in real world scenarios. Because now it looks like a lot of precious real estate is unused compared to other layouts.

Looks like I'm pretty much done with the main layer unless me, Ian or anybody else gets another light bulb moment. I'll go back to figuring out where to put the layer modifiers and edit my other layers

I do believe that modifiers should be on both sides of the board for their optimum usage and under fingers that move downwards and not sideways

[Snarfangel]

I noticed that a lot of keyboard layouts influenced by MTGAP software put the E and the O on the same finger, since they are both common letters that rarely occur together (at least semi-rarely). So I thought it might be interesting to put both in the left thumb cluster, and moving my shift to mirror the one on the right side.

It actually seems to have fairly significantly improved the score for the final layout. I still have a ton of tweaking to do, and it might feel weird when I try it out, but it's a lot of fun seeing an unexpected result like that. Of course, if you don't like E under the thumb, you probably don't want E and O under it, but I'm willing to give it a shot.

[vvp]

Isn't there a program which would find the best layout based on the given word list, the physical switch positions, and the weights of the optimized attributed (distance, same finger, same hand, ...)?

It does not look like too hard(*). It would save a tremendous amount of time for people who care about layout optimization ... not me, obviously, since I use only a slightly modified qwerty

(*) Well at least in the case the almost best solution is good enough. Finding global maximum may be more complicated.

[davkol]

There's about a dozen of such programs, but the weighs are entirely subjective.

[Sc0tTy]

I noticed that a lot of keyboard layouts influenced by MTGAP software put the E and the O on the same finger, since they are both common letters that rarely occur together (at least semi-rarely). So I thought it might be interesting to put both in the left thumb cluster, and moving my shift to mirror the one on the right side.

It actually seems to have fairly significantly improved the score for the final layout. I still have a ton of tweaking to do, and it might feel weird when I try it out, but it's a lot of fun seeing an unexpected result like that. Of course, if you don't like E under the thumb, you probably don't want E and O under it, but I'm willing to give it a shot.

Yeah it might sound like a good idea in theory but having the O so far away sounds like a bad idea.

OE combination is pretty common in Dutch actually and is one of the few things I'm not sure about for this layout.

[Sc0tTy]

I'm trying to do some updates to the layout but it seems http://shenafu.com has been down for a while.

Does anybody know what happend to it? Or if there is a mirror of his KLA ?

Found them: http://ieants.cc/code/keyboard/kla3 and http://ieants.cc/code/keyboard/klatest

I'm thinking of adding the auto shifting feature of the EZ to the layout, this should free up a lot of shift usage

[batfink]

If you're looking to test this on a variety of analyzers, there is another fork here, with well documented changes (I think with heavier emphasis on avoiding horizontal hand motions).

Would be interesting to see how the results compare.

If only there were an analyzer that could compare all the different analyzers!

[Snarfangel]

I actually used the keyboard layout analyzer at https://stevep99.github.io/keyboard-layout-analyzer/#/config and came up with the following Kinesis setup (I have an Advantage2):




It has really good stats for a Kinesis layout, and with this analyzer it's significantly better than my current layout. Since it's a couple of years since I learned my current layout, I'm about due for a change.  )

About the only thing I don't like the look of is the left and right shift keys aren't symmetric, but I really wanted backspace under my thumb (I use it a lot).

I'll attach the layout data file so others can import it and play with it if they like. I am probably weird, but I really like seeing the layouts other people come up with, and the design decisions that went into them.

 FinalBestKeyboard07252020.txt (15.68 kB - downloaded 110 times.)

[Sc0tTy]

So.... After discovering the Auto Shift feature of the ErgoDox EZ and testing it out a whole new world opened up to me.
I'm enormously convinced that, except for non-staggered layouts, it's the single best thing that has been invented in Ergo-Keyboardland.

Having XZFYQ on a secondary layer is not very efficient when having Auto Shift enabled so I'm changing the layout and moving them to the primary layer.
I then gained the insight in having most of the punctuations on the primary layer as wel as I hit those more than most of my shortcuts.
I also moved more keys to the left side of the board.
So I've come up with this layout (and it's looking quite final).

Sadly having no shifts in the KLA results in invalid reports but IMHO so I moved it to the thumb cluster for the most optimum result. Imho this still downgrades the result. I've kept shift/ctrl on the original spots in the actual layout thought because they are used a lot in during text editing operations and other tasks.

Most layouts i've seen never focus on those kinds of operations while being a mayor part of the usage: like the arrow key cluster usage.



https://configure.ergodox-ez.com/ergodox-ez/layouts/QaGnB/latest/0



I also updated the word list a bit to better reflect the 'J' usage in dutch and 'jij' trigrams:
Here is the new heatmap based on my latest WhatPulse data (over 5 years of data by now)


Hope to hear your thoughts!

Download for layout:  eNNe AS 78.38 (15.97 kB - downloaded 98 times.)
Updated wordlist:  Wordlist en Woordenlijst 2X TZJ.txt (339.29 kB - downloaded 137 times.)

[iandoug]

Re eNNE AS 78.38:

Error with eNNe AS 78.38.json : eNNe AS 78.38, duplicate character on keyboard: ; (59)
Error with eNNe AS 78.38.json : eNNe AS 78.38, duplicate character on keyboard: ! (33)
Error with eNNe AS 78.38.json : eNNe AS 78.38, duplicate character on keyboard: x (120)
Error with eNNe AS 78.38.json : eNNe AS 78.38, characters not on keyboard: X^&
Error with eNNe AS 78.38.json : eNNe AS 78.38, no Enter key assigned

Cheers, Ian

[Sc0tTy]

O indeed! Fixed now



Ian, what's the main Corpus you use for your layouts?
And what are your thoughts on the changes I've made?

[iandoug]

O indeed! Fixed now

Ian, what's the main Corpus you use for your layouts?
And what are your thoughts on the changes I've made?

1. more fixing needed :-)
Error with eNNe AS 83.86.en.json : eNNe AS 83.86, duplicate character on keyboard: ; (59)
Error with eNNe AS 83.86.en.json : eNNe AS 83.86, characters not on keyboard: ^

re the corpus, I did discuss it on Den's site a while back.. . basically it dawned on me that most if not all of the texts we were using were not good... they may be "real world" but their letter frequency does not match English (or Dutch, for that matter). That applies especially to the assorted word lists.

So I went searchabout .. if I found stuff online (news or blogposts) that looked like they might have correct frequency, I cleaned them up and ran them through a checker. Most failed.

Did the same with assorted old books on Gutenberg.

My checker tries to find the texts that most match English letter frequency. I was doing "similarity checks" to the list of characters in descending order, but existing similarity algos didn't work they way I needed them to, so I rolled my own. Ended up with three, slightly different.

Two check all characters, the other just checks the 11 most common letters (as in: your home row and space bar.... or somesuch).

At the moment I only have 8 texts that have the top 10 or 11 characters in the correct order (score 66 or 65 in Top11 column on spreadsheet) . I use this metric because letter distribution in a text is some sort of Zipf distribution curve, and the most used letters are the most important. The FreqMatch column is most important, lower == better. The Similarity column has higher== better.

I uploaded them to Den's site the other day, might as well put it here too. Note that most of them are from a blog by John Ward, it just so happens that he occasionally churns out pieces that have the right frequency distribution. They are probably copyrighted but I guess we can plead some sort of fair use case. Two are old books, and one an op-ed piece from Russia Today. Note I'm not concerned with what they say, just the letter frequency ..... :-)

Texts and analysis attached. The .csv is tab-delimited.

I don't know what to do about "programming" inputs... original idea was a mishmash of languages from RosettaCode. Should probably limit it to "popular" languages, but that's a moving target and the well known lists are heavily criticised. There's still tons of COBOL and Fortran being written/maintained but those programmers don't need to ask Google or StackOverflow for help....

So at the moment I'm just going with these English tests, and sometime KLE home page for "web dev stuff" ... but that overuses certain things like the letter k and doublequotes etc.

As for your layouts, will take a look. At the moment trying to do a round-up of recent new/updated layouts to do another round of testing. So if anyone reading this has a new layout, feel free to submit.
You can check most of what I already have here:
https://www.keyboard-design.com/internet-letter-layout-db.html

I have a bunch of others to add as well (and probably some others from Den)

May  6  2018  YPHINAFU.txt
May 11  2018  beakl9-ansi-shifted-mod.txt
Dec 10  2017 'eNNe KLA3 P 109.54.json.txt'
Jul 19  2018  phynnboi.txt
Feb 11  2019  quesnel.mtgap.beakl.ansi.txt
May  6  2018  snarfangel-UP_OIANY_4.txt
May  6  2018  spindle.txt
Aug 13 07:50  FinalBestKeyboard07252020.json
Aug 13 07:48  KFOU_DHAI_Y_Kinesis.json
Aug 13 07:31  Qwicker-KFLY.kla.json
Aug 13 07:31  Qwicker-Mod-H.kla.json
Aug 13 07:32  Qwickly-Mod-B.kla.json
Aug  9 22:31  astarte.en.ansi.json
Mar  2 09:05  balanced13-iso-shifted-final-mod-ian.json
Mar  2 07:58  balanced13-iso-shifted-final.json
Mar  2 07:57  balanced13-iso-shifted-no-spacefn.json
Mar  4 10:24  balanced13-iso-shifted-v2-mod-ian.json
Mar  4 09:34  balanced13-iso-shifted-v2.json
May 31 11:47  beakl-15.en.matrix.json
Feb 11  2019  beakl15ModPqIntl3.kla.json
May 31 22:28  beakl19.matrix.json
Jun  5 08:05  beaklArr29k1.matrix.json
Mar 11  2019  code.en.ansi.json
Aug 13 10:47 'eNNe AS 83.86.en.json'
Dec 12  2017 'eNNe KLA3 P 108.27.json'
Dec 10  2017 'eNNe KLA3 P 109.54.json'
Dec 10  2017 'eNNe KLA3 P 110.97.json'
Dec  7  2017 'eNNe KLA3 P 111.69.json'
Aug 12 23:21  hycis.en.ansi.json
Jun  9 15:00 'kla BEAKL-19 ergodox.json'
Jun  9 16:31 'kla X1 Atreus-Ergodox 44-keys.json'
Aug 10 23:13  ntsc.en.matrix.json
Aug 10 16:32  ough.en.ergolinear.json
Aug 11 09:44  ough.en.matrix.json
Aug 11 10:07  ougw.en.matrix.json
Dec 29  2017  power.en.ansi.json
Aug 12 21:33  shz.en.ansi.json
Aug 12 10:14  shz.en.matrix.json
Aug 12 13:38  shz.en.matrix.zkq-dblequote.json
Aug 13 08:42  vflm.en.ansi.json
Aug 11 09:43  wiea.en.matrix.json
Dec 30  2017  zx-1.en.ansi.json

Last present : ANSI (30-keys, not full optimization) version of my most recent layout ... scores well.

Cheers, Ian

[Sc0tTy]

Nice write up!
Yeah I looked at your site a couple of weeks back and didn't see much updates then. its a real nice resource.

One of my issues with all corpuses are that they don't represent a real world scenario. That blog writer probably did several edits performing several different text operations.
I use KLA to see if the layout performs okay on the corpusses but I use my type data as the main source of input.

For instance over 11% of all keys hit are: pgup, pgdn, home, end and the four arrow keys. That is quite a lot.

Sadly its not possible to detect the percentage of auto shifted characters, I'd really like to see the reduction size of the shift key and that finger usage.

And in that regard I updated the layout a bit for a better match in my heat map and alsy the weight for the ring finger is better on top than on the bottom. It did reduce the score a little:

[Snarfangel]

I just wanted to mention that "finalbestkeyboard" just means the best one I could come up with (plus, I was running out of names!   ) . I did reprogram my Kinesis and am using it now, though.

[iandoug]

Nice write up!
Yeah I looked at your site a couple of weeks back and didn't see much updates then. its a real nice resource.

Have been occupied analyzing Giza the last few years, but trying to spend some time daily on various sites now.

One of my issues with all corpuses are that they don't represent a real world scenario. That blog writer probably did several edits performing several different text operations.
I use KLA to see if the layout performs okay on the corpusses but I use my type data as the main source of input.

For instance over 11% of all keys hit are: pgup, pgdn, home, end and the four arrow keys. That is quite a lot.

Yeah. I'm actually busy with design for my second physical board, and first is STILL not done (fiddling with QMK at the moment, that's a whole nuther story...)

But I also use the Nav cluster a great deal (not to mention backspace ....) and putting those on a layer won't work for me. So trying to find something that works.

Two years ago (on Den's site) we discussed mini-joysticks or D-pads for the arrow keys or PgUp/PgDn/Home/End, but their build quality is sadly lacking.
Now Den himself is starting to think along those lines ... :-)

The problem I think is you often need to hold shift or ctrl when using the Nav cluster.  I think I should run a keystroke recorder and see what my actual usage is like, because most of those keys are done in "automatic mode" (ie not conscious, the fingers just move....) and we're really aware  of just how much we use the nav cluster.

Maybe tweak KLA to handle "key capture logs", at least on local versions (wouldn't want to do that on a remote site ...)

Cheers, Ian

[Sc0tTy]

Ooo physical keyboard?! A whole new design or based on existing designs. I'm looking at a Dactyl ManuForm 6x6 atm for my next keyboard.

Yeah Ctrl and Shift are used heavily with the Nav-cluster, here you can see my top 4 key combo's according to my WhatPulse data (though key combo tracking seems a bit buggy)
.

Its the main reason why I don't like having those modifiers in the thumb cluster but on the sides. Same for other key combinations. If you need to press 2 modifiers for a shortcut you're doomed on the thumb cluster.

You could start by recording your data with WhatPulse, that's done securely. Sadly it doesn't record the actual letters you've typed so you don't know the bi- and trigrams you're typing. But it does allow you to make heat maps with the type data in Excel.
For instance with the layout above this is the finger usage:

(pinky is a bit skewed due to shift key)

[Snarfangel]

(snipped)
As for your layouts, will take a look. At the moment trying to do a round-up of recent new/updated layouts to do another round of testing. So if anyone reading this has a new layout, feel free to submit.

You shouldn't have asked!

I used the keyboard layout analyzer here: https://ieants.cc/code/keyboard/klatest/?#/config

I like the results better than my previous one -- that one overloaded the pinkies with O and R.

I need to head to work, so I may post more later.



 YOP_UIAN 1.txt (15.67 kB - downloaded 110 times.)

[Snarfangel]

To add a bit to the previous post, when tweaking things I used a large corpus made up of the collected works of Mark Twain (and other works in Project Gutenberg), a bunch of programming snippets from the Rosetta Code repository, a nice collection of my own writing, plus a plethora of other stuff. It didn't quite match the etaoin srhldcum fpgwybvkxjqz that Peter Norvig (replicating Mark Mayzner) found, more like etaoni srhdlucm fwgpvbvkxjzq (i-n switched, etc). Still, it is almost 30 million characters and 516,085 lines of text.

AS a quick-and-dirty test, I replaced the QWERTY keyboard in the dozen default keyboards at https://ieants.cc/code/keyboard/klatest/?#/config and used my text corpus, and got:



Not really fair to non-kinesis / Ergodox keyboards, though I did some comparisons earlier. The layout fire for the KLA is provided in the previous post.

Heatmaps look like:


(I'm using the layout as I type, albeit a bit slowly.)