geekhack

geekhack Community => Ergonomics => Topic started by: Zustiur on Sun, 16 August 2015, 05:47:07

Title: Designing a new layout. Not very scientifically.
Post by: Zustiur on Sun, 16 August 2015, 05:47:07
I've been using Colemak on my ErgoDox for about 9 months. Prior to that I'd never left QWERTY. My reading this weekend has taken me to a number of topic all of which lead me to be curios. I finally looked up the alternative layouts (including odd ones like BU-Teck) and read the reasonably convincing logic behind the workman layout.

However, all of these layouts seem to assume the standard keyboard. Workman at least provides a weighting diagram for grid/matrix boards but assumes all peripheral keys will remain the same. When I noticed that this continued to give more preference to comma than to full stop I immediately felt that was a mistake. I then wondered about frequency of punctuation in English and felt someone should at least attempt to intermingle the keys according to frequency. Even if it's purely an intellectual excercise/thought experiment that never makes it into an actual board.

That lead me to these wiki articles:
https://en.wikipedia.org/wiki/Letter_frequency
https://en.wikipedia.org/wiki/Punctuation_of_English

I combined their findings with a few assumptions to produce this combined list:
RankKeyKeystrokes/1000
0Space130
1E127.02
2T90.56
3A81.67
4O75.07
5I69.66
6N67.49
7.65.3
8S63.27
9,61.6
10H60.94
11R59.87
12'"51
13D42.53
14L40.25
15Shift33
16Shift33
17C27.82
18U27.58
19M24.06
20W23.61
21F22.28
22G20.15
23Y19.74
24P19.29
25-15.3
26B14.92
27V9.78
28K7.72
29;:6.6
30!3.3
31J1.53
32X1.5
33Q0.95
34Z0.74

My assumptions were:
Determining the layout, I started with workman's weighting for grid boards:
4223443224
1.5111331111.5
4432442344

Then broke it down into much finer detail based purely on my own preference:
26141220302919111325
3286421817135731
34242216102827915212333

Then let excel place the keys for me:
BL'"W!;:MRD-
XSNOTUCEAI.J
ZPGShiftHKV,ShiftFYQ

For pure typing, that's probably quite good. However, we are using computers, not typing on electronic typewriters. Therefore additional elements should be taken into account. For my second version there are some additional assumptions:
In most cases this just shifts the priority of a given key by one place.

The resulting matrix is:
PL'"W;:KMRDV
!SNOTUCEAI.X
QBFShiftHZ-,ShiftYGJ

This still doesn't account for frequency of Enter, Backspace, Ctrl, Alt, Win, App, Tab, Backspace, ? and various other keys. Nor does it do anything about avoiding common same finger bigrams. However, it looks interesting enough to make me chuck it into KLE (http://www.keyboard-layout-editor.com/#/). A board built with this layout might look something like this:
(http://i135.photobucket.com/albums/q128/Zustiur/GeekHack/new%20layout.png) (http://s135.photobucket.com/user/Zustiur/media/GeekHack/new%20layout.png.html)
More
["!\n1","@\n2","#\n3","$\n4","%\n5","^\n6",{w:2},"Escape","&\n7","*\n8","(\n9",")\n0","_\n-","+\n="],
["?\n/","P","L","\"\n'","W",":\n;",{w:2},"App","K","M","R","D","V","|\n\\"],
["J",{c:"#65af4c"},"S","N","O","T",{c:"#cccccc"},"U",{w:2},"Tab","C",{c:"#65af4c"},"E","A","I",">\n.",{c:"#cccccc"},"X"],
["O","B","F","Shift","H","Z",{w:2},"Win","_\n-","<\n,","Shift","Y","G","Q"],
["Ctrl","Alt","Enter","Del",{w:2},"Space",{w:2},"BackSpace",{w:2},"Space","Del","Enter","Alt","Ctrl"]

I can't help but call this the SNOT layout thanks to that word being the home keys for the left hand.


I'm extremely interested to get other peoples views on this layout, on the steps I took to generate it etc.
I would also love to be pointed to an analyzer that will calculate the bigrams and same-finger-units for me. The workman designer must have used one but I cannot find a link.
Title: Re: Designing a new layout. Not very scientifically.
Post by: Pro XKB on Sun, 16 August 2015, 13:26:28
Quote
finally looked up the alternative layouts (including odd ones like BU-Teck) and read the reasonably convincing logic behind the workman layout.

However, all of these layouts seem to assume the standard keyboard.
BU-Teck was explicitly made for the "Truly ergonomic keyboard", not a standard keyboard.

Quote
I would also love to be pointed to an analyzer that will calculate the bigrams and same-finger-units for me.

You could use the AdNW tool from here:
 
  https://sites.google.com/site/ausderneowelt

It also comes with frequency files, including punctuation, space, upper and lower case letters separate.


Title: Re: Designing a new layout. Not very scientifically.
Post by: hoggy on Sun, 16 August 2015, 14:47:10
There's a scale invariant law about the frequency of numbers that after some assumptions, one is more frequent than two, three and so on. Can't think of the name of the law, but it could be used to detect made up numbers in financial accounts.

From memory (as good as it's proving to be), didn't programmer dvorak attempt to optimise number placements?
Title: Re: Designing a new layout. Not very scientifically.
Post by: stevep on Mon, 17 August 2015, 06:13:53
There's a scale invariant law about the frequency of numbers that after some assumptions, one is more frequent than two, three and so on. Can't think of the name of the law..

Sounds like you mean Benford's Law (https://en.wikipedia.org/wiki/Benford's_law).
Title: Re: Designing a new layout. Not very scientifically.
Post by: Zustiur on Mon, 17 August 2015, 06:56:46
BU-Teck was explicitly made for the "Truly ergonomic keyboard", not a standard keyboard.
That is correct. I didn't mean to imply otherwise. Oops.

Quote
You could use the AdNW tool from here:
 
  https://sites.google.com/site/ausderneowelt

It also comes with frequency files, including punctuation, space, upper and lower case letters separate.
Thanks. I had a look at the documentation but I'm too frazzled to make sense of it today. I'll try again later.
Title: Re: Designing a new layout. Not very scientifically.
Post by: jacobolus on Mon, 17 August 2015, 07:59:51
This is a reasonable start, but do note that what you care about ultimately is the ease of typing groups of 3–4 letters, not only single letters.

Also, you may need to sometimes type multiple-modifier shortcuts, some of which I suspect will end up quite awkward. (Though they’re also very awkward on standard keyboards.)

Are you planning to make a keyboard in a grid like that, or is that just an abstract representation? Some of the keys there aren’t going to be all that reachable, and the thumbs are pretty under-used.
Title: Re: Designing a new layout. Not very scientifically.
Post by: lydell on Mon, 17 August 2015, 15:50:00
Hi, Zustiur. I’ve created my own layout, anishtro (https://github.com/lydell/anishtro/), so I’ve got some experience to share.

I've been using Colemak on my ErgoDox for about 9 months. [...]

However, all of these layouts seem to assume the standard keyboard.

Yes, most layouts are made for standard keyboards. Creating a layout is not a one-size-fits all thing in two ways. A layout does not fit every person, because we all have different opinions and think different things are comfortable. A layout does not either fit every keyboard, because when creating a layout you must keep the physical constraints of your keyboard in mind.

Workman at least provides a weighting diagram for grid/matrix boards but assumes all peripheral keys will remain the same. When I noticed that this continued to give more preference to comma than to full stop I immediately felt that was a mistake.

That means that you and the author of Workman disagree. If so, Workman is not a good choice for you.

Even if it's purely an intellectual excercise/thought experiment that never makes it into an actual board.

If you think it is fun then by all means go for it! Even if you will be the only one using the end result it doesn’t matter as long as you like it and had fun along the way. This is the case for me: I think I might be the only one in the world using the anishtro layout (but then you should keep in mind that I didn’t even share it on geekhack until yesterday, even though I’ve used the layout for one and a half years).

I then wondered about frequency of punctuation in English and felt someone should at least attempt to intermingle the keys according to frequency.

That lead me to these wiki articles:
https://en.wikipedia.org/wiki/Letter_frequency
https://en.wikipedia.org/wiki/Punctuation_of_English

I also used those Wikipedia articles in the beginning, but I find them lacking. They’re not enough to do research on. Instead, I recommend getting a large text corpus that you think is representative and calculate frequencies from that yourself. That’s what I did: https://github.com/lydell/anishtro/blob/master/how.md#getting-data (https://github.com/lydell/anishtro/blob/master/how.md#getting-data) (also see the following section).

I combined their findings with a few assumptions to produce this combined list:

I also tried to think logically and make assumptions in the beginning. After I had gotten real data I realized how bad I was making assumptions. Mostly they were wrong.

I gave preference to the right hand over the left because I use a left handed mouse. Right handers should flip the whole layout :P
[...]
Determining the layout, I started with workman's weighting for grid boards:
[...]
Then broke it down into much finer detail based purely on my own preference:

That’s the way to go! Nail down the things you like and optimize for them!

Then let excel place the keys for me:

I’d be very interested in how you managed to get a spreadsheet program to place the keys for you.

  • We should attempt to account for the increased frequency of v, c, x, z, b, I and y due to paste, copy, cut, undo, bold, itatlic and redo respectively.
  • I assigned an increase of 1-10 per 1000 for each of these keys. Unfortunately I have no data on the frequency of these so this is gut feel
[...]
This still doesn't account for frequency of Enter, Backspace, Ctrl, Alt, Win, App, Tab, Backspace, ? and various other keys.

My gut feel often proved to be wrong when it came to saying how common certain letters are. If you really want to take all those things in consideration I’d recommend running a key logger for a month or so and then analyze the results.

Nor does it do anything about avoiding common same finger bigrams.

For me, that’s one of the most important characteristics of a keyboard layout, because that’s what bothered me the most in QWERTY. If I were you, I’d look into minimizing it. You have a few pairs I’d dislike, such as ME, TH, AR and CK. On the other hand you’ve also put some uncommon combinations on the same finger, which is good, such as LN and IY. You’ve also “protected” many vowels with punctuation characters, other vowels and uncommon letters (such as Z), which is wise since vowels are most likely to be part of common letter pairs.

I can't help but call this the SNOT layout thanks to that word being the home keys for the left hand.

Naming layouts after sequences of letters in them, especially on the home row, is common.

I would also love to be pointed to an analyzer that will calculate the bigrams and same-finger-units for me.

In my opinion it does not make much sense to use somebody else’s analyzer, because you will most likely have different opinions and find that the tool analyses things you aren’t interested in. It is better to create your own tools (but you could of course base it on your own). I’ve included the tools I used for anishtro (https://github.com/lydell/anishtro/#tools). Just for the fun of it, here’s how your layout scores in my analyzer (which uses my opinions, not yours):

Code: [Select]
{ sameFinger:
   [ [ 'ht', 3.1143912946997787 ],
     [ 'ar', 2.099149674202991 ],
     [ 'em', 1.1139997279205736 ],
     [ 'ce', 1.043975483884984 ],
     [ 'di', 0.7793704826926966 ],
     [ 'tu', 0.5794360714213668 ],
     [ 'hw', 0.25737627498436577 ],
     [ 'ek', 0.2437593721275835 ],
     [ 'ps', 0.23336242065858737 ],
     [ 'ck', 0.169687558676825 ],
     [ 'fl', 0.10791363185170741 ],
     [ 'tw', 0.09908139571573184 ],
     [ 'hu', 0.09840895606848332 ],
     [ 'bs', 0.07113635575988525 ],
     [ 'ln', 0.06022214302377488 ],
     [ 'dy', 0.05717030154780089 ],
     [ 'fn', 0.05559265468310247 ],
     [ 'iy', 0.032070198561082605 ],
     [ 'cm', 0.019332639858394556 ],
     [ 'km', 0.01761274614523972 ],
     [ 'tz', 0.01516868665812496 ],
     [ 'uz', 0.014560904669265733 ],
     [ 'bj', 0.010202978493828297 ],
     [ 'hz', 0.006595081155706504 ],
     [ 'bp', 0.004254473922014588 ],
     [ 'uw', 0.0035949658064439375 ],
     [ 'js', 0.0020302504734233747 ],
     [ 'gv', 0.0013448792944970125 ],
     [ 'jp', 0.0013319477628191568 ],
     [ 'vx', 0.000827618027382777 ],
     [ 'wz', 0.0008017549640270651 ],
     [ 'gx', 0.00034915135530210904 ],
     [ 'gq', 0.00032328829194639726 ],
     [ 'qx', 0.0003103567602685414 ],
     [ 'qv', 0.00023276757020140602 ] ],
  rolls:
   [ [ 'no', 2.1511473630796494 ],
     [ 'ot', 1.3226111969477448 ],
     [ 'ai', 0.7833404629177985 ],
     [ 'ae', 0.7195621486826131 ],
     [ 'ns', 0.49520007407181343 ],
     [ 'dr', 0.34318991919861747 ],
     [ 'mr', 0.1994430130675714 ] ],
  sameFingerTotal: 10.314978489690207,
  rollsTotal: 6.014494177965807 }

(You can compare with QWERTY which gets 7.78% same finger usage and 6.67% rolls, or anishtro which gets 0.89% same finger usage and 14.65% rolls. So for me, your layout would suck ;) But remember that we all have different opinions, which might make it rock for you.)

It looks like that layout image you posted does not really correspond to the text versions you wrote, and it looks like some letters are duplicated in the image.
Title: Re: Designing a new layout. Not very scientifically.
Post by: Zustiur on Wed, 19 August 2015, 00:24:14
Are you planning to make a keyboard in a grid like that, or is that just an abstract representation? Some of the keys there aren’t going to be all that reachable, and the thumbs are pretty under-used.
That is just an abstract. See my making stuff together thread for my actual keyboard shape designs. This thread is more about theory and curiosity. It's a speculation on what could have been the standard if qwerty wasn't so dominant at the beginning of computing.
Title: Re: Designing a new layout. Not very scientifically.
Post by: Zustiur on Wed, 19 August 2015, 00:36:07
Thanks for the input lydell. I feel that the only 'true' way to design a keyboard layout is to spend a lot of time with keyloggers on your machines. To create a layout for the masses would require logging the activity and usage of many people.
Sadly installing a logger at my workplace is out of the question.

The trouble with using a corpus is that it won't respect everyday computer usage.
Title: Re: Designing a new layout. Not very scientifically.
Post by: Zustiur on Sat, 22 August 2015, 23:10:28
Then let excel place the keys for me:

I’d be very interested in how you managed to get a spreadsheet program to place the keys for you.

It's pretty easy if you're familiar with the vlookup function.
I essentially had 3 tables:
The first table was 3 columns: rank, key, keystrokes/1000
The second table was laid out like a keyboard with each key position given a unique number to represent the key's rank. i.e. key 1 is right hand index finger home position.
The third table is shaped like the second but contains vlookup formulae like this:
=VLOOKUP(M49,$A$43:$B$77,2,FALSE)
Where M49 is the corresponding position in the table 2 and A43:B77 is table 1.
So in English the formula says "the value of THIS cell equals position number M49 (which is key rank 1) in the letter ranking table (which is the letter t). Therefore, THIS cell equals T"

In a semi related matter, this article on the creation of QWERTY was really interesting:
http://www.strath.ac.uk/media/departments/economics/researchdiscussionpapers/2013/13-24FINAL.pdf

Quote
It looks like that layout image you posted does not really correspond to the text versions you wrote, and it looks like some letters are duplicated in the image.
Well spotted. ! should be J and O is repeated because I allowed for 34 finger positions but there are only 33 characters in my table. Excel produced a 0 to indicate a blank cell and I misinterpreted it as O. I must have written up the section of my post before I discovered that I'd left ! in the table by accident. The picture is more accurate of my intended design.

I haven't taken the time to figure out how to use your tool, but it's interesting that I can fix 3 of the top 4 cases by swapping keys 1 and 2 (T and E). I wonder if that would cause a rise in other bigrams. Swapping keys 13 and 14 may also help (D, L), or possibly hinder. I'm not sure where to go after that as I've only come up with an algorithm for placing characters according to frequency. I cannot think of a non-trial-and-error method to account for same finger units etc.
Title: Re: Designing a new layout. Not very scientifically.
Post by: lydell on Sun, 23 August 2015, 00:25:19
How did you create table one? This is how I understand it:


If the above is right, and I get the spreadsheet logic right, do you mean that it works like this: For the highest rated key, choose the most frequent letter, for the next highest key, choose the next most frequent letter, and so on? That’s actually what I tried to do initially, but I quickly realized that this led to many same finger usages and missed roll opportunities (both of which were very important to me).
Title: Re: Designing a new layout. Not very scientifically.
Post by: Snarfangel on Sun, 23 August 2015, 08:44:06
In a semi related matter, this article on the creation of QWERTY was really interesting:
http://www.strath.ac.uk/media/departments/economics/researchdiscussionpapers/2013/13-24FINAL.pdf


That *is* interesting. Fun read. It would be interesting to see the set of keyboards with "Sholes Optimality" (or "Jam minimization," or whatever you want to call it), and then see which one scores best with modern criteria -- common keys on home row, rolls preferred, that kind of stuff. It could tell us the keyboard we *might* have had the luck to use rather than QWERTY. :)

[attach=1][attach=2]

It's interesting, too, that he put all of the vowels on one hand. *So* curious about that now.  :))
Title: Re: Designing a new layout. Not very scientifically.
Post by: Zustiur on Mon, 24 August 2015, 23:40:29
How did you create table one? This is how I understand it:

  • Create a column of all ranks, sorted from the best to the worst.
  • Create a table with two columns, key and corresponding keystrokes, sorted by keystrokes from the most to the least.
  • Move column of ranks into the table, as the first column.

If the above is right, and I get the spreadsheet logic right, do you mean that it works like this: For the highest rated key, choose the most frequent letter, for the next highest key, choose the next most frequent letter, and so on? That’s actually what I tried to do initially, but I quickly realized that this led to many same finger usages and missed roll opportunities (both of which were very important to me).
Close but not quite.
Table 2 and table 3 were both shaped like a keyboard alpha section. 3 rows of 12 keys. I tried to show it in forum code in the first post but it doesn't display well.
I can post pictures of the spreadsheet if people want me to.
Title: Re: Designing a new layout. Not very scientifically.
Post by: lydell on Wed, 26 August 2015, 11:23:10
I _did_ understand table 2 and 3, I was asking about how you created table 1, and how the key placement worked at a higher level :)

This is how I understand table 1 one was created (note that this says nothing about table 2 and 3):

  • Create a column of all ranks, sorted from the best to the worst.
  • Create a table with two columns, key and corresponding keystrokes, sorted by keystrokes from the most to the least.
  • Move column of ranks into the table, as the first column.

This is how I understand the key placement at a higher level:

do you mean that it works like this: For the highest rated key, choose the most frequent letter, for the next highest key, choose the next most frequent letter, and so on?
Title: Re: Designing a new layout. Not very scientifically.
Post by: Zustiur on Fri, 28 August 2015, 00:01:39
Oh sorry. Misreading on my part there.
Yes exactly.
In practice table 1 is a table with two columns. Rank and key. Yes.
It was built from the data on Wikipedia so it also contains a column for letter/symbol frequency as a number out of 1000.

The discrepancy detected in my diagrams earlier was because I forgot to remove ! from the list Aug first.