Hi everybody, I hope my humble tutorial will be useful... But English is not my native tongue so excuse me if any errors can be found...
Defining our own Keyboard Layout
I have two different kinds of hobbies: one is writing books, the other one is programming. Unfortunately both requires different kinds of keyboard layouts. Additionally, I prefer wireless keyboards, which have touch-pads instead of numerical keypads. Although I like to have a touch-pad on my keyboard, this way I haven’t got enough keys. The available space for keys is not even enough for the usual characters, not to mention that we’d also need a couple of modifier keys. With the help of modifier keys and the software, named xbindkeys we can make shortcuts to often used programs or functions.
What makes it even harder, is that my native language is Hungarian, and our alphabet includes plenty of accented letters. When I work on a program, I use English keyboard layout, but often I need to include Hungarian text as comment, or something, so I have to switch to Hungarian, because English layout doesn’t include accented letters at all. Additionally, mathematical operation keys and punctuation mark keys are being laid out differently in both.
Gave me a lot of headaches to find a solution, I had no idea how to deal with it. My keys are marked with English letters. This is not bothering me when I write, because I already know it by heart which are the keys for the accented Hungarian letters. But it’s difficult when I program, because when my keyboard layout switched to Hungarian, special characters, which I almost never need for writing, but often use in programming, are hard to find, as the keys are marked according to English layout. (for example: []{}#$ ). I guess I’m not the only one who came across discrepancies between functional and visual layouts; whoever works on their computers in different language environments and use it for completely different purposes must run into the same problems.
Solution:
We must define our own keyboard layout. This is troublesome from two aspects: one is that it’s awfully difficult to find out how X interprets keyboard-operations. What does it do and why? I had to set myself to it many times before I started to see the light. Actually, it still has kept a few mystical hokum from me in the dark. Basically the whole thing is a confusing fuzzy mess. What it does is nothing much than sending a code to the machine each time a key is being pressed, which will be indexed by X multiple times based on different types of data tables.
And the other one is that designing an adequate keyboard layout is not an easy job.
Let me describe step-by-step my conception and the reasons behind each steps:
We have to start with the most important thing; defining the modifier keys. We can find plenty of articles in the internet on this subject, but PRACTICALLY ALL OF THEM NOT ONLY INCOMPLETE, BUT ALSO WRONG.
The situation is that modifier keys can be varied in many different ways, and basically any key can work as a modifier key, plus many keys may have the same name. There are a few really strange names, which is only the same by sight as the other modifier keys, but in reality they have exceptional role. Actually under the expression modifier key we mean two completely different things, and they are often merged together:
1. The modifier keys, which define what kind of character code will be generated by X out of the code sent by the keyboard.
2. The other type of modifier keys are those we call modifier, because pressing just the key itself will not result in displaying any character. Even if it’s being held while another key is struck the character remains the same, but the state of the modifier key can be queried by different programs, and based on the answer they can do something.
If the above explanation seems to be somewhat diffuse, I can tell you that reality is even worse, because in X, it’s been decided based on xmodmap tables which keys should function as what type of modifier keys...
This is hard to understand without a sample, so let’s see one:
First of all, we need to choose one of the existing layouts as a starting point. There are two different kinds; one is the Hungarian in my case, for everybody else is their own mother tongue of course, the other one is English. I chose the Hungarian to start with; I set my machine by the command
setxkbmap hu
then, I saved this layout in a file, like that:
xmodmap -pke > modmap
Our job with it will be the following:
1. We need to edit the modmap file accordingly.
2. An executable file needs to be created for modifier keys, which will run every time X is being booted.
3. To support point 2, and to acknowledge the new modmap file the $HOME/.xinitrc file needs to be edited
As I already mentioned, first of all we need to be clear on the role of modifier keys. If we take a look at into the modmap file, we will see lines like that:
keycode 17 = 8 parenleft multiply multiply asterisk asterisk dead_abovedot
If you see something different in your code in the line that starts with “keycode 17”, don’t worry. The point is that each row starts with the word "keycode" followed by a number. The number is what the keyboard sends to the machine when we press a key. With this explanation we only scratch the surface; there are different kinds of tricky programs designed to detect the difference between the states of the key, whether it’s being held down or released already, etc. But I don’t even want to get started on it now...
So far it’s clear that each line belongs to a keycode, but the number of parameters differ. Parameters are always separated by space, so parameters can’t contain spaces. A parameter is always a well known character, or a one digit number, or some strange name.
One line may contain maximum 8 different parameters following the equal sign, but usually it’s less then that. Often a parameter with the same name will appear more than one places; even in the same row or in another lines.
So, as I said, there can be max. 8 fields in a row, that is 8 parameters. Each fields represent the adequate code that will be paired with the line belonging to the key if the right modifier key is simultaneously held down. These fields, or parameter positions mean the following:
Field 1 (this is of course the first field in the left, following the equal sign, because fields are numbered from left to right): this is the place for the code, which is generated when a single key without any modifier key is being pressed.
Field 2: here comes the code that is generated when a key is struck while SHIFT is held down.
Field 3: here comes the code that is generated when a key is struck while Mode_switch is held down. Haven’t heard of this key before? Never mind, I will shed light on it later!
Field 4: here comes the code that is generated when a key is struck while Mode_switch and SHIFT are held down together.
Field 5: this code is generated when a key is struck while the key named ISO_Level3_Shift is held down. (Twisted name for a key, isn’t it?)
Field 6: the place for the code generated when a key is struck while ISO_Level3_Shift and SHIFT are held down together.
Unfortunately I couldn’t figure out how the code is generated for Field 7 and 8. I kept trying to fetch the code for hours, but the most I could manage even with extremely special adjustments was to get a hold on code Field 8. But nothing on Field 7. Additionally when I got the code for Field 8, it has changed the path for all the previous fields. From that point those could only be reached by different modifier keys. There is a certain setting though, that makes possible to reach the code of Fields 1, 2, 7 and 8, but not the ones in between, 3, 4, 5 and 6. Great...
So I decided that 6 fields should be enough, actually it makes possible plenty of variations. Please note, that probably the reason why I only was partially successful is not that I’m an idiot; actually, if I understood correctly, it’s only guaranteed to reach the code of the first four fields on most hardware and implementations. Many times the code of Fields 5 and 6 are available, because in prefabricated xmodmap files the first two fields are practically the same as the second two, in case the crackbrained user would mix SHIFT with the other modifier key. So the rarely used characters will go to Field 5 and 6, because this two fields will be available in most hardware and software. Most, but not all, and believe me, it really sucks, when you need let’s say an accented "i", but there is no way to display it, because it’s usually is hidden under "AltGr-J".
The seventh and eight fields are absolutely not guaranteed, we’d better not to count on them, as they didn’t even exist.
So the first 6 fields should do it. As we can see, there are three mysterious keys that need to be defined; the SHIFT, the ISO_Level3_Shift and the Mode_shift.
In case of the latter two defining them means, to write their strange names into a table, assigning them to the right keycode line. They can be assigned to more then one keys, in that case the given function can be reached many different ways. In case of SHIFT this is more difficult, we can’t just write SHIFT in the code, because there are two different kinds: Shift_L and Shift_R, which logically means left and right. But this fact of course could be changed, if you wanted to.
It’s important to know how to detect the code that belongs to each of your keys. For that, you need the program named xbindkeys. It must be run from command-line, like that:
xbindkeys -k
a white pop-up window will appear, and you will have to press the key, which code’s you wish to detect. You will see a message similar to this:
Press combination of keys or/and click under the window.
You can use one of the two lines after "NoCommand"
in $HOME/.xbindkeysrc to bind a key.
"NoCommand"
m:0x2000 + c:108
Mode_switch
When I pressed down ALT on the right hand side, namely AltGr the above message has been generated. What’s interesting for you from this message is only the part "c:108", which means that the code for this key should go to line
keycode 108 =
This key will become the modifier key named Mode_switch, if we write the word "Mode_switch" in the appropriate keycode line, at least in the first field of it. It’s a must to write it in the first field, but honestly, I’m not sure whether it made any difference in the other fields. Who knows how the system interprets when a modifier key pressed down together with another modifier key? So either we write it only in the first field, or each of the fields. Important to know, that it’s optional whether how many of the eight fields we fill in. The ones towards the end might be left out.
I have to admit that I don’t consider myself the most resourceful or smartest person when it comes to the dilemma whether we should write a modifier key’s name in more than one fields. I’ve been experimenting with that key-problem for days, and I ran into such shockingly idiotic solutions from the part of our dear Xorg, that from now on I’m willing to give credit to all the vile things I’ve heard about them. I’m afraid this subject is pretty much hardware and implementation dependent. (I have found hints in different kinds of descriptions to back this up) What I mean is, that it makes a difference what kind of manufacturer produced your keyboard, which version of Xorg are you using, and what is in the xorg.conf file of xorg when it boots, if it has one at all in the given distribution. The information you provide on the keyboard matters as well; you say for example, it’s a pc104, or if its a hu, than how many keys it has, and which variant, etc. This things can cause a very nasty mess. So, let me share how I did it, how it worked out for me, but there is no guarantee it’ll work the same way for you, some experimentation might be necessary on your part. My description is based on the experiment I carried out on a Logitech K400 wireless keyboard.
Once we have finished with that key, Mode_switch, modifier key ISO_Level3_Shift needs to be assigned to a key, than Shift_L and Shift_R. as well. One of the two shifts can be eliminated, if we decide on having just one.
Than we are supposed to define similarly keys named Control_L and Control_R. (these are the ones, that usually marked with "Ctrl" on your keyboard)
The names in the table can be divided into groups, so symbols can be named the same. Practically it works like this:
xmodmap
This line of command will come up with the following code:
xmodmap: up to 3 keys per modifier, (keycodes in parentheses):
shift Shift_L (0x32)
lock
control Control_L (0x25)
mod1 Alt_L (0x40), Alt_L (0xcc)
mod2 Hyper_L (0x4d), Hyper_L (0xb4), Hyper_L (0xcf)
mod3 Super_R (0x3e), Super_R (0x86)
mod4 Super_L (0x85), Super_L (0xce)
mod5 ISO_Level3_Shift (0x42), ISO_Level3_Shift (0x5c)
In my list above the symbols, which considered to be modifier keys are listed in the first column, and followed by the character code from the corresponding keycode line. For example I have a modifier key named "Control", for which belongs the code of Control_L from the keycode table, and it’s value is indicated in parentheses (0x25). This is a hexadecimal number, which value is 2*16+5, that is 32+5, that is 37. What it means, is that key Control_L has been defined in line 37 of keycode table, so line keycode 37 will control the operation of this button.
Here, I included this only one key in the group named "control". I could have named more than one keys "control", but I only need one. The reason why, will be reviewed later on. There are other different groups, like shift, lock, mod1, mod2, mod3, mod4, mod5. Unfortunately more than that is not possible, and the group names are fix. I tried to be tricky and define a new group named mod6, but the system has shown severe reactions against it. What was I thinking?!
Once we have our xmodmap table filled in with all the lines that starts with keycode followed by the plenty of smart things we daydreamed of, then we can define those groups. For that, we need a script, which we can stylishly call "modkeys", and in our case it will look like that:
#!/bin/bash
xmodmap -e 'clear shift'
xmodmap -e 'clear Lock'
xmodmap -e 'clear control'
xmodmap -e 'clear mod1'
xmodmap -e 'clear mod2'
xmodmap -e 'clear mod3'
xmodmap -e 'clear mod4'
xmodmap -e 'clear mod5'
xmodmap -e 'add shift = Shift_L'
xmodmap -e 'add control = Control_L'
xmodmap -e 'add mod1 = Alt_L'
xmodmap -e 'add mod2 = Hyper_L'
xmodmap -e 'add mod3 = Super_R'
xmodmap -e 'add mod4 = Super_L'
xmodmap -e 'add mod5 = ISO_Level3_Shift'
It does the following:
The lines including the text "clear" will clear the previous definitions of the group, if there’s any. Than with lines including "add" we can define which names of keys belong to what groups. As we could write the same name to many lines in the keycode table, this script may gather many keys by one name, let’s say Super_L in our case, into one group, which is now mod4.
Certain programs might suppose, that conventionally there are modifier keys named shift and control in systems, and mod1 as well, which is actually nothing but key Alt.
Those who paid attention so far may raise the question, why it’s good for me to have only one Shift and Ctrl, why do I give up on the rest?
Well, it’s because I don’t have enough keys on my keyboard. I can’t afford to make many different keys to serve the same software function. This would be senseless, unnecessary waste, as I’m right-handed, and use the left shift and ctrl only. So those keys on the right side can serve some better purpose.
The next modifier keys are working in my system right now:
shift = this is the left-hand-side shift in my system, it’s function is the usual.
control = this is the left-hand-side Ctrl in my system, it’s function is the usual.
mod1 = this is the left-hand-side Alt in my system, it’s function is the usual.
These 3 grous above better be left alone, because programs are counting on them strongly. So all I did was to remove their right side pairs from the group.
Now, here come the specialities! I also have the following groups:
mod2 = it’s included in the keycode table under name Hyper_L and assigned to the key located in the upper part of my wireless keyboard marked with a home icon. This is of course not the same as what we mean under “Home” usually, actually has nothing to do with it.
mod3 = Super_R — this one is assigned to right-hand-side shift in my system
mod4 = Super_L — that’s the left-hand-side Windows® button (makes me nauseous even to look at it). Fortunately I have found a sticker on Ebay in the right size to cover it, which is made of some kind of durable aluminum-like material and portrays a penguin… it was pretty expensive, but worth every penny, because my machine is no longer in danger of being thrown up on, and also because I’m not willing to let my keyboard to be Micro$oft’s free billboard, especially because I don’t even use their operating system!
mod5 = ISO_Level3_Shift — In my system that’s the one to assigned to CapsLock...
Mode_switch = that’s right-hand-side Alt, or in another word AltGr.
Although there is a group called „lock”, I haven’t assigned anything to it; I hate if capitals are coming out endlessly, just because I pressed a key.
Now please take a look at my layout, and see how excellent it is! I’m full of usable, fluid modifier keys. Programs only occupy shift, ctrl and alt, rarely counting on key Windows®/Penguin. The rest is up for my own purposes. But this is not, what makes this layout so genial; it is the fact, what we may have not noticed at first sight, that Mode_switch and ISO_Level3_Shift are on completely different keys!
Most of the times in original xmodmap files the third and fourth fields are only the replications of the first and second. In the fifth and sixth fields there are characters that usually generated by holding down AltGr (right-hand-side-Alt). This is interesting and sad at the same time, because as we discussed it earlier, these two columns are for characters to be reached by holding down ISO_Level3_Shift. Meaning, that the makers of usual keyboard layouts are mixing together symbols ISO_Level3_Shift and Mode_switch, so practically Mode_switch will do ISO_Level3_Shift’s job. (Or maybe not, who the hell knows? Obviously there are plenty of tricks available to accomplish it.) The point is that in spite of that seemingly they use 6 columns, in reality they only use 4, because the first two are the same as the third and fourth, which is such a waste.
In order to change it, I had to separate the two symbols, Mode_switch and ISO_Level3_Shift. With this of course I ran the risk of getting into trouble, if I bumped into some strange hardware settings where there were no possibilities for fifth and sixth columns. But the chance of it low and in the worst case I can get back to standard settings.
But before I get too high, X jerks me back to reality. I have to realize that even in Linux environment all that glistens is not gold; my hands are tied when it comes to graphical content. Because let’s see basic logic, which led to this idea: modifier keys, which names start with mod will be spared for extra functions, while a separate key under the name ISO_Level3_Shift will produce the necessary characters. This way we can have one extra modifier key.
But it doesn’t work. However I tried, at the end my system always came up with unexpected stupid things. Once it happened that a key, which hasn’t been defined as modifier key in the xmodmap table, still functioned as it was one. (Could it be a bug?) Or key CapsLock thought, it might as well can do the job of control keys because it wanted to be more than just an ISO_Level3_Shift key, so why not? I never said to it anything like that, but obviously it knows better, smarter than I, and always has the last word! So this strange ISO_Level3_Shift thing only seems to be working correctly, if it’s appointed also as a "mod"-like modifier key.
Now let’s see which character goes to where, because this was our actual starting point. And I not just by any chance started with Hungarian layout; I had a purpose with it. Mainly I use that layout, but I need its accented characters only. So we will use the Hungarian layout as a basis, and change it as follows:
Each of the letters in fields 1. and 2. stay intact. That means, that all the letters remain the same, either just the key itself is pressed down, or together with Shift. So the access of lower and upper case letters hasn’t changed. In line keycode 44, which is for J, we write in the third field “iacute” and “Iacute” in the fourth, otherwise we won’t have the accented í and Í in their usual place. Let’s not forget that now AltGr is responsible for columns third and fourth instead of fifth and sixth. It’s recommended to write the words iacute and Iacute into columns third and fourth of letter I as well. The same way can be accessed the numbers and characters, which in Hungarian layout can be displayed by pressing down shift and the number together, but of course they are not the same as the ones marked on the English keyboard. Whatsoever, if we already got used to their original places in Hungarian layout, it’s practical to keep it this way.
Here comes the smartest thing! As we discussed, the fifth and sixth columns can be accessed by ISO_Level3_Shift. The fifth by pressing the key itself, the sixth by holding down Shift when strike it. And in our case this ISO_Level3_Shift key is nothing, but the CapsLock. It comes in handy for right-handed users, who mainly use their left hand on the modifier keys and use the other one for typing. So what I did was, to find the corresponding keycode line and write the English layout’s characters (signs only, the letters don’t! what for?) into the fifth column, than the characters that is marked in the upper part of the key, those that can be accessed by Shift into the sixth column. Practically that means, that if a character on my keyboard appears on the lower par of the key I can access it by holding down CapsLock, and if it’s on the upper part of a key, than Shift+CapsLock. This combination is very comfy, because the two keys, CapsLock and Shift. are adjacent, at least on my LenovoThinkPad T530 laptop and Logitech K400 wireless keyboard, so I can press both simultaneously with just one finger. This is far easier than switching between different layouts while writing a program. Or if let’s say I write a letter in Hungarian I need to remember where the question mark, or exclamation mark is hidden, because only one thing is for sure: it’s definitely not under the key that has the mark on it in the English layout. But now, it’s guaranteed that I will find every character right under the key that has the mark, I only have to hold down CapsLock, or Shift+CapsLock to access them.
Let me note, that at least punctuation signs are located the same in English and Hungarian layouts, so we have nothing to do with those. The special characters marked above numbers on keys can logically be accessed by Shift+CapsLock too. And because I respect logic a lot, I put them not only in their right place in the keycode table, but also into the fifth columns, so they can be accessed by only holding down CapsLock without Shift. I thought it wouldn’t make much sense, to get the numbers with CapsLock, because average users need to use numbers often, and everyone want to access them without any modifier keys. So to get the characters marked above numbers on each keys we don’t have to use Shift, we only have to hold down CapsLock, which is helpful, considering how often those characters are used in programming.
Now we need to save our finished table in directory $HOME after named it .myxmodmap_hu. Then make an .xinitrc file that includes the following lines:
setxkbmap hu
xmodmap $HOME/.myxmodmap_hu
modkeys
Keymap --
Line setxkbmap hu is necessary to eliminate X’s shortcomings we discussed at the beginning, which has something to do with X not knowing what kind of hardware to expect (but it’s only my guess). So unless we do something to fix this problem, our layout won’t work properly, namely the CapsLock won’t function well as mod5.
The line starting with xmodmap loads up our table, and the modkeys is the script we discussed earlier, which will set the appropriate modifier keys.
We have one more mysterious line:
Keymap --
Well, this is a little script from me that includes the followings:
#!/bin/bash
KEY="hu"
if [[ $1 == "--" ]] ; then
cp /_/P/Szkriptjeim/-/mysettings/aktkeymap.txt /Mount/RAM
exit
fi
if [[ $1 != "" ]] ; then
KEY=$1
fi
echo $KEY > /Mount/RAM/aktkeymap.txt
setxkbmap hu
xmodmap $HOME/.myxmodmap_$KEY
modkeys
exit
So, what does that do? I can tell you, that the above mentioned aktkeymap.txt file’s content is only that:
hu
Well, when this script called containing parameter -- this file will be copied in a special directory, which is located on the RAM disc, but this is not what's important. The point is, if it were called without parameters it would write parameters in the file, and set the layout based on our $HOME directory's .myxmodmap file, which name ends with the given parameter.
This is important now, because so far I sidestepped the question, whether what the right-hand-side Ctrl key would be used for? Well, I've separated it from modifier keys and named F22. As you probably know it, for those Hebrew, Arabic, Russian, Vietnamies or any other special characters you may want to include in your keyboard layout, you have to find file keysymdef.h, in which, you can find the corresponding names for your characters.
(This is part of Xproto, or something similarly named program package, which is part of X server.)
Each name starts with XK_ ; what you need, is the part of the name following it.
As we can see, in file keysymdef.h there are plenty of other "F" function key symbols, many more than just 12! I used up F22 replacing my right-hand-side Ctrl key. Originally I only have function keys up to F12, and I haven't seen a keyboard containing more. Now I also have F22, so I could refer to it in the program xbindkeys. The follwing code is from my own $HOME/.xbindkeysrc file:
# my hu keymap
"Keymap hu"
F22
# my eo keymap
"Keymap eo"
Control + F22
That means, if I press F22 (right-hand-side Ctrl), it will switch to $HOME/.myxmodmap_hu layout. If it pressed it together with Ctrl (left), than the $HOME/.myxmodmap_eo layout will be activated. Many different layouts could be defined the same way. There could be one that defined on top the following combinations:
mod1 + F22
mod2 + F22
mod3 + F22
etc, moreover, if program
xchainkeys
installed, we have endless possibilies...
That's how we can have many own keyboard layouts.
The one named eo is useful for Esperanto characters, because it includes it's accented letters, which can be reached by using AltGr and Shift+AltGr as follows:
Esperanto letter Name of letter Key combination
ĉ ccircumflex AltGr C
Ĉ Ccircumflex Shift+AltGr C
ĝ gcircumflex AltGr G
Ĝ Gcircumflex Shift+AltGr G
ĥ hcircumflex AltGr H
Ĥ Hcircumflex Shift+AltGr H
ĵ jcircumflex AltGr J
Ĵ Jcircumflex Shift+AltGr J
ŝ scircumflex AltGr S
Ŝ Scircumflex Shift+AltGr S
ŭ ubreve AltGr U
Ŭ Ubreve Shift+AltGr U
I only done this to test my idea on switching between my own keyboard layouts. Doesn't make much sense anyway, because these characters could fit in the "hu" layout. But this possibility's worth a fortune for those, who need many different languages with different character sets, such as Russian, Greek, Hebrew, Korean, Japanese, Arabic, etc.
When keymap script switches to another xmodmap file, it will store the code of the name of the actual keymap. This makes possible, to display it in the status bar, so there is no need to keep guessing the active layout while we work.
As a farewell, here's a memo how to modify lines that start with keycode, and what character can be accessed in which field with what modifier keys:
keycode text | code of key | = | pressed the key only | left Shift | AltGr | Shift + AltGr | CapsLock | Shift + CapsLock |
keycode | 44 | = | j | J | jcircumflex | Jcircumflex | iacute | Iacute |
Finally a cute little trick: characters can't only be defined in xmodmap tables by special names, but directly by their UNICODE number, like that:
keycode 27 = r R U003C1 U003A1
Now, I believe nothing can hold you back from making your own keyboard layout, you can get whatever your heart desires!
