On-the-fly Programmable Keyboard Firmware for Kinesis and Ergodox

[chrisandreae]

I've extended my previously announced USB keyboard firmware to support both Ergodox and Kinesis.

Hardware currently supported:

• Kinesis Contoured (Advantage, Professional, Model 110) with drop-in microcontroller replacement (see schematic/ directory)
• Ergodox (with native Teensy - no hardware change required)

Can support most ATMega series AVR microcontrollers, using LUFA if native USB is available and VUSB otherwise. Using the macro and program-interpreter features requires an external AT24C164 or compatible I2C EEPROM. (This EEPROM is already built in to Kinesis Advantage/Professional hardware.)

Features

• On-the-fly reprogrammable keyboard layout, including 'keypad layer' support (two independent key layouts toggled by a 'keypad' key).
• Onboard layout profiles to save/restore up to ten separate programmed keyboard layouts.
• Enhanced text macros that fully support modifier keys and can be triggered by any combination of up to four keys. (requires external EEPROM on Ergodox)
• Virtual machine interpreter that allows complex interpreted macros to be written in a simple programming language. (requires external EEPROM on Ergodox)
• Programming, macro recording and layout save/load can be performed entirely on-keyboard with no additional software.
• USB API for configuring, remapping and uploading programs to the keyboard. (C++ GUI client and Ruby library included.)
• Also appears as USB mouse, mouse functions can be be bound to keys.
• Multi-tone buzzer audio support (included in Kinesis hardware)

Pictures

GUI client


Optional external EEPROM on Ergodox
An EEPROM can easily be attached to the I2C bus in the spare space on the non-Teensy side of the keyboard:


Kinesis Hardware
The Kinesis board uses the easy-to-find Atmel ATMega32 microprocessor. There are two revisions of the hardware: the first is a mostly single-sided PCB which can be easily etched at home, the second (v2.5) is a double-sided PCB suitable for professional manufacture. If you want to order boards easily, I've uploaded them to the DirtyPCBs store here. (Full disclosure: I get a $1 credit each time the board is ordered using this link.)


The firmware and schematics can be found on GitHub under a GPL licence:
https://github.com/chrisandreae/keyboard-firmware. Binaries ready to flash and go are found here.

[hoggy]

Sounds great!

Do you have any plans to extend it to Maltron?

[chrisandreae]

Do you have any plans to extend it to Maltron?

I don't have a Maltron to play with, so I can't do it myself, but the firmware's easily extensible to other keyboard hardware if you know the matrix layout (pretty easy to work out) and don't mind designing a custom circuit board.

[wuqe]

Ooh, on-the-fly programming...

Can hardly wait to check it out on my Ergodox.

[tricheboars]

so what is the advantage of the external EEPROM on the Ergodox if all those features can exist without it?  am i an idiot who missed something.

[wuqe]

The advantages are:

• Enhanced text macros that fully support modifier keys and can be triggered by any combination of up to four keys. (requires external EEPROM on Ergodox)
• Virtual machine interpreter that allows complex interpreted macros to be written in a simple programming language. (requires external EEPROM on Ergodox)

I think they must just not fit in the onboard Teensy memory...

[tricheboars]

what i really want is media key support for windows via the massdrop configurator.  can we get support for that with this new firmware?

[chrisandreae]

what i really want is media key support for windows via the massdrop configurator.  can we get support for that with this new firmware?

As the firmware isn't based on the ergodox firmware, it's not necessary to reflash the keyboard to change the layout. You can remap keys on the keyboard itself (see docs here) or using the GUI config program.

Unfortunately though, the "consumer control device" HID media keys aren't yet supported. It's on the todo list, though.

[nomaded]

On the release page, there is this comment:

Caveat for Windows: To use the GUI client on Windows, you must first install a generic USB driver to allow libusb to access the device. An easy tool for doing this is zadig.exe. This will however prevent Windows from using the usb interface with that driver as HID input (that is to say, either typing (Interface 0) or mouse movement (Interface 1) will be lost).

What does this mean? Does this mean I would need to uninstall the generic USB driver before I could use the ErgoDox or my trackball?

[chrisandreae]

On the release page, there is this comment:

Caveat for Windows: To use the GUI client on Windows, you must first install a generic USB driver to allow libusb to access the device. An easy tool for doing this is zadig.exe. This will however prevent Windows from using the usb interface with that driver as HID input (that is to say, either typing (Interface 0) or mouse movement (Interface 1) will be lost).

What does this mean? Does this mean I would need to uninstall the generic USB driver before I could use the ErgoDox or my trackball?

No, it would affect only the keyboard itself. Here's the issue: Windows has a limitation that you can't send USB vendor commands (what the firmware uses for configuration) to a standard HID device like a keyboard or mouse. So to use the configuration software on Windows, you have to replace the standard HID driver with a generic driver allowing vendor commands. The problem with doing this though is that suddenly you can no longer use the device that you reinstalled the driver of for its standard purpose (i.e typing or mouse movement). Fortunately, the keyboard firmware presents a composite device with two interfaces - a keyboard and a mouse (for mousekey support). Replacing the driver on the mouse interface loses mouse key support, but overall that's not a very big loss.

There's a way to fix the issue: if I change the firmware to use HID feature reports for its configuration interface instead of vendor commands, it'll work without any special driver on Windows. This is definitely on my todo list, but I haven't gotten around to it yet - I use a Mac, so it doesn't personally affect me. If I get a few Windows users asking for the feature, I'll try and get around to it soon

(There's a second, much hackier way to fix the issue: make the keyboard report a third completely fake interface, and just install the generic driver on that. I've pushed a completely untested branch with this to github here)

tl;dr: replace the driver for interface 1 of the keyboard using zadig.exe - you'll lose mousekey support, but gain the use of the config tool.

[metalliqaz]

Wow, amazing stuff!

[Misterorjoe]

Any chance of a linux client? I would love to use this with my Doxen.

[chrisandreae]

Any chance of a linux client? I would love to use this with my Doxen.

The client builds and runs fine in Linux, basic instructions are here. You need to have libusb-dev and qt4-dev to build.

[Misterorjoe]

Does this allow N-Key rollover?

[chrisandreae]

Does this allow N-Key rollover?

Currently it sends standard boot protocol reports with 6-key rollover. Adding a few more to this is very easy though:

• Increase the descriptor's report count here
• Increase KEYSTATE_COUNT here by the same amount

Increasing KEYSTATE_COUNT will affect the scan speed though, so best not to add too many this way.
(To be fully boot-protocol compliant, you should also really cut the report size back to 6 when the host has selected boot protocol with HID_SET_PROTOCOL. Shouldn't actually be a problem in practice, as a boot protocol BIOS host should just ignore any extensions to the report.)

To do complete n-key support properly it'd be a bit more work, it would require:

• Move to tracking the state and debounce of all physical keys rather than a KEYSTATE_COUNT subset (may need to disable a VM or two to free up enough RAM for this)
• Switch to using a bitset (LogicalMinimum=0x0, LogicalMaximum=0x1, ReportCount 0x65, ReportSize 0x1, Input (Variable, Absolute)) in the report descriptor instead of an array, and add a new keystate_Fill_KeyboardReport to construct that.
• Add code to check whether the host has requested us to use report or boot protocol, and actually send the two different report types based on that.

This does slightly break the HID spec, which states in Appendix C (Keyboard Implementation) that "Non-modifier keys must be reported in Input (Array, Absolute) items", but it seems to be the better supported way out there to get NKRO. Myself though, I'd rather have a more compatible keyboard than the ability to press more keys than I have fingers.

[Misterorjoe]

Does this allow N-Key rollover?

Currently it sends standard boot protocol reports with 6-key rollover. Adding a few more to this is very easy though:

• Increase the descriptor's report count here
• Increase KEYSTATE_COUNT here by the same amount

Increasing KEYSTATE_COUNT will affect the scan speed though, so best not to add too many this way.
(To be fully boot-protocol compliant, you should also really cut the report size back to 6 when the host has selected boot protocol with HID_SET_PROTOCOL. Shouldn't actually be a problem in practice, as a boot protocol BIOS host should just ignore any extensions to the report.)

To do complete n-key support properly it'd be a bit more work, it would require:

• Move to tracking the state and debounce of all physical keys rather than a KEYSTATE_COUNT subset (may need to disable a VM or two to free up enough RAM for this)
• Switch to using a bitset (LogicalMinimum=0x0, LogicalMaximum=0x1, ReportCount 0x65, ReportSize 0x1, Input (Variable, Absolute)) in the report descriptor instead of an array, and add a new keystate_Fill_KeyboardReport to construct that.
• Add code to check whether the host has requested us to use report or boot protocol, and actually send the two different report types based on that.

This does slightly break the HID spec, which states in Appendix C (Keyboard Implementation) that "Non-modifier keys must be reported in Input (Array, Absolute) items", but it seems to be the better supported way out there to get NKRO. Myself though, I'd rather have a more compatible keyboard than the ability to press more keys than I have fingers.

This is very interesting. Thanks for all your hard work on this project!

[tricheboars]

please please let us know when this supports HID Media Keys in windows.  thanks!

[h1r0p]

Expecting my Ergo kit from MassDrop to show up in a month or so. Could you supply a part number for the EEPROM so I can have that available too?

[dorkvader]

wow, excellent! I may use this as my "go-to" ergodox firmware for testing.

[chrisandreae]

Expecting my Ergo kit from MassDrop to show up in a month or so. Could you supply a part number for the EEPROM so I can have that available too?

Sure. What I used (because I had it lying around) was an AT24C16AN. You can pick them up extremely cheaply on eBay. Any compatible 2kbyte I2C serial EEPROM should be fine though.

[oystein.krog]

This looks really well done!
I'm curious, what is the approximate scan rate with the default rollover config?
I know there were some issues early on with the TMK firmware with this.

[Misterorjoe]

Any chance of a linux client? I would love to use this with my Doxen.

The client builds and runs fine in Linux, basic instructions are here. You need to have libusb-dev and qt4-dev to build.

I got an error during make.

g++ -c -pipe -g -Wall -W -D_REENTRANT -fPIE -DQT_XML_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I/usr/lib/qt/mkspecs/linux-g++ -I. -I/usr/include/libusb-1.0 -I/usr/include/qt -I/usr/include/qt/QtXml -I/usr/include/qt/QtGui -I/usr/include/qt/QtCore -I. -o keyboardcomm.o keyboardcomm.cc
In file included from keyboardcomm.cc:11:0:
keyboardcomm.h: In member function ‘virtual const char* InsufficentStorageException::what() const’:
keyboardcomm.h:53:6: error: ‘class QString’ has no member named ‘toAscii’
     .toAscii();
      ^
Makefile:1470: recipe for target 'keyboardcomm.o' failed
make: *** [keyboardcomm.o] Error 1

Any ideas?

[hasu]

This?
http://www.qtcentre.org/threads/53783-toAscii()-in-Qt5

[Misterorjoe]

This?
http://www.qtcentre.org/threads/53783-toAscii()-in-Qt5

I got past that error by using qtchooser with Qt4, but now on the make command I get the following error:

g++ -c -pipe -g -Wall -W -D_REENTRANT -DQT_XML_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/share/qt4/mkspecs/linux-g++ -I. -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtGui -I/usr/include/qt4/QtXml -I/usr/include/qt4 -I/$
Makefile:550: recipe for target 'hexdump.o' failed

Any ideas?

[vvp]

Did anybody convert Kinesis Advantage schematic to KiCad? If yes, can you post it?

[Misterorjoe]

This?
http://www.qtcentre.org/threads/53783-toAscii()-in-Qt5

I got past that error by using qtchooser with Qt4, but now on the make command I get the following error:

g++ -c -pipe -g -Wall -W -D_REENTRANT -DQT_XML_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/share/qt4/mkspecs/linux-g++ -I. -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtGui -I/usr/include/qt4/QtXml -I/usr/include/qt4 -I/$
Makefile:550: recipe for target 'hexdump.o' failed

Any ideas?

Alright, I got it to build and run and build properly now, by running qmake-qt4 now instead, but now I get an error when I click download default settings.

Error downloading settings:  LIBUSB_ERROR_TIMEOUT
Segmentation fault (core dumped)


Any ideas?

[Misterorjoe]

Okay, when I don't run ./KeyboardClient as root, I get
LIBUSBError when enumerating:  LIBUSB_ERROR_ACCESS


which makes me think that libusb is giving me trouble, somehow..

[chrisandreae]

Any ideas?

I just tried building it in an Ubuntu VM and had no issues. Library versions libqtcore4 4:4.8.1-0ubuntu4.6 and libusb-1.0-0 2:1.0.9~rc3-2ubuntu1. In case you happen to be using Ubuntu, I've made (attached) a quick and dirty Ubuntu package which you could try.

[chrisandreae]

Did anybody convert Kinesis Advantage schematic to KiCad? If yes, can you post it?

I haven't, but it'd be pretty cool if someone did! Another really useful project would be to redesign it as a two-layer board suitable for professional manufacture rather than home etching. It should be possible to lay it out entirely within the DIP40 socket footprint and remove the need for the separate daughterboard.

[Misterorjoe]

Any ideas?

I just tried building it in an Ubuntu VM and had no issues. Library versions libqtcore4 4:4.8.1-0ubuntu4.6 and libusb-1.0-0 2:1.0.9~rc3-2ubuntu1. In case you happen to be using Ubuntu, I've made (attached) a quick and dirty Ubuntu package which you could try.

I am actually using Arch, and I am pretty new to it still, so I am very likely doing something wrong, but I can pry still look at the package you built as a reference. I will take a look when I get back home.

Just to give you more info on the problem that I was having, the program runs, but it doesn't specify what kind of keyboard is plugged in, and when I click load default, it crashes. It also refuses to reconnect to the keyboard until I re-flash the firmware, which I thought was bizarre.

[chrisandreae]

I am actually using Arch, and I am pretty new to it still, so I am very likely doing something wrong, but I can pry still look at the package you built as a reference. I will take a look when I get back home.

I cobbled together an Arch VM and it worked fine there too. Not sure what's breaking for you, but I'll send you the binary that worked for me in a PM.

[chrisandreae]

This looks really well done!
I'm curious, what is the approximate scan rate with the default rollover config?
I know there were some issues early on with the TMK firmware with this.

• With the LUFA core on an Ergodox, about 180Hz. This is constrained by the I2C I/O expander. It's measured with the default 100kHz I2C frequency: if your hardware copes with it, you could do a lot better by bumping it up to 400kHz. I think this is what was done to improve the TMK firmware's performance. Next time I get my hands on an Ergodox I'll do a bit of testing.

• With the VUSB core on my Kinesis, about 420Hz. This is constrained mostly by the software USB implementation.

[rethus]

Okay, when I don't run ./KeyboardClient as root, I get
LIBUSBError when enumerating:  LIBUSB_ERROR_ACCESS


which makes me think that libusb is giving me trouble, somehow..

Same Problem here at Kubuntu 14.04

[vvp]

Did anybody convert Kinesis Advantage schematic to KiCad? If yes, can you post it?

I haven't, but it'd be pretty cool if someone did!

Can you post pictures (or pdf file) of the schematic?

[chrisandreae]

Okay, when I don't run ./KeyboardClient as root, I get
LIBUSBError when enumerating:  LIBUSB_ERROR_ACCESS


which makes me think that libusb is giving me trouble, somehow..

Same Problem here at Kubuntu 14.04

If you're not running as root, you need to make sure that your user account has permissions to access the USB device. This typically involves creating a udev rule. See here and here for more details.

[chrisandreae]

Did anybody convert Kinesis Advantage schematic to KiCad? If yes, can you post it?

I haven't, but it'd be pretty cool if someone did!

Can you post pictures (or pdf file) of the schematic?

Sure - attached.

[ublick]

For the ErgoDox, is it possible to install the EEPROM on the right hand side?

[chrisandreae]

For the ErgoDox, is it possible to install the EEPROM on the right hand side?

Sure, it needs to be attached to the I2C bus (i.e. the cable going between the two halves), so you could install it anywhere you find room for it on either side, or even in-line with the cable in order to avoid modifying the keyboard itself.

[ublick]

For the ErgoDox, is it possible to install the EEPROM on the right hand side?

Sure, it needs to be attached to the I2C bus (i.e. the cable going between the two halves), so you could install it anywhere you find room for it on either side, or even in-line with the cable in order to avoid modifying the keyboard itself.

I'll preface this with "I'm not an electrical engineer," so this is all very foreign to me.

Would this chip work? http://www.digikey.com/product-detail/en/AT24C16C-XHM-B/AT24C16C-XHM-B-ND/3046491

On a side note, would you happen to know if those 2.2k resistors are needed if no left-hand keyboard is being used? If that's the case then it appears as though, I may be able to connect it to the same contacts as in the picture of the left side.

The I2C doesn't appear to be labeled on the right hand PCB so I'm just not sure. Would you be able to point out the contacts that the chip should be connected to? Given that there are resistors soldered at the points where the chip is connected on the right hand side I'm not sure if that's the proper place.

[vvp]

Would this chip work? http://www.digikey.com/product-detail/en/AT24C16C-XHM-B/AT24C16C-XHM-B-ND/3046491

Probably yes. If you already have them just try it.

On a side note, would you happen to know if those 2.2k resistors are needed if no left-hand keyboard is being used? If that's the case then it appears as though, I may be able to connect it to the same contacts as in the picture of the left side.

Those resistors are pull-ups for I2C bus. You need them if you use any I2C device (e.g. the EEPROM). Even if you would not use any I2C device the firmware would probably halt if you would try to initialize the bus. So you can get rid of them only if you do not use any I2C device and make sure the firmware will not try to initialize the bus (which probably requires software change).

The I2C doesn't appear to be labeled on the right hand PCB so I'm just not sure. Would you be able to point out the contacts that the chip should be connected to? Given that there are resistors soldered at the points where the chip is connected on the right hand side I'm not sure if that's the proper place.

You can attach it to the resistors on the right end side, their bottom (non-vcc) side.

[Nootilus]

Hello there!

First time here I am currently looking for help to map correctly my brand new Ergodox.
I am french and I use an alternate kexboard layout known as Bépo. Unfontunately, because of the accents and different layers, the Massdrop Configurator isn’t much of a help to me. I think your app is a wonderful solution.
So far I am stuck at this damn USB driver issue. I downloaded you win client, and teh Zadig app.
Whin you say…

replace the driver for interface 1 of the keyboard using zadig.exe - you'll lose mousekey support, but gain the use of the config tool.

I am puzzled. With what should I replace the Interface 1 ? I also see the Ergodox in the list of UBS devices, do I have to do something with it?
I am sorry to ask such noobs questions, but I am totally lost…

Thanks in advance to whoever will answer me

See you soon!

[chrisandreae]

I am puzzled. With what should I replace the Interface 1 ? I also see the Ergodox in the list of UBS devices, do I have to do something with it?
I am sorry to ask such noobs questions, but I am totally lost…

Open Zadig, and choose 'List all devices' from the options menu

Choose the target USB device to replace, for you it would probably be "Ergodox (Interface 1)"

Ensure "WinUSB" is selected, and hit "Replace driver"

Allow installation

Should now work!


Also for better international support, you probably will want to install the latest firmware release (https://github.com/chrisandreae/keyboard-firmware/releases/tag/1.1.1) - before this, the keyboard didn't report that it could provide the full range of keycode HID usages, which would have prevented several international extended keys (and other useful keys such as volume up/down) from working.

[Nootilus]

Thanks a lot, chrisandreae!
I’ll try that ASAP
.v

[ublick]

Thanks everyone for your replies. I have the firmware installed and it's working great!

My desired configuration is somewhat unique, so I'll be posting pictures in a little while with what I did different.

[mikew0w]

I just started using this too and I am enjoying it.

Is it possible to move the keypad button?

[chrisandreae]

I just started using this too and I am enjoying it.

Is it possible to move the keypad button?

The keypad and program keys currently can't be soft-remapped due to their special functions, however if you don't mind editing the source and recompiling, you can change their positions in the MATRIX_TO_LOGICAL_MAP in hardware/ergodox.c. Just pick another LOGICAL_KEY_ to swap with, and exchange their positions.

[ublick]

Since my need was for only one side of the ErgoDox, I ended up placing the EEPROM in a different location.

I used some spare USB cable to connect the EEPROM. I had enough and it's colored so I could tell the difference between leads easily.

I used the datasheet from Amtel to identify which leads were which on my package of the chip.

Identifying which contacts for the power was fairly easy since they're labeled on the board (on the other side).


The SCL, and SDA were a little more difficult, but is simply a mirror of the example picture in the first post.


The package of the chip that I got was not ideal. I'm not sure which would be the best, but this one is designed for surface mount and the leads are incredibly small and was a huge pain to solder the wires in place.

The chip lives on the front of the board near the controller in an available space. Luckily there was a hole in the board already through which I could route the wires.

The finished product:


Thanks again to everyone that helped. The firmware has been working quite well.

[Benaan]

Does the eeprom have to be 2kbyte? Or can it be larger? It's hard to find the one you mentioned here. Would something like http://ww1.microchip.com/downloads/en/DeviceDoc/20001203U.pdf also work?

[chrisandreae]

Does the eeprom have to be 2kbyte? Or can it be larger? It's hard to find the one you mentioned here. Would something like http://ww1.microchip.com/downloads/en/DeviceDoc/20001203U.pdf also work?

There's no reason it needs to be constrained to 2 kbyte, but the protocol for talking to the larger eeproms is slightly different (two byte addresses), so if you wanted to use one you'd have to change serial_eeprom.c to match. If you don't want to do that, other compatible 2 kbyte serial eeproms aren't that hard to find: for example 10x AT24C16 for $4 or so on eBay here. It's not necessary to specifically use the (obsolete) AT24C164: that's just what happens to be already present in the Kinesis.

[chrisandreae]

I've designed a new double-sided PCB for the Kinesis, suitable for professional manufacture, which should reduce the effort required to get your hands on one. Schematics and Gerber files can be found here, or they can be ordered from the DirtyPCBs store here