Bluefruit EZ-key with tmk_keyboard firmware

[bcg]

I purchased the Bluefruit EZ-key breakout from Adafruit recently:  http://learn.adafruit.com/introducing-bluefruit-ez-key-diy-bluetooth-hid-keyboard

Its an interesting device as it lets you easily turn just about any switch into a bluetooth keyboard, even without a microcontroller.

With a microcontroller you can send data to it over serial and it will present the data to a bluetooth host as a HID keyboard.

Recently discovered that you can send raw HID reports to it even (i think you can only do the standard 6 key variety) and it will pass those through.  So this weekend I hooked it up to an Arduino Pro Micro which is really a teensy-like atmega 32u4 breakout (but fewer pins and no onboard reset button).  Anyhow I wired it up to a terminal Model M and now I have a bluetooth Model M



UPDATE: The problem described below existed with the Bluefruit 1.0 only; you must use a Bluefruit 1.1 at least for the keyboard & mouse functions, and you must use Bluefruit 1.2 for consumer keys to work

Sort of.

I have a pretty serious bug that I need to solve.

Everything works fine as long as I only press one key at a time... if I press two, things get a little strange, I'll try to describe this as best I can...

If I press two keys at once, the first key is typed on my PC just fine, as is the second key.  However if I release the first key, another HID report gets sent to and the other key gets typed out again (as if there was a key up in between).    To better illustrate here is the debug output from pressing and holding 'g', then pressing and holding 'h', then releasing 'h', then releasing 'g':

Code: [Select]

$ sudo ./hid_listen
Waiting for device:....
Listening:
Setting host driver...
Initializing serial...
Beginning main loop
rAA wFF [ack]
RESET_RESPONSE: rBF err
RESET: rBF wFF x01wFF x01wFF x01wFF x01wFF [ack]
RESET_RESPONSE: rAA [ok]
KBD_ID: rBF rBF
CONFIG: wF8 [ack]
READY
r34
keys: 000A 0000 0000 0000 0000 0000  mods: 0000 reserved: 0000
r33
keys: 000A 000B 0000 0000 0000 0000  mods: 0000 reserved: 0000
rF0  r33
keys: 000A 0000 0000 0000 0000 0000  mods: 0000 reserved: 0000
rF0  r34
keys: 0000 0000 0000 0000 0000 0000  mods: 0000 reserved: 0000

The result of this is that 'ghg' gets typed out over Bluetooth.

Sorry if I'm not explaining clearly or with the right vocabulary, I'm kind of a helpless noob

Here's the code I'm using to implement the bluefruit protocol... am I sending the HID report incorrectly or could this be a bug or problem with the Bluefruit itself?

Code: [Select]

void bluefruit_keyboard_print_report(report_keyboard_t *report)
{
    dprintf("keys: "); for (int i = 0; i < REPORT_KEYS; i++) { debug_hex16(report->keys[i]); dprintf(" "); }
    dprintf(" mods: "); debug_hex16(report->mods);
    dprintf(" reserved: "); debug_hex16(report->reserved);
    dprintf("\n");
}

void bluefruit_transmit_report(report_keyboard_t *report)
{
    serial_send(0xFD);
    serial_send(report->mods);
    serial_send(0x00);
    for (uint8_t i = 0; i < REPORT_KEYS; i++) {
        serial_send(report->raw[i]);
    }
}

/*------------------------------------------------------------------*
 * Host driver
 *------------------------------------------------------------------*/

static uint8_t keyboard_leds(void);
static void send_keyboard(report_keyboard_t *report);
static void send_mouse(report_mouse_t *report);
static void send_system(uint16_t data);
static void send_consumer(uint16_t data);

static host_driver_t driver = {
        keyboard_leds,
        send_keyboard,
        send_mouse,
        send_system,
        send_consumer
};

host_driver_t *bluefruit_driver(void)
{
    return &driver;
}

static uint8_t keyboard_leds(void) {
    return bluefruit_keyboard_leds;
}

static void send_keyboard(report_keyboard_t *report)
{
    bluefruit_keyboard_print_report(report);
    bluefruit_transmit_report(report);
}

Thanks in advance for your help

[hasu]

Nice work!

It looks to me debug output indicates key strokes are set  into report structure in correct order.
I can't come up with any causes at this time unfortunately.

FYI, You can use debug_hex8 instead of debug_hex16 in bluefruit_keyboard_print_report.
Mods, reserved and keys of keyboard_report are all byte data(uint8_t).

[hasu]

Gotcha!

Code: [Select]

void bluefruit_transmit_report(report_keyboard_t *report)
{
    serial_send(0xFD);
    serial_send(report->mods);
    serial_send(0x00);
    for (uint8_t i = 0; i < REPORT_KEYS; i++) {
        serial_send(report->raw[i]);
    }
}
You must use keys instead of raw.

Or you will be able to write like this.

Code: [Select]

    serial_send(0xFD);
    for (uint8_t i = 0; i < REPORT_SIZE; i++) {
        serial_send(report->raw[i]);
    }


[bcg]

Gotcha!

You must use keys instead of raw.

Good catch, awesome.  I've updated that in my code

Unfortunately it didn't solve the problem, but it is more clear what is happening now

Its almost like the Bluefruit is sending a "key up" or something, or maybe my computer is expecting continuous reports even when there have been no changes or something like that.  If the second case the problem then that should be easy enough to solve by just storing the last report and sending it repeatedly in the main loop.  I think I saw you did that for another protocol which makes me think this could be the same thing.  However in the adafruit tutorial they say "You can send up to 6 consecutive keys at once, don't forget to send the release 'key up' command or the key will be 'stuck'!" which would contradict that requirement

I might explore the second possibility a little later, in the meantime I'm going to cross post to the Adafruit forum to see if they can shed some light on the issue

Thanks for your help hasu, you rock

EDIT:

for reference, asked this question on the Adafruit forum:  http://forums.adafruit.com/viewtopic.php?f=8&t=45216

[bcg]

Success finally... turns out the version of Bluefruit I was using (1.0) does not support raw HID very well.

Adafruit, being just about the greatest company I've done business with in a long time, sent me an updated Bluefruit to work with (version 1.2).  Soldered on headers and dropped it into my circuit and voila, its working fine now.

Even better news is that 1.1 version added mouse support, and 1.2 added support for consumer keys.  I didn't really have an intention to use mouse keys, but for the sake of completeness and curiosity I want to get this working.  I don't think this should be too difficult... Adafruit doesn't have anything in their tutorial about the mouse wheel, but I guess there's only one way to find out if that will work.

Now I think the last piece that I want to make is a way to switch between USB and bluetooth... is there anything special I need to do if I am going to switch protocols midstream?  Or can I just call host_set_driver whenever I want to?  Maybe it would be better if I just stored some value in the eeprom about which protocol to use and then check that value at start up, and then when switching I could just restart the microcontroller?  Not sure which way to go with this but so far so good.

Thanks for your help so far hasu... once I have this last part done I'll clean up the code and do a pull request on github if you're interested, or I'll post it here if you don't want to include it in your github project

[hasu]

Great. And I'm also glad you could get nice support from Adafruit.
OK. We just need to make sure it has the latest vesrion firmware before getting this module.

Now I think the last piece that I want to make is a way to switch between USB and bluetooth... is there anything special I need to do if I am going to switch protocols midstream?  Or can I just call host_set_driver whenever I want to?  Maybe it would be better if I just stored some value in the eeprom about which protocol to use and then check that value at start up, and then when switching I could just restart the microcontroller?  Not sure which way to go with this better so far so good.

I think you can switch protocols on the fly just with host_set_driver(), but you may need to send a keyboard report to last host to make sure all keys are up. Of course, you can also select protocol at startup.

Thanks for your help so far hasu... once I have this last part done I'll clean up the code and do a pull request on github if you're interested, or I'll post it here if you don't want to include it in your github project

Sure, I'm very interested in merging your project and I prefer a pull request on github.

[bcg]

Ok I've got mouse keys working ok...

Sorry about all the questions hasu ... but do you know what the format of the HID report for the consumer keys is?

[hasu]

In USB the format of HID report depends on how you declare HID report descriptor.  As for Usage ID you can get from this USB spec. Consumer Page is spoted on page 75.
http://www.usb.org/developers/devclass_docs/Hut1_12v2.pdf

But not sure about HID in Bluetooth.

[bcg]

In USB the format of HID report depends on how you declare HID report descriptor.  As for Usage ID you can get from this USB spec. Consumer Page is spoted on page 75.
http://www.usb.org/developers/devclass_docs/Hut1_12v2.pdf

But not sure about HID in Bluetooth.

Wow that is not exactly light reading

Turns out they make it easier than I thought, here is what Adafruit says about sending the consumer keys:

Yes you can send although it is not in document yet.

Command for consumer keys are:
FD 00 02 ?? ?? 00 00 00 00
Question marks are bit map of
"Home", "KeyboardLayout", "Search", "Snapshot", "VolumeUp", "VolumeDown", "Play/Pause", "Fast Forward", "Rewind","Scan Next Track", "Scan Previous Track", "Random Play","Stop",

For example, if you want to hold home button:
FD 00 02 01 00 00 00 00 00
Then send
FD 00 02 00 00 00 00 00 00
to release it.

Or Stop
FD 00 02 00 10 00 00 00 00
Then send
FD 00 02 00 00 00 00 00 00
to release it.

You can hold multiple keys simultaneously with an "OR" logic because it is bit map.

So here is what I came up with for the send_consumer function:

Code: [Select]

/*
+-----------------+-------------------+-------+
| Consumer Key    | Bit Map           | Hex   |
+-----------------+-------------------+-------+
| Home            | 00000001 00000000 | 01 00 |
| KeyboardLayout  | 00000010 00000000 | 02 00 |
| Search          | 00000100 00000000 | 04 00 |
| Snapshot        | 00001000 00000000 | 08 00 |
| VolumeUp        | 00010000 00000000 | 10 00 |
| VolumeDown      | 00100000 00000000 | 20 00 |
| Play/Pause      | 01000000 00000000 | 40 00 |
| Fast Forward    | 10000000 00000000 | 80 00 |
| Rewind          | 00000000 00000001 | 00 01 |
| Scan Next Track | 00000000 00000010 | 00 02 |
| Scan Prev Track | 00000000 00000100 | 00 04 |
| Random Play     | 00000000 00001000 | 00 08 |
| Stop            | 00000000 00010000 | 00 10 |
+-------------------------------------+-------+
*/
#define CONSUMER2BLUEFRUIT(usage) \
    (usage == AUDIO_MUTE           ? 0x0000  : \
    (usage == AUDIO_VOL_UP         ? 0x1000  : \
    (usage == AUDIO_VOL_DOWN       ? 0x2000  : \
    (usage == TRANSPORT_NEXT_TRACK ? 0x0002  : \
    (usage == TRANSPORT_PREV_TRACK ? 0x0004  : \
    (usage == TRANSPORT_STOP       ? 0x0010  : \
    (usage == TRANSPORT_STOP_EJECT ? 0x0000  : \
    (usage == TRANSPORT_PLAY_PAUSE ? 0x4000  : \
    (usage == AL_CC_CONFIG         ? 0x0000  : \
    (usage == AL_EMAIL             ? 0x0000  : \
    (usage == AL_CALCULATOR        ? 0x0000  : \
    (usage == AL_LOCAL_BROWSER     ? 0x0000  : \
    (usage == AC_SEARCH            ? 0x0400  : \
    (usage == AC_HOME              ? 0x0100  : \
    (usage == AC_BACK              ? 0x0000  : \
    (usage == AC_FORWARD           ? 0x0000  : \
    (usage == AC_STOP              ? 0x0000  : \
    (usage == AC_REFRESH           ? 0x0000  : \
    (usage == AC_BOOKMARKS         ? 0x0000  : 0)))))))))))))))))))

static void send_consumer(uint16_t data)
{
    static uint16_t last_data = 0;
    if (data == last_data) return;
    last_data = data;
   
    uint16_t bitmap = CONSUMER2BLUEFRUIT(data);
   
#ifdef BLUEFRUIT_TRACE_SERIAL
dprintf("\nData: ");
debug_hex16(data);
dprintf("; bitmap: ");
debug_hex16(bitmap);
dprintf("\n");
dprintf("+------------------------------------+\n| HID report to Bluefruit via serial |\n+------------------------------------+\n|");
#endif
bluefruit_serial_send(0xFD);
bluefruit_serial_send(0x00);
bluefruit_serial_send(0x02);
bluefruit_serial_send((bitmap>>8)&0xFF);
bluefruit_serial_send(bitmap&0xFF);
bluefruit_serial_send(0x00);
bluefruit_serial_send(0x00);
bluefruit_serial_send(0x00);
bluefruit_serial_send(0x00);
#ifdef BLUEFRUIT_TRACE_SERIAL
dprintf("|\n+------------------------------------+\n\n");
#endif
}

As you can see there are some consumer functions that are support by Bluefruit but not by tmk firmware... not a big deal though, at least play, pause, volume, etc are working... would be nice to have mute, calculator, and browser but no big deal... and I couldn't find any key codes that coincided with things like "Snapshot" or "Random Play" so I just ignored those for now

Thanks for your help so far hasu.  I have a couple more things to finish up and then I'll be ready to publish my work up to this point; I still don't have  a good power circuit but I hope to have to some time to work on that soon.  Here's a sneak peak at my converter so far (working fine, just needs a portable/rechargeable power source):

[bcg]

hasu - I pushed my code to github & created a pull request

I still have to work out a good power setup and add some LEDs and switches, but this works as it is.  I'm just using 5v usb charging battery pack to power the keyboard on bluetooth; its ugly right now but I'll come up with a cleaner solution soon.

Also I'm planning on cutting down my Model M, I'll post pics once that is done and give more details on the hardware setup.  Its really straightforward though and you can probably easily see what I did from the code

EDIT:

Upon further inspection this protocol should work for the RN-42 HID profile as well:  http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Wireless/Bluetooth/RN-HID-User-Guide-v1.0r.pdf

[hasu]

Merged!

[Matt3o]

very interesting bt module, and very interesting project!

would you mind sharing more about your project? (connections, pinouts, ...) thanks!

[bcg]

very interesting bt module, and very interesting project!

would you mind sharing more about your project? (connections, pinouts, ...) thanks!

Yes I definitely will... sorry I'm extremely busy right now - but I plan to draw up a schematic and more detailed instructions.  Also I am going to try to come up with decent battery/charging circuit as well as do some refactoring in the code to make it easier to integrate with other keyboards or converters.  I'll try to get something written up this week

[damorgue]

Also I'm planning on cutting down my Model M, I'll post pics once that is done and give more details on the hardware setup.  Its really straightforward though and you can probably easily see what I did from the code

Curious how long a Model M would last on a few batteries. They are known to draw quite a bit of power aren't they, like 100mA? I don't know if that is constant or upon keypress but looking forward to your results.

[bcg]

Curious how long a Model M would last on a few batteries. They are known to draw quite a bit of power aren't they, like 100mA? I don't know if that is constant or upon keypress but looking forward to your results.

The fuse is blown on my multimeter otherwise I would check... I had it in my head that it was around 120mA; plus I have an arduino pro micro and the bluefruit, so I'm guessing around 200mA - maybe less if you don't have lock LEDs

In any case, a 2500 milliamp-hour battery should give 8+ hours of use without recharging

Also eventually I plan to replace the whole controller with a teensy++ connected directly to the membrane, I think that should reduce overall power use.  Eventually I want to use wcass's custom NKRO membrane

[hargon]

I'm curious: Any news on this project?

[bcg]

Yes, tutorial will be ready very soon, I've just been quite busy.  Will post as soon as its ready.

[mohitleo9]

any news on this one ? !! waiting for this very eagerly!:)

[clickclack123]

Registering interest in this. I really want nkro bluetooth though.

[hargon]

Would this be doable directly from PS2?

[hasu]

Yes. I think it is an easy job.

You need to copy matrix.c and keymap files from ps2_usb. And ...
Only this? I can't come up with other thing to do.

[hargon]

But it is not possible without the teensy? So I cant just send the PS2 signals directly to the bluefruit?

[bcg]

Yes. I think it is an easy job.

You need to copy matrix.c and keymap files from ps2_usb. And ...
Only this? I can't come up with other thing to do.

Yep, then just make sure that you have an appropriate main.c for the bluefruit host.  This is precisely an example of that except that it uses codeset 3:

https://github.com/tmk/tmk_keyboard/tree/master/converter/terminal_bluefruit

BTW hasu I found out that you can ask any Model M to use codeset 3 and it works fine... so you can have a "generic" Model M converter that works for both terminal and "normal" Model M's.  This is how I did it, but I haven't had a chance to port it back into TMK yet (apparently my indentation is a little messy in this file):

https://github.com/bgould/arduino_tmk_keyboard/blob/master/api/PS2MatrixCodeset3.cpp

But it is not possible without the teensy? So I cant just send the PS2 signals directly to the bluefruit?

You need a microcontroller for this to work well.  If you had a keyboard that outputs serial you could hook it directly to the bluefruit module, but when you send serial data it sends both a make and break, so there is no way to hold down a key or send more than 1 key at a time.  if you went that route you could try one of these for ps/2 but this costs the same as a teensy 2.0:

http://www.adafruit.com/products/1136

[Pacifist]

interested

would be nice to go with JD40

[hargon]

OK, thanks for the explanation.

[hasu]

BTW hasu I found out that you can ask any Model M to use codeset 3 and it works fine... so you can have a "generic" Model M converter that works for both terminal and "normal" Model M's.  This is how I did it, but I haven't had a chance to port it back into TMK yet (apparently my indentation is a little messy in this file):

https://github.com/bgould/arduino_tmk_keyboard/blob/master/api/PS2MatrixCodeset3.cpp

Good catch! It is very nice for users to be allowed to connect any Model M's without awareness of its code set.
BTW I'm porting tmk to CortexM(mbed) and during the process I'll learn C++, so I would work with your Arduino code sometime later.

[Smasher816]

Feel free to PM me if you need any help with C++. I have a little bit of experience with it.

C++ can be just like C. You still have all your old functions and the syntax is the same. C++ just adds on some extra stuff for you to use like classes, std:: library, etc.

[bcg]

Running TMK on ARM is a killer feature.  I've had a Teensy 3.1 on my desk for too long waiting to become a keyboard controller.  I would really like to see if I can use the OTG usb host interface that a lot of those ARM chips have in order to be able to make a programmable bluetooth converter, like this one but better : http://handheldsci.com/kb

[JustCallMeCrash]

Any chance this could be made to work with the ErgoDox?  It's Teensy2-based and outputs only USB...

[Smasher816]

It should work with any keyboard using a teensy. You just connect the Bluetooth to the Rx,Tx pins - it doesn't matter how the keyboard is wired. Then just add some battery power if you want to make it portable.

[bcg]

It might be hard with Ergodox especially if you are using a standard case since there would be no room for the extra components.  Also the only way to do an external converter would be use something like the USB host shield, or something with a USB OTG host interface like Teensy 3.1 etc

[themitch22]

I'm interested in the TMK/Bluefuit EZ-key functionality. I have a Quickfire Rapid I want to convert, the easiest way I can think of is USB > PS/2 > Teensy 2 > Bluefruit EZ-key would this work? Else I'll have to take the 40 pin header and wire the raw matrix from the QFR to the Teensy 2 but that would be a lot more work.

[Engvard]

It might be hard with Ergodox especially if you are using a standard case since there would be no room for the extra components.  Also the only way to do an external converter would be use something like the USB host shield, or something with a USB OTG host interface like Teensy 3.1 etc

You can go with full-hand case. I suppose it has enough space for both bt board and a battery.

[justinyhuang]

after reading this thread I have some questions:
   1. Smasher816 mentioned what you need to do is to connect the bluetooth to the Rx, Tx pins of Teensy, but don't you still need to change the TMK firmware to send the key press events via Serial? or Does TMK Firmware by default will send the events to USB and Serial?
   2. It seems to me that the TMK firmware requires specifying a keyboard when building, what about building a firmware for a custom keyboard based on Teensy, like JD40? Do I need to create the keyboard files for this keyboard so that TMK firmware knows how to work with it?

Thanks a lot for any pointers/suggestions.

[hasu]

1. No, it doesn't send to Serial by default. You have to use correct protocol module, namely 'tmk_core/protocol/bluefruit'
2. Yes, you have to write some codes for the keyboard. If someone had already done the job you can use it luckily.

[justinyhuang]

thank you hasu for the info!
i am a bit confused about the protocol though:

    so i see the Teensy driver under protocol/lufa, and you mentioned in order to send via Serial protocol/bluefruit needs to be used. what protocol should i use if i would like to run the code on a Teensy, and to send the key events through the Rx/Tx ports of the Teensy to the BlueFruit board that would do the bluetooth stuff?

What i am trying to figure out here is:

    how can i change/configure the TMK firmware so that i can make a Teensy talk to a BlueFruit via Serial port.

many thanks again for all your inputs,

[hasu]

'protocol/bluefruit' talks with BlueFruit module via serial lines. You can configure with Makefile to use the protocol.
This is a converter, not a keyboard actually but I think you can learn a lot from this bcg's work.

https://github.com/tmk/tmk_keyboard/tree/master/converter/terminal_bluefruit
https://learn.adafruit.com/convert-your-model-m-keyboard-to-bluetooth-with-bluefruit-ez-key-hid/overview

[clickclack123]

Does nkro work over bluetooth using the Bluefruit, hasu??

Is it possible to do nkro over bluetooth??

[hasu]

Not possible with bluefruit.

I'm not sure about NKRO on Bluetooth but Haata said it is possible from spec.

[a-c]

Thank you to bcg for all the work you put into this and to hasu for tmk.

Got my KC60 working with a bluefruit using your terminal_bluefruit as a starting point.

Not a valid vimeo URL

[neverused]

Thank you to bcg for all the work you put into this and to hasu for tmk.

Got my KC60 working with a bluefruit using your terminal_bluefruit as a starting point.

Not a valid vimeo URL

That's awesome! Can you share how you did it? I would love to replicate that result too. Thanks!

[a-c]

That's awesome! Can you share how you did it? I would love to replicate that result too. Thanks!

I don't have LEDs so I disable them, but no reason they shouldn't work. Can probably remove the delays in the startup too.

Not a valid vimeo URL

[neverused]

That's awesome! Can you share how you did it? I would love to replicate that result too. Thanks!

I don't have LEDs so I disable them, but no reason they shouldn't work. Can probably remove the delays in the startup too.

Not a valid vimeo URL

How did you output serial to the bluetooth module?

[a-c]

How did you output serial to the bluetooth module?

View the video on Vimeo, I have links in the description.

The KC60 has pins D2/D3 exposed (ATMEGA32U4 serial port). One of the few thoughtful things they did.
http://imgur.com/a/H2SCv

[neverused]

How did you output serial to the bluetooth module?

View the video on Vimeo, I have links in the description.

The KC60 has pins D2/D3 exposed (ATMEGA32U4 serial port). One of the few thoughtful things they did.
http://imgur.com/a/H2SCv

Ah I had assumed that you were running tmk firmware.

[a-c]

Ah I had assumed that you were running tmk firmware.

It is running tmk. Did you watch the video?

[neverused]

Ah I had assumed that you were running tmk firmware.

It is running tmk. Did you watch the video?

I didn't see the link in the imgur album

[neverused]

Ah I had assumed that you were running tmk firmware.

It is running tmk. Did you watch the video?

I didn't see the link in the imgur album

I looked again, I still don't see the video link... Can you post it here please?

[justinyhuang]

I looked again, I still don't see the video link... Can you post it here please?

the video link is not in the imgur album, it is in a-c's post on Wed, 14 October 2015, 20:46:36

[justinyhuang]

so I almost get mine keyboard working with the BlueFruit, except that the keyboard uses PD5 as one of the scan matrix pins, while in talking to the BT module I believe the pin is used as XCLK. therefore some of the keys associated with PD5 would not work...

i tried digging into the atmega32u4 data sheet a bit and it seems like for synchronous mode (USART) the XCLK pin would be used, while in asynchronous mode the pin would not be used. I checked the bits set in UCSR1C and it appears to me that UMSEL10 and UMSEL11 are not set, which means it is working in asynchronous mode. and then I get confused...

not being an expert in UART/USART communication, i would appreciate any pointer/suggestion/hint to keep PD5 being used as XCLK when talking to the BlueFruit.

many thanks in advanced!
justin