[BUILD] Nyan Keys - An FPGA based USB 2.0 HS 8000hz Open Source PCB Design

[russeree]

Nyan Keys - FPGA Powered Mechanical Keyboard



Design Goal
Use an FPGA to eliminate key matrix scanning to offload the MCU and produce the lowest latency possible from switch contact to the host computer. The primary driver of this designs reduced latency is the use of USB 2.0 High-Speed (HS), which offers native support for 125µs interrupt rates, or an 8000Hz polling rate, right out of the box without the need for additional drivers. The FPGA serves as another method to allow the MCU to focus solely on transmitting data from the keyboard to the host.

Any time I use the term "USB" in this document I am referring to USB 2.0 HS 480mb/s. This is very different than the FS and HS USB 1.1 over 2.0 implementations that exist on the market.

Nyan keys is a multi part project consisting of 3 primary parts

• Nyan Keys PCB
• Nyan OS (NOS)
• Web Config Tool

Nyan Keys PCB - By Me




Open Source Design Repos - Apache-2.0 License Github
FGPA Bitstream
Nyan OS (NOS) - STM32 Firmware
Nyan OS Config Tool - Hosted
Nyan OS Config Tool - Source
PCB Schematic and Layout

Final Features

• Lattice FPGA Ice40HX4K to handle debouncing in parallel
• STM32F723 w/ USB 2.0 HS PHY for 8000hz polling
• 1Mbit EEPROM for settings and FPGA Bitstream

Final Build

• 60% Layout w/ 2.25u shift
• Nyan Keys PCB - Oshpark Fab'd
• Kailh Box Jade Switches (3ms debounce)
• Durock V2 Stabs
• Tofu Redux (black)
• PBTFans Spark V2

Assembled




FPGA
The FPGA at the heart of Nyan Keys is used to handle the debounce of each key and high speed serialization of that data so that it can be pushed out over USB. Each key gets it's own timer core. As such all keys are parallel and do not require any diode and such! This also eliminates the need to scan a matrix since everything is happening in parallel.

The switch will arm instantly, then the state of the switch can't change until the debounce count is complete. After the count reaches the max value in [debounce time] the switch can then send over a new state instantly. What this means simply is that unless you are able to press keys at under 1ms you will not have any debounce time since the key would arm and release instantly.

The communication to the MCU that is the USB device is done via a DMA based SPI channel where 9 bytes are requested from the board at a time to represent each of the 61 keys as a binary 0 or 1. The current speed of the board can run the SPI bus at 12.5Mhz and Nyan OS can fetch a full state around 80k times a second. Which is 10x faster than what the USB bus can generate interrupts at. Using the FPGA to handle all of the debouncing instead of the MCU leaves the MCU free to send packets over USB without having to wait for any scanning to complete. This also leads into the fact that each key starts it's count instantly instead of waiting for the MCU to hit that key in it's scan pattern.

STM32F7423
MCU chosen to run Nyan OS (NOS) was the STM32F723 because of the high clock frequency and onboard USB 2.0 HS PHY. Having the onboard PHY means that there is no need for additional chips/traces to get the USB 2.0 pseudo differential signals to the STM32F& MCU. The amount of RAM is 256KB which is more than enough to handle the Nyan OS without issue whilst also allowing scancodes to be sent over to the host at the 8000hz bInterval rate. The onboard flash is also modest at 512KB and is more than enough to hold the code and string values needed. This MCU is overkill for a keyboard but because of the already high BoM costs of this board there wasn't a reason to start clipping coupons.

The MCU also has upto 16 USB endpoints of which Nyan Keys uses 2. As a USB device the Nyan Keys appears as a composite USB 2.0 HS device. The composite device consists of a CDC (Serial) endpoint used for configuration and a HID device.

NyanOS (NOS)

To facilitate an easier interface with the Nyan Keys hardware I wrote a small operating system called Nyan OS (NOS) that is able to be interacted with over serial and includes various settings right now such as getting performance stats to flashing FPGA bitstreams over serial.

Responsibilities

• USB 2.0 HS HID/CDC composite device
• Serial console via USB
• EEPROM master - FPGA Bitstream Storage
• FPGA bitstream programmer - SPI Master
• Status indication - 5 Leds
• Bitcoin Miner - Opt-In Branch Proof of Concept
  
• USB HID Interface @ 8000hz Polling
• SPI Master to FPGA switch serializer and debouncer

NKRO done in a badass way thanks to USB2.0 HS
Normally NKRO has to be done using a bitmap base USB report due to the speed of the USB bus and the USB 1.1 FS limitation of 64 bytes in an interrupt packet.
None of this applies to USB 2.0 HS in any meaningful way because the interrupt packet size if 1024 bytes. Nyan Keys just sends out a HID report with 63 Bytes.
The first 8 bytes are boot compatible so you can use it for your BIOS etc. The sweet part though is with all of that extra space we can just send the rest of the
scan codes after those boot compatible bytes and it works! Nyan Keys with Nyan OS supports full NRKO at 8000hz.

NyanOS (NOS) - Web Configuration Interface



NyanOS can be controlled and configured via the Chrome web serial API. What this means is that chrome can interface with a device over serial. This means the user just has to visit russeree.github.io to start configuring their keyboard.
This website has a built in terminal to use as well as action buttons to ease the process of uploading tuned bitstreams to the FPGA.


Build Gallery

[russeree]

Reserved Slot 1

[russeree]

Reserved Slot 2

[Limentic]

Wow, your project is so cool, I would never even thoughed about using an FPGA just to avoid a scanning matrix, but it make sense.
So overkill, but I think it's a recurring theme over here 

Keep the good work!

[russeree]

Thank you so much! Yeah the FPGA combined with actual USB2.0 High Speed is showing latencies of around .147-.151 milliseconds from a standard HID driver. That result was even better than my initial estimates.

The FPGA is also incredibly nice in the fact that over time users can modify and improve the design to even further reduce latency or maybe add other features such as letting the FPGA remap the keys to various layouts on the fly.

Again thanks so much for appreciating the project! Yes it's unapologetically over engineered and it was so much fun to design and build. As a note I don't even play video games, only use it for coding.

BTW: I love your magic keyboard reverse engineering!

[The0rigina1]

No option for split right shift? I think it's a really important feature for 60 PCBs

[marcusant]

This seems really interesting but I can't do without split right shift and split backspace on a 60. 1.5-1-1.5-7-1.5-1-1.5 bottom row would also be cool.

Maybe something for me to look into if I want to make one from the open source files.

Thank you for sharing!

[lampmk]

... Bitcoin miner??? What is this? Malware?

[russeree]

No option for split right shift? I think it's a really important feature for 60 PCBs

This seems really interesting but I can't do without split right shift and split backspace on a 60. 1.5-1-1.5-7-1.5-1-1.5 bottom row would also be cool.

Maybe something for me to look into if I want to make one from the open source files.

Thank you for sharing!

Soon! Nyan Keys was just a fun project that I made for me and my inspiration was the pok3r keyboard that I have. The good news is that everything is open source so if you want to get a head start on me go for it, the board files are in the Schematics and Design Files repository.

BTW thanks so much for taking interest in this project! It's quite novel, but it does work and is very very fast.

... Bitcoin miner??? What is this? Malware?

Ahh yes you got me ⛓️👮! I inserted and advertised an open source Bitcoin Miner malware option in my keyboard firmware the is provably #ifdef'd out of the build. I was certainly hoping to use the 40 kHash a second performed by the MCU powered by USB to find the next block and win me 6.25 of those bad boys. If it weren't for these pesky Geekhack kids.

Sarcasm aside. The real answer is the I am known on Twitter as Portland.HODL https://twitter.com/PortlandHODL and am also a Bitcoin enthusiast (technology). As such I wanted to use this project as a way to show to my local Bitcoin developers meetup that literally anything can hash a blockheader... Including a 216mhz STM32 ARM processor. The other part of the this was demonstrating the challenges in creating an constructing a Bitcoin block header to perform Sha256(Sha256(80bytes)) on that was valid and would if by the impossible chance did find a hash lower than the target to win the block would pay the user out to their provided address. So yeah I just wanted a cool demo that would just be the icing on the cake to show how powerful that I built was.

So yeah at the end of the day, it can mine Bitcoin because I built this project for me to have fun, and creating whacky Bitcoin stuff is my gig. Second the design is fully open source so if you want to, don't trust me look in Src/Core/main.cpp and see that the hasher is 100% ifdef'd out of any compilation and the user would have to enable it manually.

Here is the line in main.c that shows disablement unless defined as a built constant.
https://github.com/russeree/nyan-keys-stm32-firmware/blob/53ca46ba66cd3dc3077cf32b5bf409d9c628708f/Core/Src/main.c#L151

[nathanchere]

So what you're saying is 60% keyboard PCBs are going to be the new ASIC miner? Mind blown. Where do I sign up?

[nathanchere]

No option for split right shift? I think it's a really important feature for 60 PCBs

Or split space... but looking at the chip placement I think it's safe to say that's never happening. Fascinating project regardless.

[russeree]

So what you're saying is 60% keyboard PCBs are going to be the new ASIC miner? Mind blown. Where do I sign up?

Oh that would so sick! An actual ASIC to get some real hash behind this! TBH the miner was just a proof of concept that anything can has because I love the tech in Bitcoin. None the less yeah anything is possible because the design is opens source!

No option for split right shift? I think it's a really important feature for 60 PCBs

Or split space... but looking at the chip placement I think it's safe to say that's never happening. Fascinating project regardless.

Agreed 100% that was one of the hardest parts about building this board was the routing of traces and chip placement using chips that aren't BGA for hand assembly. TKL would be my next target... But for me as keyboard enthusiast I absolutely love the blend of aesthetics and usability that 60% provides.

BTW very nice to see another Bitcoiner around here https://twitter.com/PortlandHODL

[dorkvader]

This is a very interesting project. To my knowledge there are only a few keyboards which scan all the keys in parallel (and have NKRO without diodes), and at least two of them use PISO shift registers. Offloading part of the KB duty to a co-processor is not new: (imsai did it back in the day WEY did it more recently, etc). There has been a resurgence of this idea somewhat more recently. However, I'm not aware of any other projects using FPGA for this. I think it's a great idea. I have been considering novel keyboard matrix scanning tech (just for fun) and had not considered this.

I would expect the 1xn matrix trace routing to be quite complex, but you managed it well with 2 layers: well done!

According to Digi-Key, the FPGA has 107 IO, and I see there are 4 SPI pins to the MCU: can this idea be extended to a size larger than 60% ?
edit: reading the schematics now. I see between 32 and 40 IO pins which look to be unused, but I don't know FPGA well enough to know if they can be utilized. Of course it's the full bottom edge: hard to route I'd imagine. With it being so physically large, finding space for the FPGA would be hard.

I assume the little squares on the copper are decorative, but if you ground them with vias, then you would make an efficient backside reflector metasurface for approximately 77 GHz radar.

[The0rigina1]

Btw I saw you said this PCB eliminates the need for a scan matrix, does that mean in theory the scan rate is instant?
I would be pretty interested if this is the case, CS and val here I come 

[russeree]

Completed my first order! I call it the "Peter Schiff" because of the case color and the PBT Fans bank account keys.

[russeree]

Btw I saw you said this PCB eliminates the need for a scan matrix, does that mean in theory the scan rate is instant?
I would be pretty interested if this is the case, CS and val here I come 

Correct and each key has it's own debounce core on the FPGA  that updated 12,000,000 times a second! The final data still is limited to the 8000HZ interrupt rate of USB 2.0 HighSpeed but none the less it's still much faster than anything that is commercially available. You can also tune the FPGA to get the minimum possible debounce and I am actively managing switch profiles on the Github repo with ready made bitstreams! This also uses an eager debounce method so there is no waiting for the switch to settle, as soon as it is pressed the signal is sent.

I have been using it with CSS and have been having an absolute blast!

I have 10x of my version 0.9a boards that now support a DFU mode showing up Jan 1! Send me a DM if you want a board or a complete. Boards are going to be 125 and a complete is based on the parts you want to build it with and no I don't skimp on the lube.

[russeree]

This is a very interesting project. To my knowledge there are only a few keyboards which scan all the keys in parallel (and have NKRO without diodes), and at least two of them use PISO shift registers. Offloading part of the KB duty to a co-processor is not new: (imsai did it back in the day WEY did it more recently, etc). There has been a resurgence of this idea somewhat more recently. However, I'm not aware of any other projects using FPGA for this. I think it's a great idea. I have been considering novel keyboard matrix scanning tech (just for fun) and had not considered this.

I would expect the 1xn matrix trace routing to be quite complex, but you managed it well with 2 layers: well done!

According to Digi-Key, the FPGA has 107 IO, and I see there are 4 SPI pins to the MCU: can this idea be extended to a size larger than 60% ?
edit: reading the schematics now. I see between 32 and 40 IO pins which look to be unused, but I don't know FPGA well enough to know if they can be utilized. Of course it's the full bottom edge: hard to route I'd imagine. With it being so physically large, finding space for the FPGA would be hard.

I assume the little squares on the copper are decorative, but if you ground them with vias, then you would make an efficient backside reflector metasurface for approximately 77 GHz radar.

Thanks, yeah there are a few out there, Genesis 8K comes to mind with the PIC MCU, With 100% certainty this is the first keyboard to use an FPGA for parallel input though there has been one keyboard in the past that did use an FPGA but only used it to run a softcore MCU on it. Keys were still scanned and used diodes.

Per trace routing it was an absolute ***** on a 60% but the payoff was huge! I love 60% boards ever since I got my pok3r handed down to me. The most difficult part was the only spot to place the FPGA and MCU was by the spacebar if I wanted to have soldered switches but yeah something like a TLK would be an absolute breeze to make since the FPGA can go between the arrow keys and the page up down section.

Yeah and as a bonus once the FPGA is configured you can reuse the SPI pins again for other purposes.

Yeah 100% you can go to a TLK or even larger, you could also wire some of the less performance critical keys to the MCU, example num pad and you would still be able to get 1 pin per IO.

The little squares are just via stiches. I like them and they do add capacitence to the gound plane so I will take them.

[russeree]

So this weekend I decided to produce a few Nyan Keys PCBs for people who are interested. The versions are the .9e which supports DFU mode to enable firmware updates without the need for a Tag Connect programmer! These boards also have all of the small issues with the first prototype board resolved such as the D+ and D- lines being swapped! This board is Perfect!

With that said price per PCB would be (130 USD or .003 Bitcoin) + Shipping, currently I have 4 available because one sold and will have 5 more coming pretty soon. Spending all day soldering boards was tedious to say the least but with the new stencil method things are moving along very nicely!

So why the high price tag?

• Oshpark PCBs (Made in the USA - Ex. Quality
• BoM cost, No expense spared on components, STM32F723 MCU - Lattice Ice40HX4k FPGA
• Kester R276 Solder Paste + Steel Stencil
• Engineered by me! Hand assembled by me!
• Time - Each board under best conditions is still 1-1.5 hours to assemble and flash.

Thank you all for reading and just send me a DM if you want a PCB!





Thanks,
Russeree