What about non-HID keyboards?

[aegrotatio]

With all the fuss over NKRO and anti-ghosting, keybounce, and the like, why hasn't anyone made a non-HID keyboard that takes advantage of a denser matrix, faster scanning, and ultra low-latency USB protocols so two octopi can use as many keys as they want?

People keep saying it's the cost of making a dedicated controller just for that keyboard, but at these prices, is it really that unfeasible?

[ch_123]

I don't want some ****ty driver that probably only runs on last year's version of Windows so that I can use my keyboard.

[Findecanor]

Has anyone ever done anything that has ever required more than 6-key rollover?

Of course, I can imagine that it could be useful to use the numpad to put the coffee cup on and still be able to use the rest of the keyboard ... :-þ

Better to have something that works out of the box without having to install any drivers. If there is a driver CD, then you can bet that vendors will also bundle a whole lot of useless cruft that will bog down your computer, take up disk space and lack a decent uninstaller. And the average computer user does not even know how to get into the control panel to try to remove it, thinking that removing the icon from the desktop is enough ...
For a new protocol to gain foothold, it needs to be part of the OS from the start, and therefore, go through a standards process. I don't think that n-key rollover is an advantage that is significant enough.

[ricercar]

Has anyone ever done anything that has ever required more than 6-key rollover?

Musical Keyboards mapped to qwerty. Most pianist are quite comfortable using 8 fingers at once.

[itlnstln]

I have Flash 10.1 and Chrome, and I am not seeing it either.

[Rajagra]

In FF 3.6.3 I'm seeing a white square.

I followed the link. As I watched the vid I imagined him slowly morphing into keyboard cat.

Edit> Another idea for 122 key boards:

[ch_123]

Opera, using whatever version of Flash is the latest.

[aegrotatio]

Microsoft solved it with their Sidewinder X4 but like most good technical products they don't support it after launch with decent marketing.

I was about to say my Sidewinder is HID but that's my X6 not the X4.  Silly me.  It is good for games but is a horror show for typing.

[Rajagra]

So who defined the HID standard, and when are they going to bring out HID 2.0 to remove the limitations? USB 3's release would be an appropriate time, hint hint.

[kishy]

ripster, I can't see your video, but I can see rajagra's

Are you trying to embed the 'high def' one or what?

It's virtually unwatchable on my connection anyway, probably for the best.

[Rajagra]

Let's see if this works.

Code: [Select]

]youtube]0yCDbjsNDmg[/youtube]
(With first square bracket reversed or I get a very odd effect!)

[Rajagra]

Must be how the GH site adds it in. I just wrap the youtube tag around the video's identifier string. Works better than copying youtube's provided links it seems.

[kishy]

Oh, what? I thought it worked equally both ways.

Looks like we may have found the problem then?

Testing (same video twice, embedded differently):

(above is just the video identifier)

hl=en_GB&fs=1">

[dmw]

I've thought about this often over the last few years.  The HID standard for keyboards pretty much sucks for anything other than building a commodity keyboard.  One of the big limitations in my experience is the hard-coded connection between keys and their shifted states.  For example, the identifier for the 1 key is something like "Keyboard 1 and !".  So there is no standard way to get a computer to produce an ! except by sending a shift, which makes writing a controller firmware much more difficult than it needs to be if you're trying to have complete freedom in defining your keymaps.

Another problem is the generation of international characters, since there is no real standard for that either.

I think we need to pool our resources, gather together all the development-minded geekhackers, and create a new HID Keyboard standard.  This would include drivers for all major platforms, plus a reference firmware implementation.  If we can collectively build a robust set of keyboard drivers, we can completely ignore the old HID standard, and start building or modifying all our keyboards to the new standard.  Who knows, maybe the powers-that-be will take notice and revamp the real HID standard.

I think that rather than sending "HID Usage IDs" from the keyboard to the host, the keyboard should just be sending Unicode codes.  Then, hypothetically, I could make my keyboard produce whatever character I wanted.

Who's game?

[Zalusithix]

You know, call me crazy, but what's stopping a keyboard from using the standard USB communication method, but integrating a hub into the keyboard and having the one keyboard appear to be multiple keyboards to the system? By segmenting a single keyboard to act like 2-3 in the system, they could effectively get multiple times the number of key-downs to register.

Granted it wouldn't be perfect unless it was broken into (number of keys) / 6 for reporting to the system, and it would require a rather nonstandard bit of hardware on the keyboard end of things, but it would be supported by pretty much every major OS out there without custom drivers being needed.

[dmw]

Sorry, that wouldn't work, because it would be just like a standard keyboard sending reports back to the host at a higher rate, which can already be done. It doesn't solve the N-key problem because the way that the host determines if a key is being held down is by its presence in an HID report.  As soon as the host sees a report that doesn't contain a key that was in a previous report, it assumes the key has been released.  If you send report A with 6 keys, and report B with a different set of 6 keys, the host will think that all of the keys in report A have been released, even if you're still holding them down.  The only way to fix this in the current HID system is to increase the size of the report to more than 6 keys, which would require driver changes.

[Zalusithix]

Eh, I haven't really looked into the way it reports as it's not really a problem that affects me. (I've never run into a situation where 6 + modifiers wasn't enough.) I just saw the thread, hooked up a spare keyboard for kicks, and ran the online rollover detector and got up to 10 keys before I called it a wrap.

Obviously the ideal way is to have a standard that isn't gimped, but the thing about standards, is that they're damn hard to replace. Much like QWERTY, I don't see the HID standard going anywhere anytime soon. Until USB is dead and something else replaces it mandating a new standard, we're stuck with what we've got.

[dmw]

I see your point, but I don't think that we're stuck with what we've got.  There are a growing number of geekhackers building their own keyboards (myself included), or swapping out the controller of an existing keyboard, so why not build our own drivers as well?  We could create our own 'standard', even if that standard is only applicable to a limited audience.

[Morning Song]

I just spent a lot of time trying to come up with a good way of expressing why i thought it's better for the driver/OS to handle keyboard semantics, but for nonstandard layouts, the only sensible solutions besides yours that I can come up with are way more complicated (like a protocol for the keyboard to define its layout to the OS)

But i still think there should be some way of using raw scancodes (having Fn keys not mappable by the OS irks me). So that in keyboards with non-rewritable firmware, the user should still be the final arbiter of what the keyboard does.

[Zalusithix]

It's fine if you have no expectation of it ever taking off outside of the small community. Having custom software and hardware is nothing new in that respect.

Still, it would be no small task to have a custom board created. You'd have to have a design that enough people agreed upon to make it financially feasible. Then you'd have the whole design phase of case/pcb/controller/etc followed by the manufacturing of it. The driver would most likely end up being the easiest part of the whole thing. I mean it's possible, but you're talking a major undertaking. I look at it being akin to what the Open Pandora folks did, but less complex on the hardware front.

(Of course, with such an endeavor, the possibilities are damn near endless. Particularly having a board with multiple (10+) programmable layers would be a world first.)

[dmw]

Heh, it's already done.  10+ programmable layers are possible.  I've got 8 in my keyboard already.  I'm typing on it now.

[dmw]

MorningSong - I've gone both ways myself with this line of thinking.  Imagining that keyboard firmware could be extremely simple.  For example only sending to the host some kind of key id that is completely independent of any mapping.  The keyboard itself would have no concept of a modifier key.  All the mapping would occur on the host.  This would be one way.

But having the keyboard be smarter has its benefits.  For one, you could take your keyboard with you, and, if it's programmed smart enough, it would act the same no matter what you plugged it in to.

[Zalusithix]

Heh, somehow I passed over that article. It still doesn't really say what the maximum layer count is though. Nevertheless, it is quite the board, and I like a number of the features on it. Still, I doubt a design like that would appeal to enough people to get costs down to a point where it would be manageable. It's hard enough to get people around here to agree on a switch type, let alone a layout preference.

Then again, with the matrix layout used on that keyboard combined with the programmable nature, you'd basically be able to put any style key (1,2,3 width, L style etc) in virtually any position and simply remap the board to your liking. It's also quite easy to physically split a matrix design like that down the middle allowing both ergo and standard in the same package. (ala the Kinesis Freestyle) Throw a port on the board for a keypad (or in this case, an anything-pad), and you've got a rather flexible keyboard at your disposal. I'd imagine it'd have to be a non-clicky switch though as you'd get N switches clicking at the same time when a N-width key was pushed. (Perhaps substitute real clicky switches for a pseudo-click via a built in speaker(s) ala the Kinesis Contour line.)

[dmw]

Thanks for the kind words.  I really love this board, though I certainly agree that it is not for everyone (though I think it could be, if more people were able to try it).  

Maximum layer count is a function of the size of each layer and the amount of available flash.  On the chip I'm using, my firmware + 8 layers are using < 10% of the available flash.

The point of my first comment in this thread though, wasn't to build a keyboard that everyone can agree on, but to design a replacement for the HID keyboard standard, and to build drivers for all three major platforms and also to build a reference firmware that could be used/modified by anyone who wants to mod a keyboard or build their own.  

Currently, with my firmware, a Teensy++, and some soldering skills, it's pretty easy for anyone to mod a keyboard to add full programability and multiple layers.  Mnemonix also has a project that includes a controller board and firmware.  So the number of boards that could benefit from a better standard could grow beyond just a few.

The bottom line is, I want to be able to specify, in the keyboard, that key x maps to character y, where y is any character defined in the Unicode standard.  And I want this to work regardless of which OS the board is plugged in to.

[Zalusithix]

Fair enough, but I'm not sure if this semi-standard would be used at all (outside of your own use) without hardware that takes advantage of it being available for general consumption. The amount of people capable and willing to put together a keyboard from scratch is quite few even in a community like this. Ditto to people willing to rip apart their existing board and perform surgery on it that'll effectively stop it from working normally.

I mean, from a DIY perspective, anything is fair game, and none of this matters. From a community perspective, however, where this is to actually be anything of an enthusiast standard, I think a product is needed in some form. (Imagine Arduino without any prefabbed boards.)

[aegrotatio]

HumbleHacker is neat but too much departure from what we already use.

Here's what I want.  Think of your standard PC keyboard.  Cut off the tenkey pad.  Make it clicky, tactile, and NKRO with anti-ghost/keybounce support.  I should be able to lean my arm on the keyboard and every key registers, like a MIDI keyboard does.  That's about it.

[dmw]

Regardless if the audience for what I'm proposing is small, I still think it's a good idea, and one worth pursuing.  Call me impractical, but it's hard for me to see something broken (HID keyboard standard) and not try to fix it.

[quadibloc]

driver that probably only runs on last year's version of Windows so that I can use my keyboard.

That, and not being able to hit DEL (or whatever secret key a fancy computer manufacturer might have chosen) to get into the BIOS.

If, however, the keyboard can work as a standard HID USB keyboard, but with a driver, can switch to make full use of its NKRO capabilities, and the manufacturer has a good reputation for providing updated drivers for its older products, that would be different.

[Findecanor]

OK then. You guys have convinced me that the 6-key rollover limit should be lifted.

One of the big limitations in my experience is the hard-coded connection between keys and their shifted states.  For example, the identifier for the 1 key is something like "Keyboard 1 and !".  So there is no standard way to get a computer to produce an ! except by sending a shift, which makes writing a controller firmware much more difficult than it needs to be if you're trying to have complete freedom in defining your keymaps.

You are not suggesting that keyboards should send ASCII codes, are you?

I don't see any other way to do it than to also send a shift here without sacrificing the ability to have a user-defined (user-selected) keymap on the host side.
...  But perhaps we could alleviate the problem a little bit by separating the scan-codes for modifiers and key+modifier.
There are three actors in the system: keyboard, driver and application.
Applications in general do not precisely receive ASCII codes or scan-codes, but a third type: "key-codes" (or whatever it is called in the OS), which is a set that includes both all ASCII codes and the various non-ASCII keys such as modifiers keys, cursor keys, Enter, Escape, etc.

What if, instead of the keyboard sending a normal { 'Shift down', '1 down' } to the host,
it would send { '1 down with modifier Shift' }, where the message "with modifier Shift" is significantly different from "Shift down".
The operating system would then translate '1 with modifier Shift' according to the keymap and provide the message '! down' to the application -- but not the message 'Shift down'.

But you don't want to hard-code which keys are modifiers, so you would have to send all keys that were held when the key was pressed.
So, in conclusion, I suppose that what you want to transmit over the line would be single messages of the type:
( "up"|"down", )* )
But you would still have to set a limit to the size of the array of modifiers. What should be the new limit? What is enough? 256-kro?

[AvengeR]

Because the Chinese and Taiwanese are as creative as lumps of coal.

The Topre engineers are working on a low cost HHKB but can't find a DIP switch big enough.

The Germans are too busy watching the World Cup.

Microsoft solved it with their Sidewinder X4 but like most good technical products they don't support it after launch with decent marketing.

Apple has moved on to N-Key touch screens.

is sidewinder x4 overall good?
edit: actually I meant x6 because I thought the x4 was the one with detachable numpad. Nvm

[aegrotatio]

I think both Sidewinder boards appear as HID but I could be mistaken.  The specialness of the X4 is done in software in conjunction with a modified matrix and diodes, from what I read of the literature.

[dfj]

Once windows boots a different protocol can be presented.

I have my terminal F running through PS2, currently, but I don't like this setup as it is a total pain under win 7 64, (requires booting without signature checking, and swapping a kb while the machine is sleeping - total ugly workarounds).

So, I'm jumping back onto my USB nkro HID code, on the teensy first. I'll give a headsup when I have something simple ready. After that, we'll see if there isn't some way to bring in mnemonix's layering stuff. Incidentally, my code is not nkro, it is more like 112key rollover, but, you can choose _which_ 112 keys you want to be independent, and which of the remaining keys will get stuffed into a 2kro aux field.
This HID uses a 16 byte message (via two packets), so can support up to 112 in nkro (112 = 14 * , and up to the remaining HID slots in 2kro (256 - 110 - some reserved slots... )

Heck, mebbe the humble will be willing to share some code if he likes what I've done with the HID.

until,
dfj

PS:

terminal is supporting at least 15kro via PS2, here is a snip via autohotkey, I needed to edit it to remove repeat dups, so make of it what you will...

Code: [Select]

VK  SC    Type    Up/Dn    Key
-------------------------
57  011         d    25.74    W
41  01E         d    0.00    A                  
51  010         d    0.00    Q                  
52  013         d    0.08    R                  
45  012         d    0.03    E                  
44  020         d    0.03    D                  
46  021         d    0.00    F                  
59  015         d    0.00    Y                  
47  022         d    0.00    G                  
54  014         d    0.02    T                  
49  017         d    0.06    I                  
55  016         d    0.00    U                  
4A  024         d    0.00    J                  
48  023         d    0.08    H                  
41  01E         u    0.06    A                  
51  010         u    0.03    Q                  
57  011         u    0.02    W                  
46  021         u    0.03    F                  
52  013         u    0.03    R                  
45  012         u    0.00    E                  
44  020         u    0.00    D                  
47  022         u    0.00    G                  
48  023         u    0.00    H                  
54  014         u    0.00    T                  
59  015         u    0.02    Y                  
49  017         u    0.94    I                  
55  016         u    0.02    U                  
4A  024         u    0.00    J                  
74  03F         d    3.25    F5                
Press [F5] to refresh.


[JBert]

MorningSong - I've gone both ways myself with this line of thinking.  Imagining that keyboard firmware could be extremely simple.  For example only sending to the host some kind of key id that is completely independent of any mapping.  The keyboard itself would have no concept of a modifier key.  All the mapping would occur on the host.  This would be one way.

But having the keyboard be smarter has its benefits.  For one, you could take your keyboard with you, and, if it's programmed smart enough, it would act the same no matter what you plugged it in to.

Now how is this different from the way PS/2 or USB HID does it? AFAIK the OS still remaps this when you have a layout configured.

[dmw]

Now how is this different from the way PS/2 or USB HID does it? AFAIK the OS still remaps this when you have a layout configured.

You're right that both current methods work by sending a code back to the host, and that the host then decides how to interpret that code.  The difference between the current state of affairs and what I'm suggesting has to do with two things: modifier keys and non-"standard" keys.

In my first example, I showed that the only way to produce an exclamation point is to send to the host both a shift and a one.  That is, there are certain characters that by convention are intimately tied to other characters with the use of the shift modifier.  This connection is arbitrary and makes it difficult to make a fully remappable keyboard in firmware.

The other thing is how to produce non-standard characters.  This means international characters such as é, œ and special characters such as math symbols.  For these characters, there is no standard at all.  There are different conventions depending on the language and the OS.

Both of these problems can be solved with the same solution, a solution with two main approaches: make a dumb keyboard with a smart driver, or a smart keyboard with a dumb driver.  This is what I'm interested in discussing.

[dfj]

You're right that both current methods work by sending a code back to the host, and that the host then decides how to interpret that code.  The difference between the current state of affairs and what I'm suggesting has to do with two things: modifier keys and non-"standard" keys.

In my first example, I showed that the only way to produce an exclamation point is to send to the host both a shift and a one.  That is, there are certain characters that by convention are intimately tied to other characters with the use of the shift modifier.  This connection is arbitrary and makes it difficult to make a fully remappable keyboard in firmware.

The other thing is how to produce non-standard characters.  This means international characters such as é, œ and special characters such as math symbols.  For these characters, there is no standard at all.  There are different conventions depending on the language and the OS.

Both of these problems can be solved with the same solution, a solution with two main approaches: make a dumb keyboard with a smart driver, or a smart keyboard with a dumb driver.  This is what I'm interested in discussing.

I skimmed this thread the first time, so I may be making a couple of points that were well expressed already - apologies to the original authors.

DMG: So, as you are asserting - and I surely agree, the current crop of HID  specs for keyboards define way to  set codes representing keys, rather than say, characters.

It seems clear to me that the keyboard really _must_ have this functionality, at a minimum, for it to be useful for more than generating a sequence of unicode characters. Taking gamers and those who are wanting to remap ctrl and caps as simple examples: one wants to be able to represent the state of keys, rather than just events.

But - I'm not saying that a keyboard should not be able to send events. HID kbs already do for certain situations (errors, amusingly), but just not for keys.

So - if there were a way to describe, say, unicode mappings to a key event, and if the HID were extended to support this, then one could get more general semantic mappings sent reliably to different computers, say macs, and windows, in particular - linux, well - we can write the drivers for that easily.

 So, given that drivers are much easier to write than firmware, and that config files are much easier to write and store on computers than firmware... the question for me is to find the edge cases that drive our motivation to consider stuffing the decode into firmware.

  Roughly speaking, it is straightforward to write additional linux driver
layers and to map keys to arbitrary commands : e.g. generate this unicode when I hit this key, when in this state.
  Under windows, it is less simple, and I can't say I enjoy it, but drivers are still more manageable as they get large... mumble, signing, installation notwithstanding.
 On a mac, the driver situation is more irritating still, my understanding (please correct me, on the workarounds, I know there are some) is that one must submit the device to apple and apple will largely distribute drivers for it instead of the drivers coming with the device.

 Immediately a couple've issues begin to motivate ourselves to explore the firmware option again. First, multi-OS support, bringing one's keyboard to work, a friend's and to one's home workstation or entertainment centre is a lot more possible (folks want to do each of these, not me.. other folks) if we don't need to install drivers onto each one when we attempt the attach. Secondly, Open Source drivers are much more irritating to distribute for mac and 64 bit windows due to signing and licensing constraints. Finally, not having to write and QA a different set of drivers for each version of each device and OS release that is desired to be supported is kinda nice.

  The biggest 'gotcha', of course, is that there is not a widely supported HID definition to allow one to send arbitrary unicode to the Host.

With Windows and Linux, if you build a driverless device that conforms to the spec, future versions will often be more than willing to support it _for you_, especially if it is an 'oddball case' that makes heavy use of some interesting feature. If it requires drivers, you can kiss that happenstance goodbye without re-certification.
  Apple doesn't support backwards compatibility with their own hardware, nevermind other folks'. Just sayin', heh.

So - At this juncture, and for the forseeable future that a well-made device might be in use, I do lean towards getting as much into the firmware as on can manage. The device being able to flash itself from a file on a card, rather than requiring OS support, is nice - though it could also surface a tiny FS for the OS to write firmware files to.

It is an interesting question - but you'll notice that the vast majority of coders here have gone the firmware route, just as soon as they could manage it. The PS2 terminal boards under windows case being one of the oddball exceptions where firmware was not very viable, so a driver solution was attempted. Mind, I am _not_ happy with this state of affairs, and am working on the PS2->USB HID source again (for the teensy, USBkey, etc... and compatible)

As far as a non-HID device, goes... well, that is the driver option. Might as well support the basic HID in hardware anyway, then put the rest of the goo into pass-through and hook drivers so at least the effort for each platform will be sane, albeit still high. Extra state can be exposed through the HID, still. It will merely require a bit of driver to catch it. Wont need to be a _kernel_ driver, though, so user mode drivers and programs can do the deed on most platforms. With luck, backwards compatibility will reduce the churn on rewriting the thing. Personally, I hope not to find a reason to do this, but I have terminal boards... the urge to _fill_ the thing with weird hardware and clean up my desk is tempting... it has so much space inside, and steel armor... mmmm

thanks,
dfj

PS: the PS2->USB is my current 'main project' so it should be up in a rough form soon. First for the USB avrs, then hopefully for some of the others.

[Konrad]

The reason I bumped into GH in the first place was to get info while starting my homebuilt keyboard controller. My original mission was to get unlimited simultaneous keystrokes.
 
I've chosen an overkill MCU: a $6 Parallax Propeller part with more than enough oomph and I/O lines for the job.
 
I'm still hammering out my debounce bugs on PS/2. But the problems I'm currently encountering are probably going to exist on USB as well.
 
In short, no matter how much n-key the keyb controller and matrix can support, a 6' cable seems to introduces too much latency to make baud rates > ~9600 practical. Makes timing and debounce hellish if you try to stuff too many codes down the pipe. Sadly, I'm currently able to get only 4 (max) simultaneous keystrokes at a time; when I push more I usually lose some or actually see ghosts.
 
Although USB is capable of better signalling, it has to go through HID translation into "PS/2" coding before the host controller can process. HID is a software engine which adds it's own (and the OS's) timing quirks into the equation.
 
Somebody asked why you'd need bigass n-key?
 
1) Sometimes (usually very bad times) I find myself needing to issue like 10 simultaneous commands during fps kill frenzies. This usually doesn't work out so well. Maybe more macho antighosting can improve my odds a little. (Maybe not sucking as much in these games would help too, but one issue at a time.)
 
2) Sometimes, during bursts at highest typing speed, I notice that my properly typed input gets mixed up by the keyboard. It scans all of the keys, but the timing is so close between them that the order gets a bit mixed up. Yes, no doubt many of these are my "user-error" typos, but often enough they aren't. Anti-ghosting seems to becomes an issue when the keyboard limits are approached.

[Rajagra]

I'm a little confused. Are you sending signals down a 6' cable that need debouncing at the other end? Normally the controller is a few inches away from the keys and debouncing of electrical noise from the switches is handled locally. Or do you mean a different kind of debouncing?

[Konrad]

No, all debouncing is done in the keyboard controller - as you imply - it just cleans up all the keyboard-local switching noises. The signals being sent down the cable are all clean 8-bit digital codes.
 
The cable (in my case) is 6', a fairly standard length although many keyboards use shorter runs. Sure I can just use a shorter cable, but if cheapass keyboards can make 6' work then bloody well so can I.
 
The problem I'm finding is that this cable length is a lot of linear capacitance, when considered in 5V serial-signalling terms. This means errors and latency are introduced between the two controllers; the capacitance sort of "blurs" the signals between the controllers at each end of the cable (the host is actually the one in charge; the keyb only sends signals when the host requests, not necessarily when the user presses them). This all means that there's sort of a maximum effective speed where anything faster starts causing traffic jams. For me, for now, this rate is like 4800bps; for most PS/2 keyboard makers it seems to be established at 9600bps, not an absolute limit but a seemingly common arbitrary one. The (usually 16.67MHz) host controller can handle much faster signal rates. My keyb 80MHz controller can handle much faster signal rates.  Common low-cost keyb controllers are usually matched to the capabilities of host controllers, cheap controllers are often slower.
 
Anyhow, I'm going to experiment a bit on the cable itself. Thicker gauge, coax shielding, twisted/untwisted pairs, etc ... I don't expect miracles but I might get a little more leeway. I was just hoping somebody else might have another solution for this problem.