Contents1. Inspiration for the keyboard+mouse
2. The challenges and solutions
3. Status
4. Goal
5. Vendor information
InspirationThe goal has been to make a keyboard and mouse wearable without too many compromizes.
So: fast, small learning curve (no chords), lightweight & comfortable, affordable.
Even more important than the device itself: how it can change the way we work. It’s clearly opening the door to new opportunities.
In the past, I have been using the SafeType vertical keyboard. Though their solution was radical, it was not enough (not good enough, not radical enough).
It inspired me to go hands free. This gives more ergonomic possibilities and it enables many more use cases. My vision for the future is also that AR, next to the immersive 3D applications, will be commonly used as a 2D projected display (though the Hololens may not be around to experience this)
The challengesThe keymapAt first, I didn’t know if it would be possible at all to reach 87 or so keys with a hand-held controllers. Clearly, I would need several Eureka-moments. Initially I was following the path to use combination keys with thumb and regular fingers, a bit like Layered keyboards but then even further down that path. That didn’t really work as pleasant as hoped for, which lead to a major breakthrough: “combining hand tracking with regular keys”.
1. For reference: a regular keyboard divided in 2 halves. In blue the keys that are by default near the finger tips (as is with a regular keyboard)
2. How the hand tracking works.
The non-default keys (here grey) can be reached using hand tracking. Equal to how you shift the hand a bit to the left on a regular keyboard to reach e.g. the "Tab", such a small move is now detected and translated to a shift in assignment. Alternatively a shift to the right can bemade to reach e.g. the 'T' or 'G'.
This is discrete logic to detect a relative row shift or column shift, so not something that requires accurate absolute positioning and fine motor control.
Further, there is a separate mode distinguishing mouse usage and typing. There is also a ‘numerical mode’, changing the assignment to typical num-pad and related functions.
Hand tracking mouse
Based on Accelerometer, Gyroscope and Magnetometer with sensor fusion software. Pointing to the screen is being tracked. Actually, relative movements are being tracked, so actually pointing to the screen is not required. These can be relative moves from any preferred arm positions.
Different sensitivity levels are possible. Lower sensitivity for more accurate moves. Both hands have cursor control functionality, allowing for hand specific sensitivity. This also makes multi-touch possible. In addition, calibrations will be avoided by using self-learning algorithms.
The keysSome things became immediately clear: it should not be a glove or wristband. It should be a rigid ‘table’ with physical keys, which leans on the palm of the hand. (accepting that the palm of a hand will of course never be as steady as the back panel of a desktop keyboard)
And since it’s a two-handed device, this implies that it’s not possible to grab a coffee anymore without getting the device out of the way. A major step back, but from here only steps forward.
Another challenge has been to reduce the finger travel between the nearest and the furthest keys. Here, using multiple key positions on one key (stem+shaft) has been a major breakthrough.
On-screen assistantIt’s possible to show/hide an on-screen assistant, which is more or less a ‘cheat sheet’ on the screen when somebody wants to look up certain characters.
The fasteningThe fastening has been a major challenge. It has been going back and forth between multiple concepts. When a strap needs to be closed when putting on a device, this is not ok. A spring wire steel clip does the job of keeping the strap in place when the hand is out, making it possible to simply shove the hand in.
AdjustabilityHands exist in many different shapes and sizes, so many adjustments options are needed. The image shows how we are approaching this.
It may very well be the case that additional adjustment options will be added over time.
StatusSeveral years I have been combining wearable keyboard R&D with a regular job. Since a few years now, I have been fulltime working on the development of a prototype and on more funding related distractions. With the highly appreciated help of a group of old colleagues, freelancers, trainees and volunteers.
After many iterations, we are now approaching the point that there is a prototype that is market-ready. Still much to do and many 3D printing roundtrips needed before moving to a version that is based on molded parts. Industrialization is a major challenge, accurate hand tracking software, hand gesture software are also far from trivial.
GoalBring a fully working device to the market, which really works, performs fast, and doesn’t have much of a learning curve. At this moment, we target the ones that are willing to give it a go. We wish to grow our group of sympathizers and ambassadors prior to product release. In the long run, we want to be seen as a legitimate company in a new product category, that can also make legitimate claims about long term health effects.
We are actively looking for people that want to give feedback. Keyboard knowledge is great. Experience with medical conditions like e.g. RSI, osteoarthritis, higher spinal cord injury is also welcome. (Whether as a patient or as a healthcare provider)
Vendor informationWe are Netherlands based. You can find us here:
www.typeware.tech. The web shop is the only place for pre-orders or to sign up for try outs.
So...what do you think?
