Leap Motion controller - motion detection as an input device (air keyboard)

[darkenergy572]

Have you guys seen this thing yet?  Basically, it's motion detection for hands / tools.  It should be out soon; it's supposed to start shipping in May.

Video:
http://techcrunch.com/2013/03/12/leap-motion-michael-buckwald-demo/

Official Site:
https://www.leapmotion.com/

I can imagine this being used to achieve some kind of air keyboard...
Maybe attach it to a flat board with labels for hand position consistency, and type.  One could probably map hand gestures to hotkeys (invert the hand to copy, make a fist to paste, /shrug).  Definitely interesting stuff.  I'm pretty excited about this.  Maybe more so than the Emotiv EEG headset (brain-computer interface), which I want to eventually try (but damn it isn't cheap).

For reference, Emotiv EEG headset:
http://emotiv.com/eeg/

(Note: Emotiv is a bit wacky with their licenses.  Anyone considering it should definitely read up very carefully on the developer and researcher licenses before buying.)

Anyway, point is, I'm wondering if an "air keyboard" would be very ergonomic.  Not sure if it'd be more of a pain without the fingers being supported by keyboard keys.  Any ideas?

Edit: One could likely extend the air keyboard app so that you can go beyond inputing specific keys - you could use sign language to output entire words at once (perhaps if interference between typing and sign language gestures were an issue, a foot pedal could be used to swap modes).  Of course, someone might need to code up a sign language gesture interpretation program.  Just need to deal with the variable rotation of the hands, then detect the angle of the joints and distance between fingers to determine if the hand falls within the thresholds for a given gesture.

[davkol]

http://geekhack.org/index.php?topic=32035.0

Anyway, point is, I'm wondering if an "air keyboard" would be very ergonomic.  Not sure if it'd be more of a pain without the fingers being supported by keyboard keys.  Any ideas?

No feedback would be probably an issue.

[darkenergy572]

No feedback would be probably an issue.

Hmmm, maybe one could engineer some kind of super gentle hand prop that will only need to achieve one thing, maximum comfort for typing feedback.

[laffindude]

I pre-ordered one so I can just wave my hand to do shortcuts so I don't have to move my hand to the mouse. If it doesn't perform as good as it is hyped to be, I'll just map music function to it and use it as a "remote."

[darkenergy572]

I pre-ordered one so I can just wave my hand to do shortcuts so I don't have to move my hand to the mouse. If it doesn't perform as good as it is hyped to be, I'll just map music function to it and use it as a "remote."

Heh, at that size, I think it'd be potentially useful, even if it can't do finger motions too well.  Use it as an air mouse instead.  That'd be worth $70 by itself.  Still, I hope they get things sorted out with handling those finger motions accurately (...if it has trouble with 5 fingers, both hands might be too much for it.  Maybe one could use two of them.  Might need some kind of wall/barrier to block the view of the devices from intersecting each other).

[ASETNIOP]

I'm in the developer program and I've been working on an "air keyboard" much like you're describing - I've attached my latest video below.  As noted above, the lack of tactile feedback is definitely a problem - the "air keyboard" that I put together works based on a virtual plane and responds to fingers crossing through it.  My scheme (ASETNIOP) is actually intended to function on a tabletop, with the device in a horizontal orientation; the action of touching the tabletop is what generates a response (and the feeling of your fingertip touching the surface provides the necessary tactile feedback).  The LEAP isn't ready to handle the horizontal orientation yet, though.  The last upgrade to the device was a HUGE improvement in terms of tracking, so I'm hoping for a similar leap (groan) in functionality before consumer units ship in May so I can implement my solution the way it's intended.

[ASETNIOP]

Edit: One could likely extend the air keyboard app so that you can go beyond inputing specific keys - you could use sign language to output entire words at once (perhaps if interference between typing and sign language gestures were an issue, a foot pedal could be used to swap modes).  Of course, someone might need to code up a sign language gesture interpretation program.  Just need to deal with the variable rotation of the hands, then detect the angle of the joints and distance between fingers to determine if the hand falls within the thresholds for a given gesture.

A lot of folks working on these systems are envisioning using them with sign language, which I don't think is very practical for a couple of reasons - first you're asking the user to learn an entirely new language, and second you're running into the same kind of interpretation issues that you run into with speech recognition. 

My own contribution to the "shortcuts" realm is a set of stenographic combinations; basically hotkeys that are built from the letters/chords that make up the words (for example, for "and" you'd press the left pinky (A), left ring and middle (D) and right index (N) all at the same time). 

[tp4tissue]

I'm in the developer program and I've been working on an "air keyboard" much like you're describing - I've attached my latest video below.  As noted above, the lack of tactile feedback is definitely a problem - the "air keyboard" that I put together works based on a virtual plane and responds to fingers crossing through it.  My scheme (ASETNIOP) is actually intended to function on a tabletop, with the device in a horizontal orientation; the action of touching the tabletop is what generates a response (and the feeling of your fingertip touching the surface provides the necessary tactile feedback).  The LEAP isn't ready to handle the horizontal orientation yet, though.  The last upgrade to the device was a HUGE improvement in terms of tracking, so I'm hoping for a similar leap (groan) in functionality before consumer units ship in May so I can implement my solution the way it's intended.

LOL..

use the fingers to average out a "moving" plane that will be relatively tracked.. then continue with your plane cross method.

[sordna]

The future of typing is doing it slower? Ha ha, just kidding, it's pretty interesting actually! I might get one of this when it comes to BestBuy.

[hasu]

Great. Looking forward to seeing its progress. Keep us updated.
Also chording is interesting.  I'll have to look into keyboard chording methods.

[SP534]

Yeah I was playing with the idea of an air-board for the LeapMotion too, I just hated the fact that I wasn't actually pressing anything. I personally think it could really spark a fantastic gesture-based UI though if it was done right. Been playing with a few ideas but with college exams coming up I don't really have the time to dedicate to it right now, unfortunately.
What ASET is doing looks really awesome though, kudos!