Smartphones and tablets have heralded the touch revolution. Computer mice and on-screen cursors are starting to feel antiquated. And yet there is a problem.
For all its expediency and efficiency, touch still runs mostly like a mouse cursor. So we added more contact-points: enter multi-touch. But now there's another problem, how do we plan out a schema that properly uses multiple touch points?
Well, there are different proposals for that. But perhaps the best answer is something completely different.
Paradigm Shift
We still use touch screens in a manner based off traditional computer mice: single click, single contact. Yes, we gain the benefit of moving in three-dimensions with our hand, as opposed to the two-dimensions a cursor trapped on-screen has. But we weren't taking advantage of our other nine digits.
So we developed multi-touch. Now different amounts of touch contact perform different tasks. One finger clicks. Two fingers scroll. Three fingers switch screens. Etc. This is true touch, a new mode of interface. But we still weren't exploiting the most out of our hands.
Not-So-Modest Proposals
This problem didn't just call for advances in touch screen technology but a whole new paradigm; a unique design schema. A different way of thinking. Different companies and organizations proposed their own visions of a multi-touch schema, like the notable 10/GUI.
They sought to expand the different types of functions able to be performed by varying hand configurations. Ultimately, they proposed a whole new way of interfacing with a computer, one that was faster and more efficient than even current multi-touch schemas.
Unfortunately, all these proposals missed the point entirely.
I Like To Move It, Move It
The reason why touch screens exploded in popularity was because they were supposed to make interfacing with computers easier. Rather than use awkward mice, trackballs, or pens/styluses touch screens promised users the ability to directly interact with their tool.
Other tools we can pick up in our hands. We can twirl them about, hold them at different angles, etc. We are creatures of sense, we like to be hands-on. Touch interface got us closer to the illusion that we were physically interacting with our computers.
The 2002 movie Minority Report revolutionized something, and it wasn't in the world of cinema.
It was in human-computer interaction. Seeing the protagonist manipulate his computer through gestures alone planted a seed in everyone who saw it, into the social consciousness itself. A new way of computer interface, and it was cool, and we all wanted it.
Why? Because it all looked so seamless.
Eliminating the Middle-Man
Not that there's anything wrong or inferior with multi-touch. Or rather, with the concept of touching. We like to touch, we like being hands-on. But what gets in the way is artificial interfaces between us and what we want to manipulate. Mechanics can hold parts in their hands. Coders cannot.
The original command-line interfaces (CLI) were highly abstract, and while expedient in the hands of the skilled were nevertheless highly non-intuitive. Using them required knowledge of syntax, semantics, and vocabulary; essentially a foreign language.
Then we developed graphical user interfaces (GUI) which allowed the layman to approach computers and use them effectively. The mouse and arrow cursor are a highly precise and fast mode of manipulating these visual elements, but demand working through a middle-man. It's awkward.
Touch screens promise the ability to eliminate that middle-man. But unfortunately proposed schemas like 10/GUI's only present more artifice and abstraction; in order to use a computer like that would demand learning specific gestures. It's effectively replaced the middle-man (mouse) with more regimentation, not unlike command-line. Only instead of having to learn a language, it's learning a specific dance.
The Right Answer
The best tool does what you want with the least amount of effort. Having to learn a whole regime, be it specific text-sentences or gestures, is just another obstacle in that tool's usefulness.
The holy grail of human-computer interaction is natural user interface, NUI. Completely intuitive, with almost no learning curve; all because the interaction is based on everyday, ingrained human mannerisms. Normal human gestures, everyday conversational vocal dialog. No regimentation whatsoever.
In short, Iron Man's JARVIS.
That. That is the new Minority Report. That is the bar that science-fiction has set and we are all waiting for reality to reach. A computer visual layout that reacts to us, with us, and that natively understands us.
There's almost no middle-man there. JARVIS understands Tony Stark's casual, everyday speech; even his slang. The graphical elements respond to his movements; not the other way around. A baby could operate that system. It's a computer that works like our minds do, as opposed to having to get our head into the computer's learning curve.
Living the Dream
And now we have something, mass-marketed, that can approach that vision: Microsoft Kinect. Control your Xbox with your body movements. But we didn't just stop there.
We have eye-tracking technology prototypes in development. And of course we have Siri.
Gesture-control. Eye-control. Voice-control. These technologies are still in the refinement stage, but they are coming along. Touch screens had a rocky start too and now look at their prevalence. Laptops even more so.
Many cars, not even luxury brands, feature voice control. It's far from perfect; you need to speak according to its regimented semantics, and its vocal recognition leaves a lot to be desired. Even Siri has a ways to go. We must still work ourselves around these tools, which is the mark of their immaturity.
But the tech is there, it's known by the masses, and it's in their hands. We, the people, the users, are growing up with it and the technology is revolving around us. And as each of these advancements evolve they will hopefully, likely, merge into one awesome interface. One that adjusts to where you're looking, converses with you and responds to your verbal demands, and moves with you.
Keeping It Natural
It doesn't necessarily have to be holographic. It can still be two-dimensional, still on a screen. It's not so much about the medium, but about the interface. One that has almost no obstacles between me and what I want to do, be it a middleman device or a mandatory regimen.
Now the ultimate ideal is a computer that responds to our thoughts. And while we have tech that's starting to react to our brain waves, that's still a field barely in its infancy. For now, just give me JARVIS and I'll be a happy camper.
What are your thoughts on the current abilities of multi-touch, and the possible future of true NUI?