Fatal Exception | Neil McAllister » TAG: Microsoft

October 09, 2008 | Comments: (0)

Microsoft's Touchless SDK pulls a mouse out of thin air

TAGS: Application Development, Microsoft, User experience, User experience

So where are they -- those futuristic computer interfaces, the kind they have in the movies? Most recently, Iron Man's Tony Stark designed CAD models for an entire suit of high-tech combat armor using only his bare hands. So how come we're all still using keyboards and ordinary old mice?

InfoWorld's ace reporter Paul Krill was the first to spot a new project from Microsoft Office Labs and Microsoft developer Mike Wasserman that could bring those next-generation UIs one step closer. Begun by Wasserman as a college project, the Touchless SDK is a set of .Net components that can be used to simulate the gestural interfaces of devices like the iPhone in thin air -- using nothing fancier than an ordinary USB Webcam.

Magic markers
If you've ever seen a digital camera that can automatically focus on the faces in the picture, you already have an idea of how the Touchless SDK works. Touchless-enabled apps can be trained to recognize "markers" -- objects that are easy to spot in the video feed from the Webcam. The motion of the markers in front of the camera can then be mapped to onscreen UI elements.

To see it in action, I installed the SDK's 156KB demo application on an Asus Eee PC 901 with a built-in Webcam. It brought up a video feed immediately. I held my makeshift marker in front of the screen and drew a circle around it with the mouse to identify it, as instructed, and pushed the button to start the demo. A few moments later I was controlling my PC with a Roma tomato.

The demo applications were a far cry from Iron Man, but they were effective in demonstrating the potential power of a hands-free UI. Waving the tomato in thin air, I drew on the screen, scrolled an image back and forth, and played a rudimentary game. The SDK even supports multiple markers, enabling iPod-like gestural controls.

Needs a touch-up
Not everything went smoothly. The demo apps seemed to expect the video feed from my Webcam to work like a mirror, not a camera. But my Webcam doesn't flip the image along the vertical axis. When I move my hand to the right, the marker in the onscreen video moves left -- and so does the Touchless cursor.

The current Touchless SDK still has some kinks to work out, too. For starters, its marker-location algorithm is very much keyed to color. That's probably an efficient way to identify contrasting shapes, but color response varies by camera and is heavily influenced by ambient light conditions. I also found that if a bright red scarf suddenly strolls past the camera, it's likely to be mistakenly identified as a red marker.

The detection routine also seems to require a lot of juice. The demo program managed to soak up 64 percent of my Eee PC's 1.6GHz Atom CPU, and the video from the Webcam soon developed a disconcerting few seconds' lag that made controlling the onscreen cursors difficult at best.

Gestures and beyond
Doubtless these bugs will be ironed out in future releases of the SDK. But if they're showstoppers for you, Wasserman isn't actually the first to experiment with this type of UI. ARToolkit is a cross-platform library that can achieve many of the same things as the Touchless SDK.

Its focus is a little different, however. ARToolkit is billed as a means of enabling "augmented reality," in which real-life objects are enhanced with computer generated elements onscreen -- the first-down marker on Monday Night Football, for example. Because of this shift in emphasis, ARToolkit is more limited than Touchless in some important ways. It only supports square objects as markers, for example.

On the plus side, ARToolkit's code base seems much more mature than the Touchless SDK's. Its marker-recognition algorithm seems both more accurate and more efficient than Wasserman's version, and it registered fewer false positives. It's also portable to more environments than Touchless, which requires .Net 3.0.

Of course, neither of these toolkits can yet hold a candle to the magical gesture-based interfaces of Iron Man or Minority Report. Neither algorithm is advanced enough to register gestures made with bare hands, for starters -- both require some form of easily recognized marker. But they're a start.

Better yet, they're both open source, which means anyone can help to make them better. And even in their present forms, these libraries hold exciting possibilities for desktop application developers, particularly in the area of UIs for computer users with limited mobility. Check 'em out and let me know what you think.

Posted by Neil McAllister on October 9, 2008 03:00 AM

May 04, 2008 | Comments: (0)

Was Yahoo becoming too open for Microsoft?

TAGS: Microsoft

Look at the bright side: Yahoo will save a fortune on business cards and letterhead.

Whether you think the breakdown of merger talks between Microsoft and Yahoo will lead to some other, greater good must depend to some degree, I suppose, on how much faith you have in Yahoo's new Open Strategy. I wonder -- how much faith did Microsoft have in it?

Larry Ellison, the guy who runs the Number Two software company in the world, once said that he never seriously considered buying Linux vendor Red Hat, because "they have no intellectual property" -- anyone who felt like it could take Red Hat's code and give nothing in return. He proved his own point a few months later.

So how do you suppose the guy who runs the Number One software company feels about acquiring a company that has already announced plans to open its APIs and its user data -- arguably its most valuable assets -- to third-party developers, free of charge? It certainly doesn't sound like the Microsoft way of doing things, does it?

Some analysts, including the Wall Street Journal, say Steve Ballmer is just biding his time. He'll wait until Yahoo's stock price tanks in the wake of the failed talks, then strike with a hostile bid when the company is at its weakest. I'm not so sure.

In his letter to Yahoo CEO Jerry Yang, Ballmer notes his decision not to bring his takeover bid directly to Yahoo shareholders. "Our discussions with you have led us to conclude that, in the interim, you would take steps that would make Yahoo undesirable as an acquisition for Microsoft," he writes. He refers specifically to Yahoo's dealings with Google, even going so far as to suggest that further partnership there could scare off engineers. (Sure -- developers just hate Google.)

But might those "steps" also have included Yahoo's new push for openness? Microsoft is interested in the Yahoo brand, Yahoo's audience as a channel for advertising, and Yahoo's online applications as competitors to those of its rival, Google. All this openness stuff is just a distraction -- maybe even an annoyance, considering Microsoft has plenty of its own ideas about how to engage developers.

Doubtless, this saga isn't over yet. If you were in the market for an online service that had a strong brand but a poor history of execution -- one was more comfortable acting like a walled garden than a public bazaar --where would you turn next? I daresay that, for Microsoft, there remain other fish in the sea.

Posted by Neil McAllister on May 4, 2008 09:43 PM