I’ve recently been involved with an iPhone project where we are doing a few custom UI controls, and it’s definitely proved a learning experience about the difference between designing for a computer screen and designing for the iPhone screen (either the current one or the upcoming iPhone 4 Retina Display screen).

One thing I’ve learned has to do with the characteristics of the iPhone screen, and how that influences User Interface Design choices. Over the years, I’ve become used to the what it takes to show a change on a computer monitor, which is to say, the degree to much you need to change the colour, shape, or scale so that it’s obvious, even if the user looks away for a second before the change occurs and then looks back.  This might apply to an object in its selected and unselected states, or the addition of something new on the screen, or perhaps the enabling or disabling of a button or other element.  At first, I thought this was due to the dots (or in this case, pixels) per inch of the iPhone versus computer monitors. Monitors are usually somewhere between 72 PPI (Pixels Per Inch) and perhaps 200 PPI on the best equipment. The IBM T220/T221 LCD monitors marketed from 2001–2005 were 204 PPI, and they probably set the standard for a while. These days, a 20-inch (50.8 cm) screen with a 1680×1050 resolution has a 99.06 PPI, and a garden variety Macbook (not the higher end Macbook Pros) has 113 PPI (Wikipedia has an article on how this is calculated).

However, the iPhone PPI is listed at 163 PPI, which although it’s on the high side, is certainly not significantly higher than a typical computer these days. The difference, then, must be the size of the screen. In the case of any iPhone screen 2G, 3G, 3Gs and 4G, it’s a 3.5 inch screen (compare that to the aforementioned 20-inch, and now we’re talking different.)

It might be obvious, but what I’ve noticed is that the amount of change you have to make in order to be noticeable is far more on the iPhone’s screen. The contrast must be greater, scaling or moving an object between one state and another has to be larger (or farther), and as a corollary to this rule of thumb,  it’s easy to miss subtle changes.  Several times during development of the app we’re working on, I had to report to the graphic designer that I was working with, that a selection style wasn’t distinct enough, or that a small detail of a button, such as a downward pointing arrow, had to be rendered with higher contrast (the UI had a lot of grey objects, and some of them had white or darker grey overlays).

I think the easy way to think about this is the analogy of greasepaint. What’s greasepaint? It’s the traditional makeup that actors wore (and has now been superseded by more modern stage makeup) that helps to compensate for both the washing out of facial features by the bright theatre lights, as well as help audience members to make out their faces, even though the actors were farther away (and hence, smaller in the eyes of theatregoers – perhaps the equivalent of being 4 or 5 centimeters tall depending on how far away from the stage they were sitting). I remember going backstage to a dressing room after the Play or Opera was over, and was always struck by how odd the performers looked before removing all of that extreme makeup, which brought out cheekbones or encircled their eyes (like a Raccoon, I though!).

So User Interface Designers working on iPhone apps, remember, the computer screen is the dressing room, and the iPhone screen is the stage. Don’t forget the greasepaint!

I just got back from one of our local Apple Stores and the iPads on display had quite a throng around them.  I didn’t check, but suspect that they are probably  sold out for today. My visit got me thinking about how to explain why I think the iPad is both so successful (and this is not just a belief, it’s a fact: Apple has already sold a million of them, and this past Friday they first went on sale in the rest of the world, (including here in Canada), and why Apple has once again filled a need that people didn’t know they had in the first place.

First, How to Define It

In describing what the iPad is, it’s easy to get caught up in what it doesn’t have, since that may be what strikes one at first; There’s no keyboard, no mouse or trackpad, no monitor stand, and all of the rest of that stuff that goes along with the experience of using a computer or connecting to the Internet.  That also includes a desk or table, chair, mouse pad (or with the advent of optical mice, at least a surface for moving the mouse on) or the various power, video and network cabling, external hard drive or optical (DVD) drive. There’s also a lot of upkeep and maintenance that has been taken away from the iPad;  there’s no anti-virus package that you might be reminded to get shortly after starting it up, no place to get software except the built-in iTunes store. You don’t have to worry about defragmenting a hard disk (there is none – it’s solid state memory) or even emptying a trash can on the screen to free up disk space. While all of this does get one closer to the uniqueness of the iPad, it circles around the issue somewhat, which I’ll get into in a bit.

It’s also common to define the iPad as just a large iPod Touch or iPhone, since those are devices we are already familiar with. The fact that Apple chose to use a very similar operating system and launching screen to the one on those devices only serves to bolster the opinion that the iPad is merely a larger version of these other gadgets, something I’ve heard especially from people already familiar with those existing products. I think this is an incorrect assessment, simply because there are activities and media that are obviously far more suited to the larger form factor (like watching movies) than the smaller ones. A wall clock is not merely a large wristwatch. It’s a completely different, but related timekeeping object. But again, I think this is looking at the wrong thing.

To paraphrase the philosopher Ludwig Wittgenstein, don’t look for the word, look for the use. Rather than try and define the iPad by what it is lacking or what it appears to be based on, define it by how it’s used. It’s here, I think, that you get to the really interesting and exciting thing about the iPad, which is the user model, or the totality of the experience under which it’s used.

Many of the most revolutionary technological advances are ones that embrace a new user model. Wi Fi and laptops freed people from being tethered to a single office or desk. The new 3G networks and hardware to connect to them on a Netbook allow one to be connected to the Internet not just in a Café with a local wi fi access point but perhaps sitting outside, by a babbling brook.

The iPhone’s size and weight meant that you didn’t have to be sitting down to use it. You could be waiting in line, walking, or sitting  in a seat on the bus or a car. In fact, the iPad is the first computer  that is almost intended to be used while slouching. It’s not a desktop or laptop;  it’s a loungetop! The idea that a computer is not necessarily for work (the Desktop and Laptop computers are ostensibly for that purpose) or for communication (all of the above plus the smartphone or PDA  - Personal Digital Assistant, a term coined by another Apple CEO –  plus phone) leaves the iPad a computer for casual use, mainly media-consumption with some email and web surfing. One could certainly do work on an iPad, and no doubt, some people will dedicate themselves to using it for their work tasks, but the iPad is first and foremost, the first computer designed to be used while a user is sitting back comfortably. That’s probably the big (if not one of the biggest) deal, in my opinion.

The lack of all of those other items (keyboard, mouse, external drive, cabling) meant that there is less to distract the user from the touchscreen and the content displayed on it. People often describe the experience of using an iPad as qualitatively different; that there is no longer ‘something’ in the way, between them and the Internet. While the day has not yet arrived where we ‘jack in’ directly to the Internet, the iPad comes a step closer to that consensual hallucination.

The iPad as Harbinger of a new Age of Human Control Interfaces

It’s even more interesting to take note of the fact that Steve Jobs conceived of the iPad first, and then realized that they could use a smaller version, with some of the scrolling behavior, as a way of building a telephone and internet device/iPod. The pure idea, that of a simple, flat, sheet of glass that displays content and interacts with the user was the original idea. You could put that foundation under any other gadget. People will now expect the iPad/iPhone touchscreen interface with it’s combination of mimicry of physical scrolls and easily changed collection of buttons or controls depending on the context as the default user interface for any number of other technologies. Your car will have a small iPad screen built into the dash (someone has already installed one, according to one of the tech blogs). You’ll set your thermostat or fade your lights with one of these glass interfaces, and you’ll program your microwave, dishwasher, or even toaster with one, once the technology becomes cheap enough to use everywhere.
By jettisoning the clutter and encumbrances of computing, the iPad pulls the rest of the world into an intelligent and software-driven set of controls. Physical knobs, along with raised physical buttons, will only be used where absolutely necessary. As for the rest, we all Pad.

Though the drawing doesn’t show it, Englebart’s mouse, which was one small part of Engelbart’s a larger project, aimed at ‘augmenting human intellect’ had 1 button. The drawing mainly shows how the block uses multiple rollers, which sense which way the mouse is being moved in terms of X and Y coordinates.

When Apple shipped the first Lisa computer (and of course, the first Mac) , the commandment that ‘Thy mouse shall have but  1 button’ was spoken to the masses. On the other side, the X-Window System, and the IBM PC mouse had multiple buttons (2 or 3). The two to three camps dug in for years, each claiming the ergonomic, moral or practical high ground over the others. The antipathy between the 1 or many buttons groups continues to to this day, even if this division is no longer the case. Many people believe that Apple has stayed true to their gospel and only makes or supports a 1 button mouse, but the unforutnately named ‘Mighty Mouse’, which shipped in 2005, supports multiple buttons virtually rather than physically (you click on one side or other other to simulate one or the other button), and also has a roller ball and 2 physical side buttons, providing no fewer than 5 buttons. The proliferation of mouse buttons, sometimes 2, sometimes 3, sometimes 5 or more, depends on the system and software one encounters. Some trackball devices have had 5 buttons that effectively provide even more control messages by allowing a different kind of click from different combinations of those buttons. Apple’s latest mouse (the even more unfortunately named ‘Magic Mouse’ – what group is coming up with these names?) even goes farther, making the entire mouse surface another control surface in and of itself, like the trackpad on a laptop. This, to me, is akin to attaching a steering wheel to the top of a gearshift, or some other bizarre composite, but I’ll have to withhold judgement until I try one, even though it sounds like the Industrial Design equivalent of a Turducken.

The point is, complex gestural movements, involving more than a simple click (or double click) on a pointing device have pretty much been adopted by all computer makers, with at least an accepted level of complexity, although for the most part, a user can work up to that complexity, by moving from simple gestures to more complex ones over time, hence the idea of a short cut to a function instead of making  that function only executable from a complex gesture.

As a friend of my parents puts it, ‘Anything worth doing is worth overdoing’. I shouldn’t be surprised by what I thought was certainly a post on The Onion, but no, it was serious, and it was the Open Office Consortium who was proposing this mouse:

Holy Roller, Batman! This thing is certainly the other end of the spectrum from the mice we’ve seen up until this point, at least for the general public. (More complicated mice like this one have shown up on engineering stations, imaging systems, and countless other vertical application machinery).

If you look carefully (click on the photo to see it a bit larger), you’ll see that it has no fewer than 16 buttons and a roller that are visible. The description actually boasts that it has “18 programmable mouse buttons with double-click functionality” and “Three different button modes: Key, Keypress, and Macro”.  They even show a comparison chart comparing it to other mice on the market.

While I won’t comment on the oddness of an open software consortium designing hardware (or rather, having a designer design some for them), I have to admit that this initial paragraph, on the page ‘About the OpenOfficeMouse, caught my attention:

The OpenOfficeMouse was designed with the goal of being the best and most useful mouse the digital world has seen to date. Initially inspired by the keyboards on the Treo smartphones, it was designed by a game designer who was annoyed with the paltry number of buttons available on high-end gaming mice. Because gaming mice have historically been designed primarily for FPS¹ games, not MMO² and RTS³ games, they do not possess sufficient buttons for the dozens of commands, actions and spells that are required in games that make heavy use of icon bars and pull-down menus. After discovering that the available World of Warcraft mice were nothing more than regular two-button mice decorated with orcs, dwarves, and Night elves, the idea of the WarMouse was born. After much experimentation, it was determined that 16 buttons divided into two 8-button halves were the maximum number of buttons that could be efficiently used by feel alone. However, in the process of design and development, it quickly became apparent that many non-gaming applications would also benefit from having dozens of commands accessible directly from the mouse, especially applications with nested pull-down menus and hotkey combinations. OpenOffice.org was selected as the ideal application suite around which to design this application mouse because the usage tracking feature of OpenOffice.org 3.1 permitted the assignment of application commands to mouse buttons based on the data gathered from more than 600 million actual mouse and keystroke commands enacted by users. The OpenOfficeMouse team are advocates of Free and Open Source Software, which is why we are members of the OpenOffice.org community and have created custom profiles for other OSS applications such as Mozilla Firefox, Mozilla Thunderbird, The Battle for Wesnoth, D-Fend Reloaded, and The Gnu Image Manipulation Program.

So what we have here is a design for a gaming mouse, now re-purposed for general purpose applications (like browsing the web, email, and the Open/MS Office suite of word processing, spreadsheets and presentations).

Maybe it’s because I don’t do much gaming (and by ‘don’t do much’,  I mean hardly at all),  maybe it’s because I come from the ‘make it for a klutz’ school of UI design because I’m not very coordinated, but I think that this approach to User Interface or Industrial Design will never have much of a following. It wasn’t lost on me that I had to look up some of those acronyms to provide the footnotes here. Sure, there will always be some small group of people who want more and more direct power over their work from their hardware, and they often buy the most baroque control devices. For me, however, the whole idea of taking a piece of gaming hardware and repurposing it to work on everyday tasks is about as appealing as using a flight simulator to do your banking. Sure, you might get more fine maneuverability during a funds transfer (if you could master the controls), but it hardly seems worth the effort. Maybe that’s the key here: Having a competitive advantage from  your hardware and your skill with it during a game is far more important and more likely to have you make that effort than being a whiz at moving from cell to cell in your spreadsheet or even triggering one of the 100 or so macros you’ve created for your word processing tasks.

So to the OpenOfficeMouse folks, I say, good luck, but forget about selling one of those mice to me. Now, we start seeing the ‘direct to brain’ controllers, where I don’t have involve my arms and fingers at all with typing and gesturing on the screen but just think where I want to the cursor to go, I’ll be more interested. That would be the 0 button mouse, which I think I’m going to have to address in some future post.

I particularly liked the monstrously bad user interface and industrial design, and how it mimics old ‘rotary’ phones.

I’m glad that great minds like Fabricant’s are working on solving Society’s ills.

Here’s the first one:

The Developer Conference has a very full schedule of sessions, split into 3 tracks. They are all categorized, numbered, and described in detail on the Apple WWDC Web site. While most attendees will want to go to a lot of these 150+ sessions, that’s clearly not possible, and not every session will appeal to every attendee . In fact, the schedule has been in place for nearly a month. What’s been added is the following: You can now create a personalized schedule of sessions and labs that will find its way to your hands, where you’ll need it during the conference. Using the online Conference Schedule, you click a session or lab you’re interested in, then click on the Select button in its information pop-up. (you can also add sessions and labs from an alternate Sessions and Lab page, where sessions are grouped by track rather than by the schedule):

After you’ve selected all of the sessions that you want, like this one:

…you click a link, which downloads a URL to iCal, which then subscribes to that calendar:

Then, when you then sync that calendar with your iPod or iPhone, you now have your personalized Conference schedule for each day on your iPhone:

The other web software that impressed me is the always-handy Internet Movie Database (IMDB). Whenever I’m stuck with that Now what other movie was that actor in?’ question or several like it, IMDB has been a godsend. While several sites are rolling out iPhone versions of the interface, IMDB does a spectacularly good job of it. The clear and sensible breakdown of an actor’s bio or film’s information lets you do that wonderful ‘swivel search’, where you can hop from actor to movie to cast to another actor to movie to director, etc. It keeps perfect track of your breadcrumb trail, and the performance, as well as excellent use of the ‘slide left’ animation for drilling down make it a real winner as an iPhone web app. I hope some of my other favourite sites roll out iPhone versions (Digg, Slashdot, Fark, BoingBoing and a bunch of other wonderful time-wasters, I hope you’re listening!)

I’ve been a little better about my personal blog, Loud Murmurs, but now that my contract at IBM is over, I have a little spare time. That means not only redesigning this web site (yes, look for exciting new changes in the coming days and weeks) but also starting to write in this blog once again. I will make another effort at getting my presentation of roughly a year ago up online (and fortunately, or unfortunately, depending on your point of view, it’s still just as valid today as it was then).

What you can’t see is that I have a new admin interface which I really, really like. It’s the Fluency Admin by Dean J, Robinson.Tell you what, I’ll put in a screenshot of the screen I’m writing this on:

Before the iPod, Apple’s first take on a hand-held device, was the Newton. The Newton was far more innovative in some ways, at least in terms of a user interface , approach to the data (with a unique ‘data soup’) and how a user might interact with it. Here’s a Getting Started video for the Newton someone posted on YouTube:

The Newton was about written communications, but the user interface was also far more oriented toward a give-and-take interaction with the user. For instance, you’d write ‘Lunch with Matt at 1PM on Friday’ in the calendar, and the Newton would do it’s best to try and figure out what you meant, putting a calendar entry ‘Lunch with Matt’ in the 1 PM time slot in your calendar. If you highlight someone’s name in a bit of recognized text, and then chose ‘FAX’ from the menu, the device would go to your FAX address book, do its best to locate the most likely person you were faxing to (by the first match from a find, in this case) and fill in the FAX number in send box. These best guesses were not always successful, and in some ways, reflected in microcosm some of the worst failures about the Newton. By raising expectations about how much pseudo-intelligence there was in such a device, people were all the more angry or amused when it fell on it’s flat glass face. I had a Newton, and although I was no fanatic about it, I always felt that it was falling just short of some truly amazing feats of computer-human interaction.

Fast-forward to last week: Contrast the Newton Video with this more recent iPhone demo:

Where the Newton video is more of a marketing piece that tries to convince you of the device’s worth, the iPhone video is just a voyeuristic view of someone using their iPhone to listen to music, watch a video, create an ad hoc conference call, send a photo in an email, text message someone, listen to voice mail, and use the Internet, etc.

The iPhone does not try to fill in the gaps, except where it knows such synergies can usually work. For instance, in the Google Maps based application, it allows you to dial whatever business you locate on a map (if there is a phone number). Where the Newton provided a somewhat spooky interaction with a ‘magic pad’ where the device would try and perform complex tasks based on cryptic messages from you, the iPhone puts it’s processing cycles into simpler, more physical tasks , such as how to move pages around to simulate the physics of the real world, how to flip the screen automatically when the device is put on its side and how to display lots of colourful icons and other pictures on a gorgeous screen.

The Newton was ascetic and hermetic, the iPhone is gorgeous, and perhaps even a little garish. Is the iPhone a step forward in UI Design? The Newton tried to do far more with less, but clearly the market did not want that. The iPhone is far more about ‘theatre’, which is why the voyeuristic demo works so well. It is also about applying what has been learned in the desktop and iPod world (setting wallpaper, creating an email, choosing and playing a piece of music) and applying those to a new form factor.

Although it’s arguable that the iPhone is less about really revolutionary thinking about UIs (like the Newton perhaps was), I think we may be ready for some of those. For instance, something as simple as voice recognition of certain commands should be doable on the next version of the iPhone, and synthesized voice from it wouldn’t be bad, either. This has already been done on the desktop, and since the iPhone is supposedly using the same OS, Apple (or other key third party developers) should be able to port some of these technologies to this new hardware fairly easily. I want to be able to say to my iPhone: “Make a conference call between Pam and Matt” and have it call one, notify them of the conference call and then connect the two calls.

Essentially, I want the pretty face of the iPhone with the brains (or better) of the Newton.

Besides the 100% correct prediction that we would be concerned with the deforestation of the Amazon area and its effects on the level of CO2 in the atmosphere (although it didn’t go the next step to foretell that this and other human activities would wreak havoc through climate change*), The Knowledge Navigator also demonstrated some aspects of computing that did in fact come to pass, others that are not quite there yet, and a few things that we probably won’t see for a long time to come. For those who weren’t lucky enough to see it in 1987 (or would like to take a brief walk down memory lane), here it is:

Here’s what they got absolutely right:

Touch Screen Interface
This year seems to be the year of the touch screen, whether it’s Apple’s own iPhone or Microsoft’s Surface computer (see previous posting). Tablet computers that use a stylus for input have been around for quite awhile, but the intimacy and immediacy of a finger as an input device seems to have truly taken hold in 2007.

Voice Synthesis
Despite the fact that the demo computer’s synthetic voice is a bit smoother than today’s synthetic voices, we are getting pretty close to this. (I myself went the extra mile to purchase a voice that sounds a bit like a Butler with a British accent that my computer uses for alerts and other notifications.) There are times when many have called for some service over the phone and mistaken the synthetic voice on the other end of the line for a human.

Videoconferencing
Here the film hits a home run and almost exactly the way it was shown. In fact, in a few hours I’m going to be using it to talk to my parents on the other end of the continent, back in the US. Next year, the example of the other professor sharing a screen with the local computer will be quite common, as Apple’s iChat AV will then include not just videoconferencing, but presentation and screen-sharing.

Telephony Integration
While not widespread, talking on the telephone through the computer is growing by leaps and bounds. There are several packages for integrating voice mail using the computer as the processor and storage medium for messages, and the widespread adoption of VOIP (voice over Internet protocol) will only hasten this trend. The iPhone’s ‘visual voicemail’ is surprising in that it does exactly the opposite of the demo (lists voice mails as messages, like an email), rather than announcing calls missed or the caller ID of an incoming call.

Intermingling of Professional and Personal Data
While this may be seen as a trivial detail, it’s worth remembering that in 1987, only true geeks kept their calendar and to-do lists in their computers, and everyone else continued to use day planners, faxes, snail mail and post-it notes for much of their personal organization. Only with better screens, faster processors, and software that got smarter and better designed about these tasks (and much of it has quite a ways to go), did the marginalia of personal lives find their way onto personal computers in offices. Still, this trend isn’t finished yet, and one can still find plenty of Daytimers and postits in most homes and offices.

Here’s what they were a little too optimistic on (although not completely missing):

Speech Recognition
The demo goes to great lengths to show that the professor is mumbling, giving partial information, and not talking to his device as a computer, but as another human. At one point during his telephone conversation, he even pauses and the computer fills in with an appointment time, as if it had been following in on the conversation! While speech recognition on computers has reached the point where it is common to be able to dictate an email, letter, article, or even a book to your computer and have it type the entire thing out as you speak it, one still has to speak clearly and the computer still does make some mistakes (although a lot fewer than it used to). I once saw a great T-Shirt slogan on someone who was on the team at Apple working on this technology: “I helped Apple Wreck a Nice Beach” (If you didn’t get it, say it out loud).

Full Colour Mapping of Geological Data
Google Maps and Google Earth, along with mashups of data from a variety of sources has started, but there are few simple, easy ways of doing a geo-plot with any data source without some massaging of the data (although this may not be as much the case in certain circles). There are hundreds of companies in the field of GIS (Geographical Information Systems), and it’s possible that some standards for interoperability will start to arise, especially with the rise of XML-based data sources. RSS feeds of Real Estate listings have already been mashed up, and traffic management and weather forecasting have both employed computer-based mapping for years. It’s only a matter of time before geophysical data is also available to the general public in an easy-to-assemble format.

Form Factors and other Hardware details
While there has been a move by most computer users to laptops from desktops, there are still a number of things in the demo (Speech recognition, perhaps) that are doable on a desktop but that most laptops still don’t quite have the horsepower for. This will change in the next 2 to 3 years (if not sooner). It’s surprising that there is no keyboard visible anywhere, but that is probably more to make a point (as is also the case with Microsoft’s Surface demos). Also, there is the conspicuous (at least for me) absence of wires for the connection to the phone, and no clear microphone. While wi-fi connections to the Internet are a reality, a wireless (voice) telephone interface built into a laptop would probably only work for Skype or some other VOIP solution. Still, perhaps there is an interesting potential product for people who want to use their computer as a cell speakerphone with omni-directional microphone.

Here’s what we are still a long way from:

Artificial Intelligence
Wow, did they ever miss on this one! AI is the flying car of computing. The idea that there would be an ‘agent’ as they referred to him, living in your computer, responding to you intelligently and with knowledge of both you and the outside world… well I suspect we are still a few decades away from this. There are many points in the demo where I said ‘Nice try’, imagining how today’s technology would make a mess of searches, taking calls, connecting you to data, etc.

Still, that’s not a bad hit rate for 20 years. John Sculley should give himself a little pat on the back for some impressively accurate vision, and the computer industry (and Apple in particular) should be pleased that a lot of this has come true. Now, it will be interesting to see when the next generation of college professors (and others) use these tools as much and as easily as the actor in the demo did.

*At the very least, one has to give Sculley credit for being one of the very first people in public life to sound the alarm about this ecological issue. It’s very appropriate that 20 years later we find Al Gore on Apple’s Board of Directors. It’s also perhaps a little ironic that Al Gore’s office setup — 3 huge screens plus a dedicated flatscreen TV, mountains of paper and bookcases filled to overflowing) is absolutely nothing like the climatology professor in the video.

But I’m getting ahead of myself here. Surface, besides all of that multi-touch stuff, is a strangely exotic and futuristic product. Most folks would call it ‘bleeding edge’. It’s frankly not something I expect from Microsoft. When I think of Microsoft, I think of Windows™. Windows is not bleeding edge. It’s market-tested, well worn, doesn’t take chances, and is definitely not exotic and futuristic. It runs on hardware that is getting cheaper by the day, and most of the time that hardware is, well, ugly (with a few exceptions from Sony and maybe one or two others).

Surface is none of that. It’s a 30 inch acrylic display with touchscreen built into a rather austere-looking coffee table that’s 22 inches high, 21 inches deep and 42 inches wide. There’s no keyboard and no mouse, although it does have Wi-Fi, Bluetooth and wired Ethernet connectivity. The main way that you interact with it is by touching the screen. Actually, Surface is designed to work not only with more than one finger touching at once, but with more than one person interacting with it at once. Oh, and yes, in case you forget, it’s a computer running Microsoft’s own Vista OS (Windows is still in there somewhere!) and will probably cost somewhere between $3,000 and $5,000. (Update: I just found out that these numbers are $5,000 to $10,000. No big surprise there.) What remains is the question of what you actually do with a $10,000 coffee table touchscreen computer with Internet and wireless connectivity.

Here are some examples of what Microsoft has in mind: (click on these thumbnails to see a larger image).

In the Fact Sheet on their web site, Microsoft says that they will ship Surface (and yes, this is a shipping product) “… to partners with a portfolio of basic applications, including photos, music, virtual concierge and games, that can be customized to provide their customers with unique experiences.” The web site for showing off Vista has several video demos, ranging from happy-smiling-people (a term I learned from my days at Fidelity that refers to those models you see in business ads who always seem to be having a better day than you are) arranging digital photographs, planning a trip, playing cards, and interacting with their cell phones as they place them on the glass table-top display. One of the demos that seems just a little unrealistic has a little girl using Surface to paint a cute picture. I’m sorry, but even though the top 1% of the very wealthy in the US are getting wealthier, it’s hard to imagine anyone but the Gates families and a few others that can afford a 10K electronic paint toy for their kid, much less place it in the living room. What did look the most interesting, however, were the instances where someone was interacting with one or more cell phones, allowing people to download trip information into them by dragging the information into a box that stood for the phone on the screen or moving music tracks from one cell phone/audio player (iPhone? Zune?) to another. The one that made me think ‘Hmm, I don’t think they’ve thought through the social dynamics of this one…’ was an example where a group of diners in a restaurant split the bill and dragged their portions to their cell phones. It might be nice to be able to split a bill with perfect math accounting for each dish ordered, but somehow having that appear on the screen felt a bit…awkward? I suppose it’s no worse than the waitress splitting up the bill, but can you imagine the tug-of-war that might ensue when one or more diners tries to pay for the other?

The fact is, this is a product that is probably going to be seen in casinos and some upscale restaurants and hotels first. The living room will have to wait. I have to say that I like that Microsoft is thinking outside the box (or rather, the desktop or laptop), and some of the applications do look fun. Will this catch on? I’m not sure. For a long time, people were hot on touchscreen kiosks for some of this activity, and they never really took off, and I can’t really see Surface working as a real restaurant table:

Oh, I spilled my Curry sauce all over the computer screen…
Don’t worry sir, I’ll just wipe it up, oh, whoops, sorry to order that round of champagne…
Oh, how cute, little Dylan is playing Blackjack between courses…Oh, don’t stab at the table with your fork, sweetheart…

Damn that messy physical world, full of food, klutzes and kids getting in the way of our cool software technology!

I used and designed interfaces for a touchscreen years ago, and I was struck then by the intimacy of the interactions. Rather than move a cursor via a physical proxy (the mouse, trackball or trackpad), you can touch a part of the screen, and sometimes that piece of the screen would change, just like in the physical world. The examples that Microsoft showed had this happening most of the time. You touch something, it either ripples, moves or highlights. The Surface UI is meant to be more than simply a new display, form factor and method of input, it’s a different style of interactivity that looks like the iPhone writ large (and for two or more people). I’ll bet it just kills the Surface team to hear that.

*It’s worth mentioning that Jeff Han demoed a surface interface at TED in 2006, and much of what he demonstrated is reflected in Microsoft’s new product. Han’s demo also got oohs and ahs, but it has only been seen by a relative few, where the hype for the iPhone definitely went farther into the mainstream media.