Full fathom five thy father lies;
Of his bones are coral made;
Those are pearls that were his eyes:
Nothing of him that doth fade,
But doth suffer a sea-change
Into something rich and strange.
— from Ariel’s Song, The Tempest by William Shakespeare

I loved the almost anal-retentive display of data through a heads-up display about the scenery and other details in the opening scenes of the movie ‘Stranger than Fiction‘:

Now, imagine that kind of data display about everything; The chemicals in the soil around you, the wavelengths of light as they strike your skin, the building materials of the structures you walk by; all are a sea of data that is not so much invisible as it is inaccessible. Now imagine, if you had a heads-up display on your glasses (or on contact lenses, as is suggested in Vernor Vinge’s Novel Rainbow’s End). If you are ‘wearing’ as Vinge calls it, you now have the possibility of superimposing all sorts of data on top of the reality you see around you. In fact, if you prefer, you can replace that reality with one as rich and strange as you like.

Rather than a real place, what if this were done with, say, a Fairy Tale. Tomas Nilsson, a design student at Sweden’s Linköping University, decided to do just this with the Little Red Riding Hood story, which started out as a class project:

As computing and access to data becomes more ubiquitous, I think this will start to change our view of reality. It’s a subtle thing, but the fact that many people now carry some sort of device (either a smart phone or a portable GPS device), so they are never truly lost. That’s a big change of their reality, right from the start.

The other evening, my iPhone had some problems, so I headed home to try and fix it (I did, the software needed to be reinstalled). The ride on the bus felt very strange without being able to listen to podcasts or music. I couldn’t check the time. I couldn’t call anyone, or check my email. It wasn’t until then did I realize how much I rely on this little brick of metal and glass.

I’m glad that great minds like Fabricant’s are working on solving Society’s ills.

Rather than try to write something profound (at least on the surface), I thought I’d start writing in this blog again with an observation about today’s date, at least in terms of the History of Computer Science:

On today’s date, HAL, the computer from 2001: A Space Odyssey turns 17 year’s old, as the movie says:

I am a HAL 9000 computer, Production Number 3. I became operational at the HAL Plant in Urbana, Illinois, on January 12, 1992. My instructor was Mr. Langley, and he taught me to sing a song. If you’d like to hear it I can sing it for you. It’s called ‘Daisy’

I always thought that the production number being 3 was intriguing. It couldn’t be a nod to Windows 3.1, the first successful version of that software because the book was written decades before that appeared on the scene.  What happened to production numbers 1 and 2? (It was mentioned, I seem to remember, that HAL 1000-8000 series had problems of some sort and were “not entirely successful”).

The idea of a mutinous, murderous central computer is a theme that is still alive and well in movies today: the movie WALL-E has one of these, the Autopilot computer that looks a bit like HAL’s red eye inserted into an old fashioned ship’s wheel (and the voice actor who gets to do it, in the credits is, wait for it… Macintalk, the speech synthesis software on the Macintosh (!))

Needless to say, in this year, 2009, there is no HAL 9000, no similar level of Artificial Intelligence, no ships to Jupiter, and no permanent base on the moon. We do have a space station, but Pan Am airlines never survived to create that beautiful space liner, and although there is talk of private citizens doing flights, it is Virgin Airlines that is going to be doing that.

A great example of good information design, if there ever was one. I’m thinking that it might be easy to read, once you get used to it. And pretty, too. Will make a nice screen saver.

Besides the 100% correct prediction that we would be concerned with the deforestation of the Amazon area and its effects on the level of CO2 in the atmosphere (although it didn’t go the next step to foretell that this and other human activities would wreak havoc through climate change*), The Knowledge Navigator also demonstrated some aspects of computing that did in fact come to pass, others that are not quite there yet, and a few things that we probably won’t see for a long time to come. For those who weren’t lucky enough to see it in 1987 (or would like to take a brief walk down memory lane), here it is:

Here’s what they got absolutely right:

Touch Screen Interface
This year seems to be the year of the touch screen, whether it’s Apple’s own iPhone or Microsoft’s Surface computer (see previous posting). Tablet computers that use a stylus for input have been around for quite awhile, but the intimacy and immediacy of a finger as an input device seems to have truly taken hold in 2007.

Voice Synthesis
Despite the fact that the demo computer’s synthetic voice is a bit smoother than today’s synthetic voices, we are getting pretty close to this. (I myself went the extra mile to purchase a voice that sounds a bit like a Butler with a British accent that my computer uses for alerts and other notifications.) There are times when many have called for some service over the phone and mistaken the synthetic voice on the other end of the line for a human.

Videoconferencing
Here the film hits a home run and almost exactly the way it was shown. In fact, in a few hours I’m going to be using it to talk to my parents on the other end of the continent, back in the US. Next year, the example of the other professor sharing a screen with the local computer will be quite common, as Apple’s iChat AV will then include not just videoconferencing, but presentation and screen-sharing.

Telephony Integration
While not widespread, talking on the telephone through the computer is growing by leaps and bounds. There are several packages for integrating voice mail using the computer as the processor and storage medium for messages, and the widespread adoption of VOIP (voice over Internet protocol) will only hasten this trend. The iPhone’s ‘visual voicemail’ is surprising in that it does exactly the opposite of the demo (lists voice mails as messages, like an email), rather than announcing calls missed or the caller ID of an incoming call.

Intermingling of Professional and Personal Data
While this may be seen as a trivial detail, it’s worth remembering that in 1987, only true geeks kept their calendar and to-do lists in their computers, and everyone else continued to use day planners, faxes, snail mail and post-it notes for much of their personal organization. Only with better screens, faster processors, and software that got smarter and better designed about these tasks (and much of it has quite a ways to go), did the marginalia of personal lives find their way onto personal computers in offices. Still, this trend isn’t finished yet, and one can still find plenty of Daytimers and postits in most homes and offices.

Here’s what they were a little too optimistic on (although not completely missing):

Speech Recognition
The demo goes to great lengths to show that the professor is mumbling, giving partial information, and not talking to his device as a computer, but as another human. At one point during his telephone conversation, he even pauses and the computer fills in with an appointment time, as if it had been following in on the conversation! While speech recognition on computers has reached the point where it is common to be able to dictate an email, letter, article, or even a book to your computer and have it type the entire thing out as you speak it, one still has to speak clearly and the computer still does make some mistakes (although a lot fewer than it used to). I once saw a great T-Shirt slogan on someone who was on the team at Apple working on this technology: “I helped Apple Wreck a Nice Beach” (If you didn’t get it, say it out loud).

Full Colour Mapping of Geological Data
Google Maps and Google Earth, along with mashups of data from a variety of sources has started, but there are few simple, easy ways of doing a geo-plot with any data source without some massaging of the data (although this may not be as much the case in certain circles). There are hundreds of companies in the field of GIS (Geographical Information Systems), and it’s possible that some standards for interoperability will start to arise, especially with the rise of XML-based data sources. RSS feeds of Real Estate listings have already been mashed up, and traffic management and weather forecasting have both employed computer-based mapping for years. It’s only a matter of time before geophysical data is also available to the general public in an easy-to-assemble format.

Form Factors and other Hardware details
While there has been a move by most computer users to laptops from desktops, there are still a number of things in the demo (Speech recognition, perhaps) that are doable on a desktop but that most laptops still don’t quite have the horsepower for. This will change in the next 2 to 3 years (if not sooner). It’s surprising that there is no keyboard visible anywhere, but that is probably more to make a point (as is also the case with Microsoft’s Surface demos). Also, there is the conspicuous (at least for me) absence of wires for the connection to the phone, and no clear microphone. While wi-fi connections to the Internet are a reality, a wireless (voice) telephone interface built into a laptop would probably only work for Skype or some other VOIP solution. Still, perhaps there is an interesting potential product for people who want to use their computer as a cell speakerphone with omni-directional microphone.

Here’s what we are still a long way from:

Artificial Intelligence
Wow, did they ever miss on this one! AI is the flying car of computing. The idea that there would be an ‘agent’ as they referred to him, living in your computer, responding to you intelligently and with knowledge of both you and the outside world… well I suspect we are still a few decades away from this. There are many points in the demo where I said ‘Nice try’, imagining how today’s technology would make a mess of searches, taking calls, connecting you to data, etc.

Still, that’s not a bad hit rate for 20 years. John Sculley should give himself a little pat on the back for some impressively accurate vision, and the computer industry (and Apple in particular) should be pleased that a lot of this has come true. Now, it will be interesting to see when the next generation of college professors (and others) use these tools as much and as easily as the actor in the demo did.

*At the very least, one has to give Sculley credit for being one of the very first people in public life to sound the alarm about this ecological issue. It’s very appropriate that 20 years later we find Al Gore on Apple’s Board of Directors. It’s also perhaps a little ironic that Al Gore’s office setup — 3 huge screens plus a dedicated flatscreen TV, mountains of paper and bookcases filled to overflowing) is absolutely nothing like the climatology professor in the video.

And now they have a new video that explains the concept of Wikis and why you would use one:

They are well done, and charming the way that they employ old fashioned stop-motion technique to create a sort of animated white board. My only negative comment is that I wouldn’t mind them using a different voice without the twang (just a personal thing I have with accents). On the other hand, some would probably say that using an amateur voice helps to suggest a non-professional feel for the presentation, which makes the subject less threatening.

It’s only a matter of time before some advertising exec sees one of these and rips off the technique for a ‘hip’ spot about a car or razor. In the meantime, I’m pleased to be able to point people to these videos when they need a quick (and are done at breakneck speed) tutorial on a few new concepts and technologies that are very popular these days. If you don’t quite get the concept of each the first time through, try once more through. It won’t take long.

A recent paper by Michael Schmitz, a student at Saarland University (a fascinating institution near the borders of Germany, France, Luxembourg and Belgium), surveys several different user interfaces from science fiction movies throughout film history. Human Computer Interaction in Science Fiction Movies cites several films, including Metropolis, Johnny Mnemonic, The Matrix, Logan’s Run, Forbidden Planet, Gattaca, Minority Report, Total Recall, Star Trek: The Next Generation (one of the movies, I assume), X-Men, 2001: A Space Odyssey, Dark Star, the 2002 remake of The Time Machine, and even Galaxy Quest.

It is intriguing the way Schmitz has organized the UIs he is taking note of:

The movie clips in the main section of his survey are categorized according to their area of real-life applications and research:
Neuro Technology: Technologies that connect to brains are introduced here.
Identification: Electronical identification of individuals
Displays: Various kinds of displays as an output medium
Speech: Including speech synthesis and recognition as well as intelligent assistants/avatars as special subgroup.
Other I/O technologies: All technologies that are more specialised and that were difficult to categorize according to the fields above, for example gesture recognition or tangible user interfaces

I think these categories arose because of the scenes that he chose to include. There are perhaps some other movies that were not chosen that may have brought in other areas of classification (for instance, Tron, Blade Runner, Serenity, any of the Star Wars films, The Fifth Element, Ghost in the Shell, any of the James Bond films, 12 Monkeys or Contact).

The fact is, the UIs that we often see in the movies are not always accurate because they aren’t necessarily built for good usability, but instead for good dramatic effect. Sure, Tom Cruise could have done some of his research in Minority Report using a mouse and an LCD screen, but it was so much more dramatic for him to don VR gloves, and perform the exhausting 3D manipulation in virtual space in front of him. Even everyday, mundane events like receiving an email get full-screen 3D animations in the movie Disclosure with Michael Douglas and Demi Moore in 1994.

Nevertheless, movies do have a way of placing imagery and expectations in the mind of many movie viewers. In fact, because a surprisingly large number of people can’t tell the difference between movies and reality, some of these UI inventions may be adopted sooner after they are developed for real, everyday use. What James Bond sees on his computer screen may very well be built by some small startup determined to bring just such a tool to market, at whatever price the public wants to pay. They know they want it, though, because they saw it work just fine at the theatre.

It’s done using Flash, but I bet with a little work it could be done in AJAX.

Besides allowing you to zoom in on the data quickly, it encourages you to explore it to look for what might be unusual aspects that you might not have known. For instance, I learned that Germany has not had a Nobel prize winner in Medicine since 1908, which indeed, surprised me.

According to the report, this symbol is…

the result of a 5-year project conducted in 11 countries around the world. the new symbol, developed by human factor experts, graphic artists, & radiation protection experts, was tested on a total of 1,650 individuals in Brazil, Mexico, Morocco, Kenya, Saudi Arabia, China, India, Thailand, Poland, Ukraine & the United States to ensure that its message of “danger – stay away” was crystal clear & understood by all.

It’s interesting to note that this symbol follows a bunch of the ‘rules’ of UI design regarding warnings:

1. It’s red. Red is the colour of blood and is often associated with danger. This is the reason that I am always upset by web sites (like Webnames.ca) who decided that their link colour should be red to match their colour scheme. Red should almost never be the colour for your links, people! I’d even go so far as to say that red should be reserved in most UIs for only warnings or errors.
2. It uses a triangle shape, which is frequently used in warning signs because of the sharpness of the angles, connoting danger or at the very least, unpleasantness (if you got poked with it)
3. Unlike the old Radiation symbol (which is the three-sided symbol at the apex of the 3 figures within the sign), it has a verb in it! The arrow with the person running, which looks very much like an imperative ‘Go!’
4. It has some scary looking lines that show the radiation. Although this is not a rule per se, it does illustrate some activity, which is a good example of how an invisible force like radiation can be portrayed.
5. It shows the Death symbol, (skull and crossbones), though this could be misread as ‘Radiation can cause Pirates and make you lose your left foot’ if you wanted to be thickheaded/silly about it. (For a whole series of misreadings of IKEA warnings for comic effect, my friend Matt made up a whole slew of hysterical examples ).

I’m glad that they tested the symbol in countries like Brazil, Mexico, Morocco, Kenya, and Saudi Arabia, etc.. So often ‘Universal’ signs or icons don’t take into account differences in other cultures. I remember some years back something about Men’s and Women’s room signs not working well in countries where pants vs. dresses or skirts were not necessarily the sole marker of secondary sex characteristics. Who knows, maybe some Kilt-wearers in Edinburgh had to think for a minute.