I’ve recently been involved with an iPhone project where we are doing a few custom UI controls, and it’s definitely proved a learning experience about the difference between designing for a computer screen and designing for the iPhone screen (either the current one or the upcoming iPhone 4 Retina Display screen).

One thing I’ve learned has to do with the characteristics of the iPhone screen, and how that influences User Interface Design choices. Over the years, I’ve become used to the what it takes to show a change on a computer monitor, which is to say, the degree to much you need to change the colour, shape, or scale so that it’s obvious, even if the user looks away for a second before the change occurs and then looks back.  This might apply to an object in its selected and unselected states, or the addition of something new on the screen, or perhaps the enabling or disabling of a button or other element.  At first, I thought this was due to the dots (or in this case, pixels) per inch of the iPhone versus computer monitors. Monitors are usually somewhere between 72 PPI (Pixels Per Inch) and perhaps 200 PPI on the best equipment. The IBM T220/T221 LCD monitors marketed from 2001–2005 were 204 PPI, and they probably set the standard for a while. These days, a 20-inch (50.8 cm) screen with a 1680×1050 resolution has a 99.06 PPI, and a garden variety Macbook (not the higher end Macbook Pros) has 113 PPI (Wikipedia has an article on how this is calculated).

However, the iPhone PPI is listed at 163 PPI, which although it’s on the high side, is certainly not significantly higher than a typical computer these days. The difference, then, must be the size of the screen. In the case of any iPhone screen 2G, 3G, 3Gs and 4G, it’s a 3.5 inch screen (compare that to the aforementioned 20-inch, and now we’re talking different.)

It might be obvious, but what I’ve noticed is that the amount of change you have to make in order to be noticeable is far more on the iPhone’s screen. The contrast must be greater, scaling or moving an object between one state and another has to be larger (or farther), and as a corollary to this rule of thumb,  it’s easy to miss subtle changes.  Several times during development of the app we’re working on, I had to report to the graphic designer that I was working with, that a selection style wasn’t distinct enough, or that a small detail of a button, such as a downward pointing arrow, had to be rendered with higher contrast (the UI had a lot of grey objects, and some of them had white or darker grey overlays).

I think the easy way to think about this is the analogy of greasepaint. What’s greasepaint? It’s the traditional makeup that actors wore (and has now been superseded by more modern stage makeup) that helps to compensate for both the washing out of facial features by the bright theatre lights, as well as help audience members to make out their faces, even though the actors were farther away (and hence, smaller in the eyes of theatregoers – perhaps the equivalent of being 4 or 5 centimeters tall depending on how far away from the stage they were sitting). I remember going backstage to a dressing room after the Play or Opera was over, and was always struck by how odd the performers looked before removing all of that extreme makeup, which brought out cheekbones or encircled their eyes (like a Raccoon, I though!).

So User Interface Designers working on iPhone apps, remember, the computer screen is the dressing room, and the iPhone screen is the stage. Don’t forget the greasepaint!

I particularly liked the monstrously bad user interface and industrial design, and how it mimics old ‘rotary’ phones.

Full fathom five thy father lies;
Of his bones are coral made;
Those are pearls that were his eyes:
Nothing of him that doth fade,
But doth suffer a sea-change
Into something rich and strange.
— from Ariel’s Song, The Tempest by William Shakespeare

I loved the almost anal-retentive display of data through a heads-up display about the scenery and other details in the opening scenes of the movie ‘Stranger than Fiction‘:

Now, imagine that kind of data display about everything; The chemicals in the soil around you, the wavelengths of light as they strike your skin, the building materials of the structures you walk by; all are a sea of data that is not so much invisible as it is inaccessible. Now imagine, if you had a heads-up display on your glasses (or on contact lenses, as is suggested in Vernor Vinge’s Novel Rainbow’s End). If you are ‘wearing’ as Vinge calls it, you now have the possibility of superimposing all sorts of data on top of the reality you see around you. In fact, if you prefer, you can replace that reality with one as rich and strange as you like.

Rather than a real place, what if this were done with, say, a Fairy Tale. Tomas Nilsson, a design student at Sweden’s Linköping University, decided to do just this with the Little Red Riding Hood story, which started out as a class project:

As computing and access to data becomes more ubiquitous, I think this will start to change our view of reality. It’s a subtle thing, but the fact that many people now carry some sort of device (either a smart phone or a portable GPS device), so they are never truly lost. That’s a big change of their reality, right from the start.

The other evening, my iPhone had some problems, so I headed home to try and fix it (I did, the software needed to be reinstalled). The ride on the bus felt very strange without being able to listen to podcasts or music. I couldn’t check the time. I couldn’t call anyone, or check my email. It wasn’t until then did I realize how much I rely on this little brick of metal and glass.

Before the iPod, Apple’s first take on a hand-held device, was the Newton. The Newton was far more innovative in some ways, at least in terms of a user interface , approach to the data (with a unique ‘data soup’) and how a user might interact with it. Here’s a Getting Started video for the Newton someone posted on YouTube:

The Newton was about written communications, but the user interface was also far more oriented toward a give-and-take interaction with the user. For instance, you’d write ‘Lunch with Matt at 1PM on Friday’ in the calendar, and the Newton would do it’s best to try and figure out what you meant, putting a calendar entry ‘Lunch with Matt’ in the 1 PM time slot in your calendar. If you highlight someone’s name in a bit of recognized text, and then chose ‘FAX’ from the menu, the device would go to your FAX address book, do its best to locate the most likely person you were faxing to (by the first match from a find, in this case) and fill in the FAX number in send box. These best guesses were not always successful, and in some ways, reflected in microcosm some of the worst failures about the Newton. By raising expectations about how much pseudo-intelligence there was in such a device, people were all the more angry or amused when it fell on it’s flat glass face. I had a Newton, and although I was no fanatic about it, I always felt that it was falling just short of some truly amazing feats of computer-human interaction.

Fast-forward to last week: Contrast the Newton Video with this more recent iPhone demo:

Where the Newton video is more of a marketing piece that tries to convince you of the device’s worth, the iPhone video is just a voyeuristic view of someone using their iPhone to listen to music, watch a video, create an ad hoc conference call, send a photo in an email, text message someone, listen to voice mail, and use the Internet, etc.

The iPhone does not try to fill in the gaps, except where it knows such synergies can usually work. For instance, in the Google Maps based application, it allows you to dial whatever business you locate on a map (if there is a phone number). Where the Newton provided a somewhat spooky interaction with a ‘magic pad’ where the device would try and perform complex tasks based on cryptic messages from you, the iPhone puts it’s processing cycles into simpler, more physical tasks , such as how to move pages around to simulate the physics of the real world, how to flip the screen automatically when the device is put on its side and how to display lots of colourful icons and other pictures on a gorgeous screen.

The Newton was ascetic and hermetic, the iPhone is gorgeous, and perhaps even a little garish. Is the iPhone a step forward in UI Design? The Newton tried to do far more with less, but clearly the market did not want that. The iPhone is far more about ‘theatre’, which is why the voyeuristic demo works so well. It is also about applying what has been learned in the desktop and iPod world (setting wallpaper, creating an email, choosing and playing a piece of music) and applying those to a new form factor.

Although it’s arguable that the iPhone is less about really revolutionary thinking about UIs (like the Newton perhaps was), I think we may be ready for some of those. For instance, something as simple as voice recognition of certain commands should be doable on the next version of the iPhone, and synthesized voice from it wouldn’t be bad, either. This has already been done on the desktop, and since the iPhone is supposedly using the same OS, Apple (or other key third party developers) should be able to port some of these technologies to this new hardware fairly easily. I want to be able to say to my iPhone: “Make a conference call between Pam and Matt” and have it call one, notify them of the conference call and then connect the two calls.

Essentially, I want the pretty face of the iPhone with the brains (or better) of the Newton.

I’ve become fascinated with the new web application called Twitter. It’s a way of simply and tersely updating others on your status: This makes it kind of like a cross between a blog, instant messaging and perhaps SMS (cell phone text messaging). Twitter is like a blog, because it broadcasts your thoughts, moods, impressions, and other personal mumblings to the world at large. It’s like instant messaging because it consists of short messages, no more than 140 characters in length. It’s like SMS messaging because twitters (or is the singular a ‘tweet’?) can be recieved on a cell phone (and I believe you can update your own status via a cell phone, although this is much easier and cheaper in the US than Canada or overseas). Oh, and your twitter status updates only go to the people who choose to ‘follow’ you (and likewise, you only get updates from those people who you know and want to be updated about. Your posse is your update audience, and you get their updates as well.

To use twitter, you bring up the Twitter web site, and if you have an account and have left a cookie that logs you in automatically, you make a twitter/tweet by typing your message into the form. To help you stay under the 140 character limit, there is a countdown javascript that tells you the number of characters you have left. It’s not that inconvenient or confusing, but I keep wondering if there’s a better way.

I suppose that using an SMS cell phone to update my status would be best, but even then, typing is a problem. Perhaps there could be a special twitter application for phones and PDAs (or both, like my Treo or a Blackberry), with set phrases that you could use by pressing buttons or the option to insert your location, if it knew you were near a particular place that you are often located at, like Work, Home or a friend’s residence. The key here is that twitter has a different User Model than the web, or Instant messaging, or even perhaps SMS.

A User Model, as I define it, is everything about a given situation that a user experiences when they are accessing or operating software or hardware, including their posture, how much time they have, their level of comfort (or discomfort), what else they might be busy doing, the amount of attention they can/want to dedicate to the activity, etc.. A lot of software assumes that you are sitting in a task chair, have a keyboard, mouse or trackpad and a monitor of decent size, that you have a block of time to dedicate to the activity you are engaged in and you can devote nearly your full attention to the task at hand. Some software assumes that you are connected to the Internet, but other (non-web, of course) packages don’t. In contrast, Instant Messaging always assumes that you are connected continuously to the Internet, are a pretty fast typist, and can devote all or perhaps part of your attention to the conversation you are having. A web application called ‘Do I Need a Jacket’ (or www.doineedajacket.com) is designed to be used just as you are just leaving your home or office. You look at it for a second, perhaps from a standing position and peering over at your screen, and it remembers the last place you put in for it’s ‘setting’ (you can also change the thresholds for cold, chilly or wind speed that trigger a yes or no answer for the question of whether you need a jacket or not). One click is all it needs once this is set up. SMS assumes only that you have your phone or Blackberry with you and that it is turned on. It also assumes that you don’t have a full keyboard (although the remarkably good keyboard on the Blackberry has begun to change this a bit).

Twitter needs only a little bit of attention (like IM), but it is required sporadically, like an incoming SMS or IM. Like a blog, it requires that you think about yourself, or at least what you want to say, but unlike a blog, you don’t have to be a writer (or even be able to write a full sentence!) Twitter can serve some very useful purposes, like letting a bunch of friends know where you are if you are meeting up later, or receiving constant status reports (even from RSS feeds, like the local weather), or perhaps even some applications we haven’t thought of yet.

At any rate, this is a new kind of application, with some very interesting challenges related to the User Model as well as the role that online messaging can have in our lives. For a version 1.0 it’s crude, but then again, IRC (Internet Relay Chat) was also pretty crude as well, and look what that gave rise to (AOL chats, Minitel, AIM, Gtalk, iChat AV, Meebo, and who knows what else?!)