I’ve recently been involved with an iPhone project where we are doing a few custom UI controls, and it’s definitely proved a learning experience about the difference between designing for a computer screen and designing for the iPhone screen (either the current one or the upcoming iPhone 4 Retina Display screen).

One thing I’ve learned has to do with the characteristics of the iPhone screen, and how that influences User Interface Design choices. Over the years, I’ve become used to the what it takes to show a change on a computer monitor, which is to say, the degree to much you need to change the colour, shape, or scale so that it’s obvious, even if the user looks away for a second before the change occurs and then looks back.  This might apply to an object in its selected and unselected states, or the addition of something new on the screen, or perhaps the enabling or disabling of a button or other element.  At first, I thought this was due to the dots (or in this case, pixels) per inch of the iPhone versus computer monitors. Monitors are usually somewhere between 72 PPI (Pixels Per Inch) and perhaps 200 PPI on the best equipment. The IBM T220/T221 LCD monitors marketed from 2001–2005 were 204 PPI, and they probably set the standard for a while. These days, a 20-inch (50.8 cm) screen with a 1680×1050 resolution has a 99.06 PPI, and a garden variety Macbook (not the higher end Macbook Pros) has 113 PPI (Wikipedia has an article on how this is calculated).

However, the iPhone PPI is listed at 163 PPI, which although it’s on the high side, is certainly not significantly higher than a typical computer these days. The difference, then, must be the size of the screen. In the case of any iPhone screen 2G, 3G, 3Gs and 4G, it’s a 3.5 inch screen (compare that to the aforementioned 20-inch, and now we’re talking different.)

It might be obvious, but what I’ve noticed is that the amount of change you have to make in order to be noticeable is far more on the iPhone’s screen. The contrast must be greater, scaling or moving an object between one state and another has to be larger (or farther), and as a corollary to this rule of thumb,  it’s easy to miss subtle changes.  Several times during development of the app we’re working on, I had to report to the graphic designer that I was working with, that a selection style wasn’t distinct enough, or that a small detail of a button, such as a downward pointing arrow, had to be rendered with higher contrast (the UI had a lot of grey objects, and some of them had white or darker grey overlays).

I think the easy way to think about this is the analogy of greasepaint. What’s greasepaint? It’s the traditional makeup that actors wore (and has now been superseded by more modern stage makeup) that helps to compensate for both the washing out of facial features by the bright theatre lights, as well as help audience members to make out their faces, even though the actors were farther away (and hence, smaller in the eyes of theatregoers – perhaps the equivalent of being 4 or 5 centimeters tall depending on how far away from the stage they were sitting). I remember going backstage to a dressing room after the Play or Opera was over, and was always struck by how odd the performers looked before removing all of that extreme makeup, which brought out cheekbones or encircled their eyes (like a Raccoon, I though!).

So User Interface Designers working on iPhone apps, remember, the computer screen is the dressing room, and the iPhone screen is the stage. Don’t forget the greasepaint!

Though the drawing doesn’t show it, Englebart’s mouse, which was one small part of Engelbart’s a larger project, aimed at ‘augmenting human intellect’ had 1 button. The drawing mainly shows how the block uses multiple rollers, which sense which way the mouse is being moved in terms of X and Y coordinates.

When Apple shipped the first Lisa computer (and of course, the first Mac) , the commandment that ‘Thy mouse shall have but  1 button’ was spoken to the masses. On the other side, the X-Window System, and the IBM PC mouse had multiple buttons (2 or 3). The two to three camps dug in for years, each claiming the ergonomic, moral or practical high ground over the others. The antipathy between the 1 or many buttons groups continues to to this day, even if this division is no longer the case. Many people believe that Apple has stayed true to their gospel and only makes or supports a 1 button mouse, but the unforutnately named ‘Mighty Mouse’, which shipped in 2005, supports multiple buttons virtually rather than physically (you click on one side or other other to simulate one or the other button), and also has a roller ball and 2 physical side buttons, providing no fewer than 5 buttons. The proliferation of mouse buttons, sometimes 2, sometimes 3, sometimes 5 or more, depends on the system and software one encounters. Some trackball devices have had 5 buttons that effectively provide even more control messages by allowing a different kind of click from different combinations of those buttons. Apple’s latest mouse (the even more unfortunately named ‘Magic Mouse’ – what group is coming up with these names?) even goes farther, making the entire mouse surface another control surface in and of itself, like the trackpad on a laptop. This, to me, is akin to attaching a steering wheel to the top of a gearshift, or some other bizarre composite, but I’ll have to withhold judgement until I try one, even though it sounds like the Industrial Design equivalent of a Turducken.

The point is, complex gestural movements, involving more than a simple click (or double click) on a pointing device have pretty much been adopted by all computer makers, with at least an accepted level of complexity, although for the most part, a user can work up to that complexity, by moving from simple gestures to more complex ones over time, hence the idea of a short cut to a function instead of making  that function only executable from a complex gesture.

As a friend of my parents puts it, ‘Anything worth doing is worth overdoing’. I shouldn’t be surprised by what I thought was certainly a post on The Onion, but no, it was serious, and it was the Open Office Consortium who was proposing this mouse:

Holy Roller, Batman! This thing is certainly the other end of the spectrum from the mice we’ve seen up until this point, at least for the general public. (More complicated mice like this one have shown up on engineering stations, imaging systems, and countless other vertical application machinery).

If you look carefully (click on the photo to see it a bit larger), you’ll see that it has no fewer than 16 buttons and a roller that are visible. The description actually boasts that it has “18 programmable mouse buttons with double-click functionality” and “Three different button modes: Key, Keypress, and Macro”.  They even show a comparison chart comparing it to other mice on the market.

While I won’t comment on the oddness of an open software consortium designing hardware (or rather, having a designer design some for them), I have to admit that this initial paragraph, on the page ‘About the OpenOfficeMouse, caught my attention:

The OpenOfficeMouse was designed with the goal of being the best and most useful mouse the digital world has seen to date. Initially inspired by the keyboards on the Treo smartphones, it was designed by a game designer who was annoyed with the paltry number of buttons available on high-end gaming mice. Because gaming mice have historically been designed primarily for FPS¹ games, not MMO² and RTS³ games, they do not possess sufficient buttons for the dozens of commands, actions and spells that are required in games that make heavy use of icon bars and pull-down menus. After discovering that the available World of Warcraft mice were nothing more than regular two-button mice decorated with orcs, dwarves, and Night elves, the idea of the WarMouse was born. After much experimentation, it was determined that 16 buttons divided into two 8-button halves were the maximum number of buttons that could be efficiently used by feel alone. However, in the process of design and development, it quickly became apparent that many non-gaming applications would also benefit from having dozens of commands accessible directly from the mouse, especially applications with nested pull-down menus and hotkey combinations. OpenOffice.org was selected as the ideal application suite around which to design this application mouse because the usage tracking feature of OpenOffice.org 3.1 permitted the assignment of application commands to mouse buttons based on the data gathered from more than 600 million actual mouse and keystroke commands enacted by users. The OpenOfficeMouse team are advocates of Free and Open Source Software, which is why we are members of the OpenOffice.org community and have created custom profiles for other OSS applications such as Mozilla Firefox, Mozilla Thunderbird, The Battle for Wesnoth, D-Fend Reloaded, and The Gnu Image Manipulation Program.

So what we have here is a design for a gaming mouse, now re-purposed for general purpose applications (like browsing the web, email, and the Open/MS Office suite of word processing, spreadsheets and presentations).

Maybe it’s because I don’t do much gaming (and by ‘don’t do much’,  I mean hardly at all),  maybe it’s because I come from the ‘make it for a klutz’ school of UI design because I’m not very coordinated, but I think that this approach to User Interface or Industrial Design will never have much of a following. It wasn’t lost on me that I had to look up some of those acronyms to provide the footnotes here. Sure, there will always be some small group of people who want more and more direct power over their work from their hardware, and they often buy the most baroque control devices. For me, however, the whole idea of taking a piece of gaming hardware and repurposing it to work on everyday tasks is about as appealing as using a flight simulator to do your banking. Sure, you might get more fine maneuverability during a funds transfer (if you could master the controls), but it hardly seems worth the effort. Maybe that’s the key here: Having a competitive advantage from  your hardware and your skill with it during a game is far more important and more likely to have you make that effort than being a whiz at moving from cell to cell in your spreadsheet or even triggering one of the 100 or so macros you’ve created for your word processing tasks.

So to the OpenOfficeMouse folks, I say, good luck, but forget about selling one of those mice to me. Now, we start seeing the ‘direct to brain’ controllers, where I don’t have involve my arms and fingers at all with typing and gesturing on the screen but just think where I want to the cursor to go, I’ll be more interested. That would be the 0 button mouse, which I think I’m going to have to address in some future post.

I particularly liked the monstrously bad user interface and industrial design, and how it mimics old ‘rotary’ phones.

Here’s the first one:

The Developer Conference has a very full schedule of sessions, split into 3 tracks. They are all categorized, numbered, and described in detail on the Apple WWDC Web site. While most attendees will want to go to a lot of these 150+ sessions, that’s clearly not possible, and not every session will appeal to every attendee . In fact, the schedule has been in place for nearly a month. What’s been added is the following: You can now create a personalized schedule of sessions and labs that will find its way to your hands, where you’ll need it during the conference. Using the online Conference Schedule, you click a session or lab you’re interested in, then click on the Select button in its information pop-up. (you can also add sessions and labs from an alternate Sessions and Lab page, where sessions are grouped by track rather than by the schedule):

After you’ve selected all of the sessions that you want, like this one:

…you click a link, which downloads a URL to iCal, which then subscribes to that calendar:

Then, when you then sync that calendar with your iPod or iPhone, you now have your personalized Conference schedule for each day on your iPhone:

The other web software that impressed me is the always-handy Internet Movie Database (IMDB). Whenever I’m stuck with that Now what other movie was that actor in?’ question or several like it, IMDB has been a godsend. While several sites are rolling out iPhone versions of the interface, IMDB does a spectacularly good job of it. The clear and sensible breakdown of an actor’s bio or film’s information lets you do that wonderful ‘swivel search’, where you can hop from actor to movie to cast to another actor to movie to director, etc. It keeps perfect track of your breadcrumb trail, and the performance, as well as excellent use of the ‘slide left’ animation for drilling down make it a real winner as an iPhone web app. I hope some of my other favourite sites roll out iPhone versions (Digg, Slashdot, Fark, BoingBoing and a bunch of other wonderful time-wasters, I hope you’re listening!)

But I’m getting ahead of myself here. Surface, besides all of that multi-touch stuff, is a strangely exotic and futuristic product. Most folks would call it ‘bleeding edge’. It’s frankly not something I expect from Microsoft. When I think of Microsoft, I think of Windows™. Windows is not bleeding edge. It’s market-tested, well worn, doesn’t take chances, and is definitely not exotic and futuristic. It runs on hardware that is getting cheaper by the day, and most of the time that hardware is, well, ugly (with a few exceptions from Sony and maybe one or two others).

Surface is none of that. It’s a 30 inch acrylic display with touchscreen built into a rather austere-looking coffee table that’s 22 inches high, 21 inches deep and 42 inches wide. There’s no keyboard and no mouse, although it does have Wi-Fi, Bluetooth and wired Ethernet connectivity. The main way that you interact with it is by touching the screen. Actually, Surface is designed to work not only with more than one finger touching at once, but with more than one person interacting with it at once. Oh, and yes, in case you forget, it’s a computer running Microsoft’s own Vista OS (Windows is still in there somewhere!) and will probably cost somewhere between $3,000 and $5,000. (Update: I just found out that these numbers are $5,000 to $10,000. No big surprise there.) What remains is the question of what you actually do with a $10,000 coffee table touchscreen computer with Internet and wireless connectivity.

Here are some examples of what Microsoft has in mind: (click on these thumbnails to see a larger image).

In the Fact Sheet on their web site, Microsoft says that they will ship Surface (and yes, this is a shipping product) “… to partners with a portfolio of basic applications, including photos, music, virtual concierge and games, that can be customized to provide their customers with unique experiences.” The web site for showing off Vista has several video demos, ranging from happy-smiling-people (a term I learned from my days at Fidelity that refers to those models you see in business ads who always seem to be having a better day than you are) arranging digital photographs, planning a trip, playing cards, and interacting with their cell phones as they place them on the glass table-top display. One of the demos that seems just a little unrealistic has a little girl using Surface to paint a cute picture. I’m sorry, but even though the top 1% of the very wealthy in the US are getting wealthier, it’s hard to imagine anyone but the Gates families and a few others that can afford a 10K electronic paint toy for their kid, much less place it in the living room. What did look the most interesting, however, were the instances where someone was interacting with one or more cell phones, allowing people to download trip information into them by dragging the information into a box that stood for the phone on the screen or moving music tracks from one cell phone/audio player (iPhone? Zune?) to another. The one that made me think ‘Hmm, I don’t think they’ve thought through the social dynamics of this one…’ was an example where a group of diners in a restaurant split the bill and dragged their portions to their cell phones. It might be nice to be able to split a bill with perfect math accounting for each dish ordered, but somehow having that appear on the screen felt a bit…awkward? I suppose it’s no worse than the waitress splitting up the bill, but can you imagine the tug-of-war that might ensue when one or more diners tries to pay for the other?

The fact is, this is a product that is probably going to be seen in casinos and some upscale restaurants and hotels first. The living room will have to wait. I have to say that I like that Microsoft is thinking outside the box (or rather, the desktop or laptop), and some of the applications do look fun. Will this catch on? I’m not sure. For a long time, people were hot on touchscreen kiosks for some of this activity, and they never really took off, and I can’t really see Surface working as a real restaurant table:

Oh, I spilled my Curry sauce all over the computer screen…
Don’t worry sir, I’ll just wipe it up, oh, whoops, sorry to order that round of champagne…
Oh, how cute, little Dylan is playing Blackjack between courses…Oh, don’t stab at the table with your fork, sweetheart…

Damn that messy physical world, full of food, klutzes and kids getting in the way of our cool software technology!

I used and designed interfaces for a touchscreen years ago, and I was struck then by the intimacy of the interactions. Rather than move a cursor via a physical proxy (the mouse, trackball or trackpad), you can touch a part of the screen, and sometimes that piece of the screen would change, just like in the physical world. The examples that Microsoft showed had this happening most of the time. You touch something, it either ripples, moves or highlights. The Surface UI is meant to be more than simply a new display, form factor and method of input, it’s a different style of interactivity that looks like the iPhone writ large (and for two or more people). I’ll bet it just kills the Surface team to hear that.

*It’s worth mentioning that Jeff Han demoed a surface interface at TED in 2006, and much of what he demonstrated is reflected in Microsoft’s new product. Han’s demo also got oohs and ahs, but it has only been seen by a relative few, where the hype for the iPhone definitely went farther into the mainstream media.

A recent paper by Michael Schmitz, a student at Saarland University (a fascinating institution near the borders of Germany, France, Luxembourg and Belgium), surveys several different user interfaces from science fiction movies throughout film history. Human Computer Interaction in Science Fiction Movies cites several films, including Metropolis, Johnny Mnemonic, The Matrix, Logan’s Run, Forbidden Planet, Gattaca, Minority Report, Total Recall, Star Trek: The Next Generation (one of the movies, I assume), X-Men, 2001: A Space Odyssey, Dark Star, the 2002 remake of The Time Machine, and even Galaxy Quest.

It is intriguing the way Schmitz has organized the UIs he is taking note of:

The movie clips in the main section of his survey are categorized according to their area of real-life applications and research:
Neuro Technology: Technologies that connect to brains are introduced here.
Identification: Electronical identification of individuals
Displays: Various kinds of displays as an output medium
Speech: Including speech synthesis and recognition as well as intelligent assistants/avatars as special subgroup.
Other I/O technologies: All technologies that are more specialised and that were difficult to categorize according to the fields above, for example gesture recognition or tangible user interfaces

I think these categories arose because of the scenes that he chose to include. There are perhaps some other movies that were not chosen that may have brought in other areas of classification (for instance, Tron, Blade Runner, Serenity, any of the Star Wars films, The Fifth Element, Ghost in the Shell, any of the James Bond films, 12 Monkeys or Contact).

The fact is, the UIs that we often see in the movies are not always accurate because they aren’t necessarily built for good usability, but instead for good dramatic effect. Sure, Tom Cruise could have done some of his research in Minority Report using a mouse and an LCD screen, but it was so much more dramatic for him to don VR gloves, and perform the exhausting 3D manipulation in virtual space in front of him. Even everyday, mundane events like receiving an email get full-screen 3D animations in the movie Disclosure with Michael Douglas and Demi Moore in 1994.

Nevertheless, movies do have a way of placing imagery and expectations in the mind of many movie viewers. In fact, because a surprisingly large number of people can’t tell the difference between movies and reality, some of these UI inventions may be adopted sooner after they are developed for real, everyday use. What James Bond sees on his computer screen may very well be built by some small startup determined to bring just such a tool to market, at whatever price the public wants to pay. They know they want it, though, because they saw it work just fine at the theatre.

Set for English: (lower case)

Those same keys set for Photoshop:

Wait, what it if was set for the game Quake?

On top of this extraordinary keyboard, there is also a brilliant, if less ergonomically spot-on mouse:

In addition to these bits of Industrial design (and geek fancy-tickling), Mr. Lebedev, who along with his studio, is in Moscow, also writes a sort of blog, although it is not an RSS feed (but I wish he would make it so) called Mandership. According to Mr. Lebedev, it’s been a project since 1997 (which would account for it’s pre-RSS structure). I also notice that the most recent entry is in 2006. I hope he does another few this year. Nearly all of the posts I read were gems of clear thinking about design, the way the world works, and the way people think. No wonder he and his compatriots at his studio keep coming up with such brilliant products, even if some of them are harder to bring out of the concept stage.

A visitor to this blog noted the lack of contact information (such as my email address) in plain view. My answer to him was that this is partly on purpose. As many people know, when you leave your email address on a web page, software that is designed to harvest email addresses can easily grab that address and put it in SPAM databases. I learned from my Kendall Group web site that doing this resulted in a nearly unusable email address (ddrucker@ that address is constantly inundated with hundreds of SPAM a day, and only after I completely removed it and put up a ‘closed for business’ page up has this begun to let up , but not entirely).

There are some solutions for this problem, but I’m not sure which to adopt. Here are the ones I know of:

1. List my address as ‘name “at” domain name’ , rather than an address that is actually written out. This is inconvenient, and relies on the ability of people to figure it out (and machines to be unable to – I’m not sure if they have improved the SPAMbots so that they can get around this). It’s a relatively simple but user unfriendly solution, that is potentially useless if the software has been made more ‘intelligent’ to get around this subterfuge.
2. Create a ‘Contact Me’ form for the initial email, much like a comment, but on it’s own page and with a simple mailto form script. While it’s not particularly elegant (and mailto scripts have their own security problems), it might do the trick. Again, there’s no guarantee that SPAMers might find a way around this one.
3. Include the text “To contact me, please use a comment.” somewhere on the home page. This is easiest, and would have a pretty good chance of getting by the SPAMbots. It is, however, less ‘friendly’, since initial communications would be public and not everyone likes having their opening communication visible to all (even though I can actually choose to not publish the comment and still respond via email).
4. Years ago I’d heard of services where email sent to an address for the first time required the sender to validate themselves (essentially respond to a link in an automatic responder email). I’ve forgotten what it was called, but it sounds good in this case, but I’m not sure what kind of reaction it might cause (since it puts most of the responsibility upon the person who is initially trying to get in touch to verify that they are not a SPAMer)

So, there is my quandary: How to make myself more ‘contactable’ without opening up the door to the inevitable flood of SPAM. I already receive about 2-300 SPAMs a day from my old address, so this is no small issue. I know that there are probably some other solutions, but I do not want to run extra SPAM software on my Mac, and do not want to have to buy a PC to run SPAM software either. I want to stop or discourage these emails before they are sent, not have something sift through my mail to remove them afterward. I sometimes access my email from the road via webmail, so extra SPAM-filtering software doesn’t help there.

Any ideas?

According to the report, this symbol is…

the result of a 5-year project conducted in 11 countries around the world. the new symbol, developed by human factor experts, graphic artists, & radiation protection experts, was tested on a total of 1,650 individuals in Brazil, Mexico, Morocco, Kenya, Saudi Arabia, China, India, Thailand, Poland, Ukraine & the United States to ensure that its message of “danger – stay away” was crystal clear & understood by all.

It’s interesting to note that this symbol follows a bunch of the ‘rules’ of UI design regarding warnings:

1. It’s red. Red is the colour of blood and is often associated with danger. This is the reason that I am always upset by web sites (like Webnames.ca) who decided that their link colour should be red to match their colour scheme. Red should almost never be the colour for your links, people! I’d even go so far as to say that red should be reserved in most UIs for only warnings or errors.
2. It uses a triangle shape, which is frequently used in warning signs because of the sharpness of the angles, connoting danger or at the very least, unpleasantness (if you got poked with it)
3. Unlike the old Radiation symbol (which is the three-sided symbol at the apex of the 3 figures within the sign), it has a verb in it! The arrow with the person running, which looks very much like an imperative ‘Go!’
4. It has some scary looking lines that show the radiation. Although this is not a rule per se, it does illustrate some activity, which is a good example of how an invisible force like radiation can be portrayed.
5. It shows the Death symbol, (skull and crossbones), though this could be misread as ‘Radiation can cause Pirates and make you lose your left foot’ if you wanted to be thickheaded/silly about it. (For a whole series of misreadings of IKEA warnings for comic effect, my friend Matt made up a whole slew of hysterical examples ).

I’m glad that they tested the symbol in countries like Brazil, Mexico, Morocco, Kenya, and Saudi Arabia, etc.. So often ‘Universal’ signs or icons don’t take into account differences in other cultures. I remember some years back something about Men’s and Women’s room signs not working well in countries where pants vs. dresses or skirts were not necessarily the sole marker of secondary sex characteristics. Who knows, maybe some Kilt-wearers in Edinburgh had to think for a minute.

I thought that 9 by 9 might be the limit for icon size, and that this was so small that one colour (or black and white) might also be a rule. Although they are slightly larger at 16 by 16, the free set of icons from FamFamFam called “Silk Icons” are actually in colour. There are 700 of them and many of them are quite good. Let’s hope that a lot of people use them, as they provide some good examples of good icons on the net. I’m particularly impressed with and .

Nevertheless, I believe that when you get below 16 by 16, an interface element probably is for display only. The only exception might be the arrows you see on heirarchical menus, and even then it’s connected to a much larger item (the menu).

Update: My friend Jan has noted that these icons show up as not much more than ‘dots’ on his high-resolution laptop screen. I forgot to take into account that as screen resoloutions go up (and more devices are miniaturized), one has to take this into account. So, I’m thinking that these days the smallest one can get in a UI element is probably closer to the teens or near 20 pixels for any sort of meaningful information. Unless you consider a dot meaningful, which it is at the ending of a sentence, like this one.