Book summary “Sketching User Experiences”: summary

Introduction

You face a constant challenge when creating digital products. Traditional design focuses on objects and features rather than the actual user journey. You might build functional technology that ultimately fails because it ignores social and physical contexts. Incremental updates often trap your team in legacy constraints while competitors capture new markets.

Product success depends on experience-centred design, exploratory sketching, and balancing process innovation with robust prototyping techniques.

Bill Buxton explains how to bridge the gap between technology and human experience. You will learn to use sketching as a cognitive tool, navigate the design funnel, and build studio ecologies that foster innovation.

• How do you design for the wild instead of the laboratory?
• What separates a disposable sketch from a functional prototype?
• How can you test interactive experiences before writing code?

1. Design for the Wild and Cognitive Tools

Technologies gain meaning only within a specific social and physical context, and tools restructure how problems are solved.

The focus of design must shift from material objects to the overall behavioral, experiential, and emotional context of their use. Emerging products feature embedded microprocessors, wireless capabilities, and networking, making them interactive to an unprecedented degree. You are no longer just creating physical items or software interfaces, but rather contexts for experience that act as social entities depending on their environment and who you are with.

Short Messaging Service was originally designed as a simple text protocol between phones. Its real nature emerged through social uses like voting on American Idol or flirting in a dance club.

To create successful products, you must understand and experience the actual social and physical context where the technology will live, rather than relying on abstract laboratory conditions. Technologies do not exist in a vacuum. To engineer the future, you must experience manifestations of your designs in the wild during early development cycles.

Saul Greenberg survived an avalanche because of a coordinated system of equipment, training, and rapid peer problem-solving in a real mountain environment. Avalanches can travel at speeds reaching 200 km/hr, and burial survival often allows a mere 2 minutes of time reference.

Cognitive Tools

Tools do not merely enhance human speed or calculation. They recast problems into different representations that bypass the need for difficult mental tasks entirely. Physical devices and notations act as representational systems that restructure how problems are solved within a specific physical and cultural ecology.

The Ammassalik people of east Greenland carve 3D wooden maps of coastlines and islands that float, fit inside mittens, withstand drops, and require no batteries. These objects pass a 10 metre drop test standard and remain readable through 6 months of darkness in the arctic.

Computing user interfaces have suffered from a lack of fundamental innovation since the 1980s, remaining anchored to legacy desktop computer paradigms. Modern digital devices continue to replicate the conceptual models and flaws of early graphical user interfaces rather than evolving novel interaction paradigms.

A person falling asleep in 1984 knowing how to use a Xerox Star computer would be just as capable of using today's personal computers as driving modern cars. This gap reflects Washington Irving's Rip Van Winkle story from the 1819-1820 publication period, where the main character slept for 20 years and missed the American Revolutionary War.

2. The Apple iPod and the Limits of Incremental Releases

Breakthrough success requires bold industrial design, an ecosystem approach, and avoiding the straightjacket of legacy software architectures.

Apple Case Study

On February 4, 1997, Apple Computer bought NeXT Computer for 427 million dollars, bringing back Steve Jobs. Apple reported a second-quarter loss of 708 million dollars during its period of financial trouble. Jobs was named interim CEO on September 16, 1997, and launched the Bondi Blue iMac in May 1998. This turnaround relied on existing talent like Jonathan Ive, who had been at Apple through its decline, demonstrating that executive vision is required to unlock existing talent.

Taking bold design risks involves inevitable failures, but these missteps are necessary for achieving breakthrough success. Jobs experienced notable failures such as the hockey-puck mouse of the original iMac, which lacked tactile orientation cues, and the Power Mac G4 Cube. The Power Mac G4 Cube, announced in July 2000, was visually stunning and entered the Museum of Modern Art's collection. It suffered from hairline cracks, frequent overheating because its top exhaust vent invited blocked papers, and a steep fall in Apple's share price four months after the dot-com bubble burst on March 10, 2000. Playing it safe is ultimately more dangerous than taking calculated design risks.

The immense success of a breakthrough product like the iPod requires an entire ecosystem, patience, and iterative design across multiple generations. Apple launched the first-generation iPod in October 2001, featuring a mechanical scroll wheel, which evolved through touch sensitivity and touch buttons into the fourth-generation click wheel by July 2004. HanGo Electronics had commercialized the Personal Jukebox PJB-100 with a 6 gigabyte disk size in November 1999, but Apple took three years to become an overnight success following its introduction. The iPod depended on a broader ecosystem including iTunes, music industry negotiations, and distinct marketing campaigns. Apple inverted the traditional Gillette model by making margins on the player hardware rather than the music downloads.

The Limits of Incremental Releases

Relying strictly on incremental n+1 products becomes economically unsustainable as development costs rise and addressable markets shrink. As software products reach late maturity, around the 9.0 release, legacy architecture acts as a straightjacket, user bases restrict workflow changes, and low-hanging fruit disappears. Development costs increase while the size of the addressable market diminishes, causing the law of diminishing returns to take over. Japanese companies lost Y35bn on combined revenues of Y1,000bn in 2004 because developing Japan's third-generation CDMA cell phone technology required fifty times as many people as the second-generation PDC standard.

Software development fails because it lacks an explicit, upfront design phase equivalent to preproduction in filmmaking or automotive clay modeling. Typical software projects jump straight from a green-light stage into engineering without a true design phase, causing them to ship late, over budget, and with bugs. General Motors automotive executives evaluate a full-size preproduction clay model that costs 250,000 dollars and takes over a month to build before committing a new model to production. Inserting an explicit, filtering design process prior to engineering allocates proper resources and defines the product before financial commitments are made.

To apply holistic design:

• Conduct a design process that acts as a funnel to generate, eliminate, and filter multiple concepts.
• Evaluate the form, business plan, and engineering plan of the concept before entering the green-light stage.
• Separate the design process from product engineering management.

3. Anatomy and Characteristics of Sketches

Sketching serves as an external graphic means for technical exploration and aids thought. Mariano di Jacobi detto Taccola first used rough notebook sketches in the fifteenth century to rapidly generate alternatives. In the first half of the fifteenth century, Taccola filled the unfinished pages of his books with hundreds of rough sketches of attack boats.

Sketching serves as an external cognitive conversation defined by quickness, disposability, and appropriate ambiguity.

The Anatomy of Sketching

Sketching functions as a cognitive conversation between you and the external drawing. Ambiguity allows you to discover unintended features and new ideas through seeing and interpreting. Proficiency in sketching requires competence in both rendering and extracting insights. Reading skills develop through years of practice.

Characteristics of Sketches

Sketches are defined by specific intrinsic attributes rather than high fidelity. You must keep them inexpensive and disposable so cost does not inhibit exploration. Their visual vocabulary uses loose lines to signal that the design is open to change. Designer Michael Sagan used loose thumbnail sketches and semantic guide words to capture gestures and themes before refining them into three-dimensional models.

To create an effective design sketch:

• Make it quick or give the impression of being quick.
• Provide the sketch when needed in a timely manner.
• Keep the sketch inexpensive so cost does not inhibit exploration.
• Ensure the sketch is disposable so execution investment does not prevent throwing it away.
• Keep sketches plentiful as part of a series rather than in isolation.
• Use a clear visual vocabulary with conventions like lines extending through endpoints.
• Maintain a distinct gesture with fluidity, openness, and freedom.
• Include minimal detail by adding only what is required for the intended purpose.
• Match the appropriate degree of refinement to the level of certainty.
• Suggest and explore rather than confirm or tell.
• Intentionally include ambiguity to allow multiple interpretations.

4. Experience Design and the Design Funnel

Product design must shift from techno-centric interface design to human-centric experience design because users ultimately judge products by their overall experiential quality over time. Two products can share the exact same user interface, usability, and functional output. Users still experience vastly different emotional responses due to subtleties like physical feel, cadence, and sensory feedback during interaction.

For his 55th birthday, Bill Buxton was gifted the OrangeX juicer by his wife to replace the noisy CitrusMate. Despite the Mighty OJ and OrangeX having identical interfaces, the OrangeX delivered a superior experience due to its mechanical camming effect.

Renderings can be categorized by their intent and purpose into five distinct types to improve design literacy and select the appropriate visual style:

• Sketch for ideation;
• Memory Drawing to record ideas;
• Presentation Drawing to communicate with untrained clients;
• Technical Drawing for actual construction;
• Description Drawing to explain how things work.

Experience vs Interface Design

Sketches are distinct from prototypes and occupy the early divergent stage of the design funnel to handle choice and compromise.

Sketches dominate the early ideation stage when investment is low, options are plentiful, and disposability is high. Prototypes concentrate at later stages as concepts converge and financial investments increase.

A ceramics teacher divided students into a quantity group graded by total weight of pots produced and a quality group graded on a single perfect pot. The highest quality pots were produced by the quantity group who learned through hands-on practice. The quantity group required 50 pounds of pots for an A grade and 40 pounds for a B grade.

Sketches vs Prototypes

Design is fundamentally about making choices among distinct options using clear criteria and accepting compromise among diverse stakeholder priorities. The design process alternates between divergent concept generation and convergent selection. Scott Jenson refers to the balancing of competing priorities between designers and engineers as Yin/Yang Design.

Sketches are social objects that require physical environments and portfolio walls to foster shared awareness, background baking, and collective critique. Displaying work in shared studio spaces allows designs to bake in the background and encourages impromptu team collaboration. General Motors VP of Design Wayne Cherry walked studios at 6:30 A.M. to monitor visible paper work, a practice disrupted by computers until the creation of the Portfolio Wall.

Physical object repositories and mixed-reality toolkits like the Tech Box and CoWall extend studio sharing cultures beyond flat drawings. Sharing 3D physical references, gadgets, and material samples allows designers to evaluate tangible properties like weight and balance. IDEO maintains a physical collection of hundreds of gadgets called a Tech Box across all its global studios, managed by local curators.

5. Studio Ecology and the Twenty-Year Rule

Foundational technologies take decades to mature, requiring shared studio ecologies, constructive peer critiques, and mobile sketchbooks.

Studio Tools

Designers need fluent and simple tools to annotate dynamic media such as video and animations. Film editors in the predigital era used China Markers to indicate dissolves, fades, and wipes directly on 35 mm film. Another example is the 1953 CBS children's show Winky Dink and You, where children used grease pencils on acetate overlays on the TV screen.

Physical prototypes like the Boom Chameleon enable designers to walk around a 3D digital model and annotate it visually and verbally with natural fluency. Around 2000, Buxton's group built the Boom Chameleon prototype to allow studio heads to walk around a digital model of a car, point with a virtual flashlight, and draw on its surface.

Hardcover sketchbooks serve as mobile studios that allow you to safely capture raw notions and half-formed thoughts in the field. Colleague Richard Banks used a Moleskine notebook, known as the legendary notebook of Van Gogh, Chatwin, Hemingway, Matisse, and Céline.

The Twenty-Year Rule

A huge product success is often the second worst thing that can happen because it forces the company and its successors to live with early architectural decisions for decades. Work on Maya began in 1994 and it was released in 1998, and because it remained on the market, design decisions made years prior still impacted maintainers years later.

History shows that it takes approximately 15 to 20 years for foundational technologies to transition from initial laboratory inception to becoming a one billion dollar industry. Doug Engelbart and William English built the first mouse in 1964, but mice did not become ubiquitous until Microsoft released Windows 95 in 1995, taking about 30 years. James Russell conceived the CD around 1965, patented it in 1970, and the industry reached $1 billion in 1990.

A shared corporate vision acts like a fly-fishing strategy, guiding long-term resource allocation and ensuring that creative ideas are successfully productized rather than left in the clouds. Alan Kay noted that it takes almost as much creativity to understand a good idea as to have it in the first place.

6. Prototyping and Storytelling Techniques

Simulating backend functionality through Wizard of Oz techniques and role playing allows teams to test experiences before coding.

Simulation Techniques

You can evaluate user experiences quickly without premature software engineering. The Wizard of Oz Technique lets you test an interactive system before it exists by having a human operator secretly perform backend functions.

John Gould and his colleagues at IBM used a fast typist hidden behind a screen to capture dictated text and display it properly spelled and punctuated, successfully simulating a listening typewriter.

Smoke-and-mirrors prototypes use clever hardware configurations instead of a human operator to create the illusion of a working futuristic system. George Fitzmaurice built the Chameleon device in 1992 by connecting a Casio LCD TV to a video camera pointed at an SGI workstation, using an Ascension Technologies Bird motion-capture cube to track spatial position.

Cobbling together low-tech components allows your team to evaluate collaborative interactions without digital distractions.

Storytelling and Role Playing

Verbal storytelling and playful exploration allow your design team to discover functions not apparent to visual forms.

Seymour/Powell won the British Rail contract for InterCity trains not with drawings, but by describing the design as heroic in the manner of the British Airways Concorde.

Assuming the role of the user with negative prosthetics provides visceral insights that intellectual understanding cannot match. Patricia Moore assumed the role of an 85-year-old woman from 1979 to 1981 by using baby oil drops to blur vision, wax plugs to impede hearing, bandages and splints to limit agility, and taped fingers to restrict dexterity.

Theatrical performances and acting out scenarios help your team validate ideas. Richard Harper wore pajamas to a workplace meeting at Microsoft Research in the winter of 2004 to demonstrate how populating the home with office technology violates the moral order of the home.

When conducting theatrical sketches, you should determine the variables:

• Determine the level of scripting for the scenario.
• Establish who acts as the director or coach.
• Select who will perform the acting roles.
• Define the target audience for the performance.
• Choose the setting location such as in the wild or in a conference room.
• Decide whether the session is a rehearsal with interruptions or a continuous performance.
• Select and design the props used in the performance.

7. Visual Storytelling and Animation

Simple animations, rotoscoping, and video sketches enable rapid prototyping of physical interactions and temporal dynamics.

Designers with limited drawing skills can bypass their shortcomings by tracing real-world photographs in drawing software. Bill Buxton created a recognizable sketch of a mobile phone by tracing a photo in under five minutes.

You can execute this technique using specific steps:

• Take digital photos or scans of images and load them into a drawing program.
• Add a transparent layer on top of the image and trace over it.
• Hide the background photo once tracing is finished to leave only the hand-drawn version for use in other drawings.

Storyboards combined with state-transition diagrams help designers explore screen sequences, interaction flow, and temporal dynamics. Ron Bird and Scott Jenson explored a PDA agenda concept, showing a four-screen transaction sequence linked to an 8-screen grid map. Intentional simplicity in sketched interfaces amplifies our ability to understand experience by removing distracting high-fidelity details.

Simple Animation

Simple frame-by-frame animations can be rapidly constructed to illustrate interface dynamics without heavy video tools. Bill Buxton used a Tablet-PC and drawing software to create a 4-frame animation of a handheld moving, and later built 20- and 22-frame animations to compare one- and two-finger resizing.

Follow this sequence to build animated GIFs:

• Create an image containing elements common to all frames to eliminate frame-to-frame jitter.
• Make variations showing moving parts in different locations across sequential frames.
• Save and transmit the sequence as an animated GIF format for seamless playback in web browsers or PowerPoint.

Picture-driven animation uses hand-drawn motion curves to automate movement along paths, drastically reducing production time. Bill Buxton used PowerPoint to animate a hand holding a PDA moving along a dotted path representing Alexander Mackenzie's route of 1793.

8. Physical Interactions and Bifocal Displays

Combining physical models with video editing and information visualization creates the illusion of fluid system behavior.

Physical Interactions

You can simulate novel interaction concepts like wrist computers with foam-core models. A paper strip represents screen sequences while a touch-sensitive underside controls a magnetic cursor.

Nima Motamedi at the Ontario College of Art and Design built a wrist computer model using a foam-core sandwich, a pull-through paper strip of watch screens, a magnet acting as a cursor, and a map-pin on the underside serving as a surrogate finger. Video editing lets you cut out transition segments so user actions directly cause screen changes.

Information Visualization

The Bifocal Display solves browsing large documents on small screens by keeping the active reading area flat while receding surrounding text. Mark Apperley and Bob Spence at Imperial College London created a 1980 video showing a broadsheet newspaper page pinned to a corkboard and viewed through a square hole in matte board.

You can replicate this basic video in 30 minutes or less:

• Step 1: Cut out newspaper column
• Step 2: Make paper guides
• Step 3: Feed paper over guides
• Step 4: Create Window
• Step 5: Add Desktop of your operating system of choice
• Step 6: Align Camcorder and shoot the Desktop

9. Interactive Paper Interfaces and Parallel Testing

Static and dynamic paper interfaces allow you to test early concepts rapidly before committing to expensive mechanical tooling. You can create and experience an interactive paper interface with Post-it notes and a pen in just 2 minutes.

Testing multiple design alternatives using paper interfaces prevents inflated usability ratings and surfaces original user ideas.

Paper Interfaces

Designers create rough sketches or use Post-it notes and paper proxies to simulate user interaction. A facilitator guides the user through scenarios, manually swapping or updating sketches to mimic system responses, allowing teams to catch design flaws early.

In the 1980s, engineers tested proposed oscilloscope front panel layouts at Tektronix using paper sketches with expert users. This revealed user confusion over an 8x2 keypad arrangement compared to the expected 3x4 layout, prompting a redesign before mechanical tooling was ordered.

Parallel Testing

Users evaluating a single poor design give it significantly higher ratings due to reluctance to be critical. Showing three distinct alternatives helps users judge relatively against what is possible rather than absolutely.

A 2005 study by Maryam Tohidi, Bill Buxton, Ron Baecker, and Abi Sellen tested three versions of a programmable home climate controller using dial-based, tabular, and timeline-based design languages.

When you ask users to sketch their ideal version after experiencing multiple design options, they successfully communicate alternative design ideas through drawing. Maryam Tohidi asked 48 users in the study to sketch their ideal home climate controller on paper, resulting in diverse original design sketches.

• Get a better taste of paper-based techniques, just so you have a bit better sense of the overall flavour.
• Talk about what is wrong or missing in most discussions about so-called paper prototypes.
• Explore how these techniques fit into the larger picture of interaction design, especially in terms of mind-set and methodology.

10 Key Ideas

1. Technologies gain meaning only within a specific social and physical context, and tools restructure how problems are solved.
2. Breakthrough success requires bold industrial design, an ecosystem approach, and avoiding the straightjacket of legacy software architectures.
3. Sketching serves as an external cognitive conversation defined by quickness, disposability, and appropriate ambiguity.
4. Sketches are distinct from prototypes and occupy the early divergent stage of the design funnel to handle choice and compromise.
5. Foundational technologies take decades to mature, requiring shared studio ecologies, constructive peer critiques, and mobile sketchbooks.
6. Simulating backend functionality through Wizard of Oz techniques and role playing allows teams to test experiences before coding.
7. Simple animations, rotoscoping, and video sketches enable rapid prototyping of physical interactions and temporal dynamics.
8. Combining physical models with video editing and information visualization creates the illusion of fluid system behavior.
9. Testing multiple design alternatives using paper interfaces prevents inflated usability ratings and surfaces original user ideas.
10. Product success depends on experience-centred design, exploratory sketching, and balancing process innovation with robust prototyping techniques.