More from Alex Baldwin
For press releases, advertiser requests, or similar sales campaigns you’re sending a small batch of, usually cold, emails out with a specific ask. Finding potential recipients, writing an email that’s helpful, and tracking the results doesn’t need to take all day or be painful. I’ve watched my fellow product people struggle with the pitfalls of outbound; spend all day trying to find emails, copy and pasting boring boilerplate copy, and then never following up. By stringing together Clearbit Connect, Google Sheets, and Streak, you can knock out these outbound projects super quickly, personalize them, and track progress through a funnel. I’ll walk you through exactly how it works using my most recent campaign, a sponsorship request for Hack Design. Researching for the right audience First up, you’re looking for the most likely people to be interested. For a press launch, that may be who frequently cover products in that space. In our example, Hack Design’s sponsor list, I was quickly able to look at who else was sponsoring comparable websites. For me, that meant researching the advertisers on Offscreen, Sidebar, Dribbble, recent design conferences and podcasts. It’s super simple to save your research and move on to finding the right people at those companies. Add companies to your list Start a Google Sheet with the column name Company. Research the companies most relevant to your outbound campaign. Great places to look are Angel List, Product Hunt, job boards (to see who is hiring in a space), Crunchbase, etc. Anywhere that let’s your group and filter through relevant companies. Add the company name to your list. Finding anyone’s email in seconds Now that you have a list of potential companies, let’s find the emails for the best people to talk to there. For my Hack Design list, I was lucky and had about a dozen sponsors from previous years. However, since we haven’t accepted sponsorship in over a year, a lot of my contacts at those companies were out of date. This process made it trivial to find the new people in those roles and be able to reach out. Get the right contacts from Clearbit Connect Add the column names Email and Full Name to your Google Sheet. Install Clearbit Connect if you haven’t already. This is the secret sauce that will allow you to find anyone’s email, for free. Connect does have a limit but for small batches, you shouldn’t have any problems. Disclosure: I’m a small-time investor in Clearbit. In Gmail, hit the Clearbit button in the top nav and then press Find email. Yep, it’s really that easy. You must start with the company and then narrow down by name or title. Copy and paste that into your Google Sheet. More coming soon Next up we’ll write our email and learn how to quickly customize every single one of them. This article is part one of a three part series, released weekly. Subscribe to get access first.
While in Korea, I used Shazam to record all the songs played in coffee shops. Here’s a mix I put together of all the discovered tracks.
When I started at thoughtbot a year and a half ago, my first project was given to me within an hour of getting my laptop’s dev environment set up. We kicked off a green field project for a non-profit in the education space. They wanted to raise funds digitally for public schools. From our armchairs we talked everything out and drew up the blueprints for a sensible solution. We even clearly defined what success should look like, a working app that could take money and specify what school it should go to. But at the end of the engagement with our working app and success met, if someone asked me if I thought it was going to be a successful endeavor, I would have responded with a greatly exaggerated shrug. The goal of Product Design Sprints is to get from zero to a high confidence ratio in a really quick and collaborative manner. Making a golf analogy, this is hitting off the tee with a driver. You’re doing the bulk of the work in a single stroke and need to aim yourself in alignment with the core problem. Product Design Sprints as we know them, are an invention of Google Ventures’ design team. Their intention is to improve the chances of making something people want, reliably. We want to go from shrug to smug. A formal Product Design Sprint is a five-phase exercise which uses design thinking to reduce the total level of risk in a project by thinking about it holistically and attacking it in a linear fashion. We started doing product design sprints at thoughtbot last year and it’s become integral to our design process. They’ve been so successful, that we started requiring them for green field projects and the exercises used in the sprints have started creeping into the rest of our workflows. Participating in a Design Sprint orients the entire team and aims their efforts at hitting a clearly defined goal. Sprints are useful starting points when kicking off a new feature, workflow, product, business or solving problems within an existing product. Integrating design sprints and design thinking into our product development process keeps us aligned with our goals, and helps us invest our time and money wisely. We’re really grateful for Google Ventures releasing documentation on how they run their sprints, but they are far from first to the game here. Doing historical design research we’ve discovered that small dedicated task forces focused on outcomes appears at least as far back in the 1960s within architecture. The first literature I found about applying the Design Sprint mentality to a digital design problem was actually applied to game design by Valve for their first project, Half-Life. Here’s a very in-depth example of how even with no design sprint experience and a little process, Valve was able to run totally change the game. Cabal Valve’s target ship date was November 1997, a year before the game actually would ship. In development for a year was a Quake total conversion with all new levels and art. Pretty simple, take the existing working concept and make it something novel. But by late September, two months before the original ship date, there was a major problem — the game just wasn’t any fun. There were good things and bad things. Cool monsters, but if you strayed out of the intended play style, the monsters did dumb things. Individually there were interesting levels, but as a whole they didn’t fit together well. Great technology, that was only shown off in one or two levels. As a cohesive whole, it wasn’t working. The obvious answer for Valve, was to work a few more months, gloss over the worst of the problems and ship what they had. For most companies who are left at the whim of their publishers, this is usually the route taken — with obvious results. Since Valve is independent studio, and their team didn’t believe the end result would be something they would want to play themselves, they chose a different route. A very painful route — they decided to start over and rework every stage of the game. So what was it that took this game that objectively sucked, and within the same time frame, make it into what IGN called “the best first-person shooter of all time”. Their solution was to boil the best parts of the existing game into a short single level prototype demo and iterate until it was fun. Then scale it up to about 100 cohesive levels. Valve formed a small group of people to work on the prototype, specialists representing different departments, while the rest of the organization sat around for the month. When making the prototype, if any feature didn’t directly add to the outcome of being fun, it was cut. Features that needed to be added were hacked together the minimum needed to proceed. In the end, they had a short level where Die Hard met Evil Dead, it was perfect. After making the prototype level their dream come true, they were able to extract some theory as to what made it so great. Pacing was distance based, not time based. Allowing players to go through at their own speed. Player acknowledgement, anything a player did would affect the game world. Success and failure was based on player input, not an unfair game engine. During their first failed year, they search for a mythical “game designer” that could oversee the entire process and make it all come together. Problem was that despite looking at hundreds of applicants, no single person lived up to the mythical unicorn like standards they had created. Finally they came to the conclusion that this single person didn’t exist, but they could create a cross-sectional team to combine strengths. This formed their working group called the “Cabal”. Which is a great vocab word, meaning a secret political clique. The Cabal’s task was to come up with a complete product spec including all the game’s monsters, level design, special effects, and design standards. It was to be a massive paper prototype. The Cabal would meet four days a week for six hours a day with minimum five people in the room. One of which would be a dedicated note-taker, the others were engineers or designers that were responsible for shipping components of the game. Throughout the formal Cabal process people rotated in and out to contribute. Cabal meetings went for five months straight and then off and on until the end of the project. By the second month, there was enough of a foundation to begin development on key areas. Into the third month, they were able to start play testing. Play testing was critical, a two-hour session would lead to about 100 action items that would need to be dealt with and feedback was given to the Cabal team. Early feedback allowed them to remove what wasn’t working and expand the best parts. They learned some things about running the Cabal: Everyone contributed, but not every day. Having many different perspectives is what allowed them to not stall out. Artificial constraints caused them to create their best work. Sessions couldn’t last longer than six hours because everyone would be completely drained. Their final document was over 200 pages detailing everything from how buttons worked to what time of day it was in each of the levels. Included tons of rough sketches for level design and specified which technologies, sounds, and animations were required. The script was larger than a 30-hour movie and was showing maintenance problems, so they ended up bringing in a professional writer to maintain the document and craft cohesion amongst the entire work. Within six months of the Cabal at Valve finishing their massive outline, Half-Life shipped and games haven’t been the same sine. Looking back “When you look at the history of first-person shooters, it all breaks down pretty cleanly into pre-Half-Life and post-Half-Life eras.” Clearly it’s a monumental game and the processes Valve created can be attributed to their continued success. This story about the Cabal system was first published in 1999, not too long after the release. What I love so much about it is that it describes a process for solving the ultimate problem, what do we do when the individual pieces work great, but the cohesive whole doesn’t work. In software of all flavors, we struggle with this zoomed-out outcome perspective all the time. This is exactly what product design exercises are meant to overcome, look at individual pieces and evaluate them in the context of the whole. Phases Understand Diverge Converge Prototype Validate There’s five key phases to a design sprint, but they are really more types of activities. Viewing the activities of Valve’s Cabal from a product design sprint perspective, they align in the following ways. Understand: Figure out which existing pieces were adding to fun experiences and what core technologies would power them. Diverge: Try out as many monsters, sound effects, level options, story lines, characters as possible. Converge: Decide which elements were the most fun in the game and fulfilling the purpose while being congruent within the spec. Prototype: Create a master script that will guide the entire game’s development cycle. Validate: First with the prototype and then with added levels, get direct user feedback from play testing sessions. To be clear, it’s obvious that this is very non-linear process and in application this is almost always how it is. For our formal design sprints we try to focus the process to be as linear as possible to meet deadlines and have a set schedule but in the future I think it will be much more fluid. Within the different design sprint phases, we have tested and written scripts for a handful of exercises and then per project choose which we think are the best suited for the particular problems. This is done in a rough pass before beginning work and then refined on the fly as we go through and learn more about individual client needs. Now, I want to share with you a bit more about each individual phases and the exercises that we commonly do during each phase with examples. Prepare Start the process for sourcing user study subjects before you begin. We have some templates through our playbook on how to do this through Craigslist. Having people scheduled to come in on a certain time is the ticking time bomb you need to create a sense of urgency and get full compliance in the sprint process. You need a few other things when starting to do your own sprints. Make sure to have a dedicated conference room. It’s especially important that everyone can leave their stuff in one place and not have to worry about it being moved before the next day of the sprint. Invite the whole team. Bring marketing, business, design, developer, customer support, really anyone who has thought or is affected by the product together. You’ll want to make sure that results are actually feasible and those that will be producing the solution understand enough of the background to build it. If possible, try to have a dedicated facilitator and note-taker for running the sessions. Ideally people not involved in production of the product afterwards, this frees up the actual team to contribute as much as possible and focus on the sprint exercises. Schedule it on everyone’s calendar for the full day. Since things move so fast, having people coming and going makes it to easy to lose track of what’s going on. We’ve found it enormously helpful to go low-tech. Ditch the laptops, with the exception of the dedicated note-taker. Anything that can be printed should be. Take all notes and display them on the wall. By the end of a week, you’re covered in a cocoon of insanity, but being able to quickly filter and sort the results on a wall has led us to some of our greatest insights. For each phase of the sprint, you’ll be going through structured exercises. This structure is what separates a design sprint from a randomly assembled meeting. Each of these exercises should have a recommended time to completion. We enforce this by using a phone timer. Trust me, do not let any of these go over time. Deadlines are wonderful for creativity. You do need someone to play the role of editor, for most of our client projects this is the CEO or some type of manager. This is what separates design sprints from design-by-committee. You’ll be making product decisions based on fitness and informed by the data the entire team has. Each phase concludes with distinct deliverables before moving on, it’s the facilitator and note-taker’s responsibilities to make sure that these are shared before moving on to the next phase. Understand The understand phase is a huge kickoff meeting. For us this is what broke the Google Ventures mold. Some of our clients just have too much existing research to go through in a single day. Thus why at thoughtbot, we split our sprints by phase, not day. The goal of the understand phase is to develop common thoughts including the problem, the business, the customer, the value proposition, and how success will be determined. By the end of this phase, we also aim to have identified some of our biggest risks and started to make plans for reducing them. Clients that come in with existing research rock at this part. We love to review any kind of competitive analysis or ideally any user studies done in the past. One client even brought in her first ten list of companies she had already sold and wanted to onboard to a new product, this made designing a solution for that group of companies much more realistic. Activities for this phase include: Writing a definitive challenge statement. This is a lot harder than it sounds. A great scaffolding for a challenge statement is “Design a way for [specific group of people] to better [situation]”. In example, “Design a way for iPhone owners to better find their keys”. Definitions of key terms. Making a huge dictionary of all proprietary words. You would be surprised with how diluted the word user becomes. Running through competitors products and doing short demos to gauge the space. A key deliverable for understanding the product is filling out a business model canvas. This is to get a top down view on generally how all the pieces of the business fit together. By putting the pieces together here, we usually get our first indications of what’s been previously tested or ignored. We’ve commonly had to use two special sections for findings here that are used throughout the sprint. The first is the assumptions board, this is used to mark things that are wild guesses this early in development. Later we can sort those by what we deem the riskiest or critical and then focus on them. The other special board is for “ideas”. For the sake of time, we can’t spend hours going into some pet ideas, however, we do want to acknowledge them and then refer to them later in the process during our divergence. Diverge The goal of diverging is to generate insights and potential solutions to our challenge statement. Explore as many ways of solving the problems as possible, regardless of how realistic, feasible, or viable they may or may not be. It allows you to get rather crazy about what solutions might or might not work. Even the ones least likely to succeed have an aspect of merit to them. Constantly ask, “How might we…”. It doesn’t have to fit all the constraints. My favorite example of this was a coworker in design school was tasked to come up with 100s of sketches for solutions to a better walker for the elderly. He drew up a plan for a backpack attached to seagulls mid-flight. Seriously let your “How might we…”s be insane and pull out anything that’s relevant. Generate, develop, and communicate new ideas. It’s guaranteed that someone will come up with something wild and unexpected. Group sketching on whiteboards. Mind Mapping individually and as a group. The major deliverable for diverge is the critical path diagram. It’s goal is to highlight the critical path or outline the most important steps required from a user starting to finishing the task at hand. We want to specify what needs to happen, but not exactly how. For generating lots of ideas to use with along our critical path we do quick and iterative individual sketching. Specifically the crazy-eights exercise, which is just dastardly to run with people. You have them fold 8x10 paper twice over to create eight panels, then draw eight different sketches representing ideas for solutions to a small aspect of a problem. Usually giving them less than 5 or 10 minutes. I like to be a bit devious and not specify how many rounds. Three rounds of this gets people groaning like crazy, but has led to some of our most creative solutions. These panels are definitely reused and referred to throughout the entire project. Converge The goal of converging is to take all of the possibilities exposed, eliminate the currently unfeasible and hone in on the ideas we feel have the best chance for success. These ideas will guide the implementation of a prototype that will be tested with existing or potential customers. When converging, we want to: Identify the ideas that aim to solve the same problem in different ways. Eliminate solutions that can’t be pursued currently. Vote for good ideas, we usually do this posting up crazy-eights and doing silent votes with stickers. Our main deliverable for this stage is the final storyboard that we’re going to prototype and pit against the challenge statement. It’s nice to work with previous crazy-eight’s panels that can piece together some of the key steps with the team’s best ideas, then you can fill in the gaps. It’s the mediator’s job to make sure the end result is something that can actually be built, fits the challenge statement, and gets final client sign-off. Prototype Build a prototype that can be tested with existing or potential customers. The prototype should be designed to learn about specific unknowns and assumptions. Its medium should be determined by time constraints and learning goals. Paper, Keynote, and simple HTML/CSS are all good prototyping media. The prototype storyboard and the first three phases of the sprint should make prototype-building fairly straight forward. There shouldn’t be much uncertainty at this point around what needs to be done. For this phase we work individually, having half a dozen people in Photoshop at the same time isn’t super productive. We’ll let our clients go back to their office and do a final review of the prototype at the end of the day. Here’s an example that was made using Flinto, which is great for rapid prototyping iOS apps. The brief was “Tinder for sports recruiting”. This was made pre-sales to make sure that we were on the same page. Short answer was that we weren’t, but she was impressed by how quickly we moved and were able to iterate. Flinto is great because you can send it directly to your users and then it’s installed like a home screen app with it’s own app icon and everything. It’s perfect because it allows them to use the prototype on their own device. My preference is to get as close to the real thing as possible, but still have it be throw-away-able. Seriously avoid the sunk cost fallacy with these things, it’s too easy with a solid code base to say well, it works well enough let’s build on top of this. For that reason I usually don’t recommend using HTML & CSS prototypes unless your team is really comfortable with it and using a framework to speed things up. Keynote is another brilliant option. You get all the Apple transitions and effects you need to fake something looking native, there’s great support with Keynotopia for stock elements, and with iCloud sharing you can give others access to it to tweak copy really easily. Keeping it out of git control is a great way to get everyone on board. Keynote’s a really accessible option. Your results will ultimately only be as good as your testing script. This document should specify what the requirements are and be specific enough with wording that can be used consistently across your team. It’s also great because you’ll have it for the next testing session if you’re going to try and improve a baseline experiment from before. I’m personally pretty fanatical about creating accurate and reusable scripts. For me the testing phase is the closest design gets to science, which is so much fun. If you can’t repeat your experiment, you will be all over the place and not able to use the result of the entire sprint. We’ve had to throw out data points before and it’s usually for asking inherently leading questions. Validate For the validation phase, we typically set up to run three to six people through a very specific script. These are far from statistically significant sample sizes, but are enough to find the major strengths and gaps in the prototypes. The smallest research team can possibly be is two people, a facilitator and a note-taker. When we run our studies, we actually don’t allow the rest of the team to participate or watch. For us, we just don’t have the space and that’s okay. Our research team’s goal is to synthesize the information into a TL;DR: and provide access to the raw data for those interested. Typically research sessions will go in two phases. One’s more of an interview, where I’ll ask them general questions about the problem. Starting with the baseline of having them talk through how they would solve it without technology. Then after a handful of questions around the problem, we will present them with the prototype and a handful of tasks. This photo was taken during a session with one of my favorite participants. She ended up pitching me on her Kickstarter afterwards, which I gladly backed. For this particular client, we were doing research for their product which added “Find my phone” like functionality to household objects, like car keys, using small fobs. We had half a dozen variants for their companion apps to test. Our critical learning was that any visual interface actually distracted them from the task at hand. It didn’t matter how pretty or what information was on the screen. The best thing to do to minimize time to finding the objects was to make the devices ring loudly, letting their ears help them search. It’s a good thing we had a challenge statement that was “How might we minimize the time people who have lost their keys take to find them” and not “How can we improve our visual interface finder”. Our final deliverable is a summary of research. This has proved to be the quickest way to digest an entire day’s worth of research. I like to create a PDF of each of the views and then overlay the information over it. Then summarize with bullet points of the good, bad, and things that just don’t work. By the way if you want your team to practice doing user research, I highly suggest Soundcloud as a demo. It’s an all-around confusing interface that jumps around with different templates depending on what part of the site you’re on. You can learn a lot about how much people hate software by forcing them to use Soundcloud for a few minutes. By the way, I love them, you can a handful of my DJ mixes there at https://soundcloud.com/alexbaldwin, wonderful service with a terrible interface for new users. Reflecting on a year of design sprints has led to a lot of insights into how we’ve adapted them for our needs. First off, it led to a distinctly different behavior in our clients. No longer was it acceptable to give feature requests without significant reasoning and research. Aspects of a design sprint: Starts with a challenge statement. Structured collaborations with set expectations. Many contributors involved across disciplines. Has key deliverables with a feedback mechanism. Future of design sprints at thoughtbot: Design thinking exercises shouldn’t be confined to only be used the formal sprint process. It won’t be a sprint any more, these techniques should be used at all points in the development process. Templates should be available that are customizable to your organization and needs. Better preparation will come in time with ability to anticipate needs of different exercises. For example, with crazy-eights, we’ve printed out pads of work sheets that are custom tailored for that exercise. It feels way more official and structured. The business model canvas is a great example of how useful, repeatable, and flexible these work sheets could be. IDEO’s Human Centered Design Handbook has lots of really nice examples. I can imagine a toolbox of exercises to get you unstuck. Like little open-source UNIX tools for creative collaboration. More info: The Product Design Sprint: A Five-Day Recipe for Startups by Jake Knapp, Google Ventures The Product Design Sprint by Galen Frechette, thoughtbot IDEO Human Centered Design Toolkit Stanford d.School Bootcamp Bootleg thoughtbot playbook
Jonathan Corum, New York Times See Practice until your eyes can see it See what others have done See what’s possible Look at more than you can use Think Find a clear thought and get it on paper Sketching is visual problem solving Sketches are not commitments What are you trying to do? Set the pattern, then show the variations Ugly is fine Sketch with data Sketch in the browser Find something your brain recognizes Remember that aha! moment Communicate that understanding Design Good design is clear thinking made visible. - Edward Tufte Design for someone else Don’t be your own audience If you’re talking to yourself, you’re not communicating Remember what it’s like not to understand If I could have gone back in time, what would help me get to understanding. Explain overview + relevant detail In order to get anything out of this [research paper], it’s going to be a translation project. Show change Don’t collect trivia There’s a reason that I’m collecting that and then presenting it. Collections should add up to something Visualization ≠ counting Find and show meaningful patterns Avoid the language of infographics Avoid the facade of communication. Don’t introduce false patterns Is this pattern a true pattern or an artifact of how I arranged the data? Visualization ≠ explanation Don’t count diapers! Be skeptical of vanity data Compared to what? Produce When your idea hits the real world. Embrace limitations Point and annotate Put the text where it’s relevant. Combine art + label Anticipate confusion It doesn’t say here’s some lines, now you figure it out. Help the reader through Keep honing your ideas We are not designers! - Matthew Ericson Ruthlessly apply common sense Brute force works but tools can help Understand every step in the process Understand what you can control: accuracy, clarity, legibility, empathy, and simplicity What might emerge? Understanding, elegance, and beauty. What if I am tricked? What if I believe in this just because it is beautiful? - Andre Lim See & think = understand Design & product = explain Understand then explain Respect the reader Bret Victor, worrydream Links to examples Static displays Dynamic displays Dynamic data Dynamic Displays Summary and detail graphics - Two levels of questions for different readers. In the example it’s “what are the main contributing pollutants to global warming”, with the follow up being “what are the causes of each pollutant”. Slice and filter - Every decision is put within context. Where you see information, you can interact to filter by that aspect. Narrative context - While going through the data, you’re going through a story. These real world events are tied to the chart. Emotion - Switching between datasets that can tell a story from different perspectives. Don’t let readers sleepwalk through the graphic. When you have the interactive medium you can design a learning curve, similar to game design. Starting a reader off with a basic task and then after awhile let actions grow in complexity. Dynamic Data Black box explanations - you don’t care about what’s going on under the hood, you just want the output. Elements you can adjust and see the consequences of those adjustments. Open box explanations - what’s under the hood is exactly what you care about. Explaining how an algorithm works. Nouns are data objects, verbs are the transitions. Series of interactive graphics explaining each step of the process. Making a graphic is not about putting a large spreadsheet in the middle of the page with the data. It’s much more so like show and tell. Each cell telling a piece of the story, much like a comic strip. Representations for systems design - Show comparisons between original and intended change. Show the state of the individual steps in a system. Come up with a design language and then show the steps through the system. Ladder of Abstraction The graphics we make for displays must be better than their paper counterparts. By showing an animation, you take away the control that interaction allows. Animations are the pie charts of interactive graphics. First, give readers control over time. Let them move at their own pace and answer their own questions as they arise. Second, give control over each of the parameters while abstracting time. When you control a parameter, you need to be able to see the result. Be able to see the entire behavior over time. Third, give control back by letting the reader control time and variable by marrying them together. See also — Steven Wittens, Amit Patel Mike Bostock, New York Times If one of the best designers of all time doesn’t know if what he makes is good, how can anyone else expect to? Good design is as little design as possible. - Dieter Rams Fitt’s law, it’s easier to click on targets that are closer and bigger. Two phases: create and edit. Get fresh eyes frequently; invite criticism. New York Times Preview tool, displays all of the ongoing work. Can switch between each branch and interact with works in progress. Takes a snapshot of each commit on each branch. Prototypes should emphasize speed over polish. Transition from exploring to refining near deadline. Delete code as you go. Be ruthless. Make your process reproducible. Try bad ideas deliberately. Don’t be afraid to fail. Let’s make a bubble map. pbpaste allows terminal to access system clipboard. Edward Tufte, Author The Thinking Eye Code is now design and design is code. What was observed by us in the third place is the nature or matter of the Milky Way itself, which, with the aid of the spyglass, may be observed so well that all the disputes that for so many generations have vexed philosophers are destroyed by visible certainty, and we are liberated from wordy arguments. - Galileo Galilei Organizations which design systems… are constrained to produce designs which are copies of the communication structures of these organizations - Melvin E. Conway Seek verbs and actions. See now, words later. A sense of the relevant. Seek forever knowledge with a taste for excellence. Understanding the difference between evidence and inference. Only two industries refer to customers as users; software and the drug industry. Silence sharpens and deepens seeing. Most of social science studies are false. Social science is harder than rocket science. It does not have the fundamental truths of nature’s laws. Image quilts. Distribution of excellence is extremely logarithmic. A hard look at the practice and results of conventional asymptotic theory easily leads one to three points: We ought not to expect it to be widely useful (since it refers to the ultimate oversimplification). Rather often it has been useful (surprise!) We have developed no basis for telling when it is likely to help and when it is not….. 123 - John W. Tukey Standards of graphical excellence: Google Maps and Swiss Mountain maps. Future is maps moving in time. How do you know that? How does what I see come to be seen by me? Real science has it easy because they have forever laws. Begin with a question, a problem, then measure. You can have your own point of view, but not your own facts. - Daniel Patrick Moynihan
More in design
I haven’t driven a car in decades (New Yorker). But from my pre-New York life, I remember how sun glare could turn a pleasant road trip into a blinding gauntlet. Enter “Sunward,” a uniquely specific demo app my Automattic colleague Peter Westwood just whipped up in Spacefast. Spacefast – for your next idea Sunward – demo app Peter Westwood – software engineer & engineering director Automattic – my work home P.S. Apologies for layout wonkiness. Trying something new. The post Better Day, Sunshine appeared first on Jeffrey Zeldman Presents.
A decade ago, working at a design agency, I lived in fear of one sentence: “this looks nice, can we see a few other options?” I was not an ‘options’ guy. I fashioned myself as Paul Rand, telling Steve Jobs off when he asked for multiple options: “No. I will solve your problem for you and you will pay me. You don’t have to use the solution. If you want options, go talk to other people.”1 But I didn’t have the guts to tell my clients off, so I dutifully designed a few other options.2 Recently, I’ve gotten to work with some of the best designers in the field. Looking at their digital workspace — in Figma, Paper, Claude Design, or something else — I see hundreds of artboards and variants, laid out in a sprawling grid, representing branching parallel universes of design directions. Sometimes I see little panels filled with dials to sweep through the possibilities, enabling a single person to see thousands of combinations in the hope of picking a single, perfect specimen. Lots and lots of options. This makes me feel like an alien. My files are just a few artboards; sure, I tweak and tune the tiny details inside, but I rarely keep the multiverse in view as I do. I’ll give myself a dial here and there, but usually for a single variable that I can’t easily visualize. Am I just a bad designer? Assuming (hoping) the answer is ‘no’, I want to revisit a thread I pull on from time to time: as designers, we’re all just trying to find the right solution in a vast multiverse of possibilities. The solution exists, and we typically know it when we see it. So most of what we call ‘design’ is groping around a dark room looking for a black cat. Different designers take different approaches. Some are changing their approaches with the advent of generative AI, but at the end of the day, design is a search problem. Computer science has worked on search problems for decades. Depending on the particularities of a given search, there are a variety of algorithms and strategies that can result in faster, more direct, or more computationally efficient paths to the solution. We can apply this wealth of research from some of my favorite scientists (looking at you, Claude Shannon) to the design process, to hone our own craft. ‘Search’ has its own formal vocabulary in computer science. I’ve put a glossary at the bottom of the post, and I’ll use dotted underline to link to a definition if you need it. To picture how a search works, imagine a Rubik’s cube. A scrambled cube is the starting state; each turn you can make is a state change, leading to an adjacent state. All the possible combinations of colors on the faces (43 quintillion for a typical 3×3 Rubik’s cube) make up the search space. Searching involves evaluating candidates that can be reached from the current state of the cube; the desired state is the solved cube with each face a solid color. Each possible version of the cube is a state. Each turn is a change that leads to an adjacent state. Now, a few basic search algorithms and how they apply to design: Depth-first search As far as search algorithms go, depth-first search (DFS) is as simple as it gets. Here’s how it works: From the current state, look for adjacent states we haven’t visited. If there is one, move to it. It doesn’t matter which. If there isn’t, go back to the previous state and start from step 1. Repeat until we reach the goal or run out of states. Depth-first search commits to the first change it finds and follows that all the way down before it backs up. Solutions evaluated: 16 In a design sense, this is akin to navigating a hedge maze by picking a random direction at each fork until you hit a dead end. It’s not a very good design process, but it’s a good intro to how these things work. Breadth-first search Breadth-first search (BFS) is just a touch more complicated, and starts looking more like the design process. From the starting state, visit every adjacent state. From each of those, visit every adjacent state we haven’t visited. Keep going, moving one step further from the start each round. Repeat until we reach the goal or run out of states. Breadth-first search tries every single change, then every pair of changes, and so on. Solutions evaluated: 21 Both DFS and BFS are naive algorithms; we don’t proactively evaluate candidates while we search. Both are used on search problems with small search spaces and/or when evaluation is cheap. But many of the most interesting search problems aren’t small or cheap. Take chess, for example; winning at chess is a search problem. But the search space is unfathomably large,3 which is why it wasn’t until the mid-2000s that computers could reliably beat grandmasters. Most product design problems I’ve worked on look more like chess than a Rubik’s cube. Design covers search spaces of hundreds of dimensions — sizing, spacing, hierarchy, color, motion, interactivity, responsiveness, etc. And evaluation is relatively expensive: beyond changing a button or color, we often need to ship something before we truly know it works. There are other algorithms meant for this kind of problem, and they resemble design more and more as they get more complex. Beam search Beam search is the first approach we’ll explore that actually evaluates potential candidates before moving. From the starting state, score every adjacent state. Keep the three best and discard the rest. Score every state adjacent to the three we kept, and again keep only the three best. Repeat until one of the states we kept is the goal, or we run out of states. Beam search scores each option by how many parameters still look wrong, keeping the best three each round. Solutions evaluated: 21 This is called ‘beam search’ because it’s like using a flashlight’s beam to look at new states; the width of the ‘beam’ above is 3. Beam search is the ‘show me some options’ approach that most of my agency work took. It’s limited by the width of the beam: a narrower beam is faster and cheaper, but may miss the best solution at a given step. Now is an important moment to introduce the idea of a heuristic. In most cases, we don’t know we have the ideal candidate until we find it. Given that inescapable fact, we have to pick a test that we think will separate the good candidates from the bad ones. Heuristics usually involve tradeoffs: they can be fast/cheap and error-prone, or slow/expensive and accurate. Here’s the problem: beam search often fails to find the best solution! It depends entirely on the width of your beam and the quality of your heuristic. For example, if the desired state is hiding behind a low-scoring upstream branch, the algorithm might never find it. This can be overcome with an infinite beam, but then we’re just back at a slightly fancier BFS, which is slow and inefficient. When I learned the shortcomings of beam search — that simply checking a few options and picking the best one might not lead to the best solution — my feelings about my option-hungry clients were validated. And yet, designers are still using the beam, because AI has given us a better flashlight: producing a handful of options for a SaaS website used to take a week, and now it takes seconds. The beam has gotten so wide that ‘give me 10 options’ feels like an effective way to prompt AI towards the best design. Still, it’s not how I design. And again assuming I’m not just in the permanent design underclass, I thought maybe there was another algorithm that explains why I tend to explore so few branching paths. A* A* (A-star) is the algorithm behind most route-finding, from maps apps to video game characters. From the starting state, give every adjacent state a score, made up of: The number of changes it took to get there from the start, plus The heuristic: our estimate of how many changes remain to reach the goal. Put them all on a list of states we could visit next. Move to the state on the list with the lowest score. This might not be adjacent to our current state! Score the new state’s adjacent states and add them to the list. Repeat until we’re at the goal. A* scores each possibility as changes made so far plus its estimate of changes left. Partway through, it’s able to revisit previous candidates. Solutions evaluated: 14 This is similar to beam search, but with a few twists: A*’s heuristic incorporates a notion of distance from the goal. A* doesn’t always move smoothly from adjacent state to adjacent state. That last bit is crucial: unlike beam search, A* can go backwards. If the path we’re on moves far from the goal, its score climbs. Eventually a previous candidate looks better and we go back to it. More importantly, how much exploring we do in A* depends almost entirely on the quality of our heuristic. With a perfect estimate, A* goes straight to the solution. A* matches how I design. I’m more deliberate with exploration, only following thoughts until I feel they’re drifting away from the solution. I typically only keep old explorations around if they’re promising; an exhaustive record of every possible variant is stressful. I’ve managed to go the whole essay without mentioning it, but with my A* approach, taste4 does in fact matter; while there are many designers with better taste than I have, what I do have is hard-won by years of designing, and I can use it to sniff out the fastest path to the best solution. Conclusion AI has given us a wider beam, but it hasn’t yet given us a better heuristic. We’re still designing for humans, so we still have a leg up on the robots: we can use empathy, or even our own biases and human shortcomings, to feel out the right solution. Meanwhile, understanding the design process is just as important as ever. This kind of introspection — learning about search algorithms out of despair over my own shortcomings — has added a few tools to my arsenal. There are many variations on these algorithms that may describe your own search process; there are even more exotic variants that may be worth trying on. It might seem reductive to call design a search problem. Where’s the creativity? But with humanity, humility, and curiosity, pursuing the optimal solution with a well-tuned algorithm can help us design faster and more efficiently. Special thanks to Josh Petersel for feedback on a draft of this essay. Definitions Adjacent Two states are adjacent when they’re separated by a single state change. ↩︎ Candidate A state we’re considering. ↩︎ Distance The number of changes that turn one state into another. Usually we measure distance as the least number of changes required to get from one state to another. ↩︎ Heuristic A test we think will separate the good candidates from the bad ones, used because we usually can’t know we have the ideal candidate until we find it. Heuristics involve tradeoffs: they can be fast and cheap but error-prone, or slow and expensive but accurate. ↩︎ Search The process of arriving at a desired state (scoring well on a fixed set of criteria) by evaluating a set of candidates. ↩︎ Search space The set of possible states. We call it a ‘space’ because often there are ‘dimensions’ to search, like the width and height of a shape, and we can make analogies to the dimensions of a physical space. Sometimes these dimensions are literal, as when a wayfinding app picks the best route from point A to point B. ↩︎ State A possible step in the movement from the beginning of a search to the end. We start with the ‘initial state’; as we evaluate candidates we might change our ‘current state’; the end of the process is the ‘final state’ (and may not be the ‘desired state’). ↩︎ State change Also called a mutation. How the system moves between states. ↩︎ Footnotes & References As recounted by Steve Jobs in “Steve Jobs interview about working with Paul Rand,” 1993. ↩︎ The client almost always picked the original (or at least the classic ‘let’s do A with a bit of B and C’). ↩︎ Claude Shannon estimated it at 10¹²⁰. For comparison, the observable universe has about 10⁸⁰ atoms. ↩︎ Say the line, Bart! ↩︎