Recruiting and hiring is among the most difficult and time-consuming aspects of a design manager’s job, and wherever they gather and share experiences, the subject of design exercises inevitably comes up. We wrote about it briefly in our book:
Design Tests?
A topic of some controversy within product design circles is whether candidate interviews should involve some kind of design test or challenge akin to what happens in engineering interviews. Our firm, resolute response to this is “no.” Design tests set up an unhealthy power dynamic in the interview environment, when instead you should be fostering collegiality. The context in which the challenge is given (typically narrowly time-boxed and with only a little information and little support) is wholly artificial—and so whether a candidate succeeds or fails is not a meaningful indicator of actual practice. There is nothing you will find out in such a test that you couldn’t better learn through probing the candidate about their portfolio.
I had hoped that this would be sufficient and never need to be discussed again. Judging by lengthy multi-party threads on Twitter, I was wrong. Forthwith, a lengthier set of reasons for why design exercises are bad interview practice.
Design Firms Don’t Do Them
At Adaptive Path, we hired world-class designers without ever having them conduct a challenge. Same thing back at the first design firm I worked for, Studio Archetype, which was a standard-bearer for early digital design. These are companies whose sole purpose was the delivery of superlative design, and where the value was the talent of the people on staff. How were we able to assess their abilities? As alluded to in the passage above, portfolio reviews, including discussions of how they tackled design challenges.
They are a waste of time
If there’s nothing you can get from a design exercise that you can’t get from a portfolio review and a well-structured thoughtful interview, then it follows they are a waste of time. I call this out because recruiting and hiring is already monumentally time consuming, and anything that needlessly takes up time should be excised from the process.
Design Is Not Engineering
I can’t say for certain, as I haven’t done the research as to where design exercises emerged as an interviewing practice (it’s not from traditional design practice), but my guess is that they came about in technology companies where software engineering was the dominant practice. Design had to overcome its perception as squishy, soft, “make it pretty,” by demonstrating rigor, relying on data, and generally making the practice of design operate more like engineering.
And engineering hiring interviews involve technical exercises (coding challenges and the like), so shouldn’t design hiring interviews?
The thing is, coding challenges are waaaaay more straightforward than design exercises. There are demonstrably better ways to solve engineering problems. And in most coding exercises, the outcome is predetermined — it’s a matter of how would you realize it?
The same is not at all true for design. You’re not applying process to realize an already known outcome. You’re taking in a massive amount of input in order to navigate your way through the problem space. Unlike engineers, you need to consider business context, user needs, goals, and capabilities, brand concerns, technical constraints, channels of use, and god knows what else. And good designers know that there are many potential solutions to a problem, and require testing and iteration to get to anything like a good solution.
Design Exercises Bias Towards Facile Problem-Solving
Designers don’t all solve problems the same way. Some take in a lot of data, go off into a cave, noodle on it for a while, and come out with something great. Others iterate and prototype almost from the get go, uncovering solutions through refinement. Some require thinking out loud, and deep collaboration to get their best work. A great design organization has people with a variety of problem-solving modes and approaches, which enables the organization to better tackle a wide array of challenges.
The artificial constraints of design exercises (typically time-limited; a problem that the candidate isn’t prior familiar with, but which the interviewers are; performing under the scrutiny of others) biases toward a narrow range of problem solving.
A design exercise, by its very nature, is inclined towards facile solutions, and so biases teams towards facile designers. There’s not really any room for grokking depth.
Design Exercises Exacerbate An Already Problematic Power Dynamic
Design exercises ask candidates to perform on demand. In the context of a job interview, this only heightens the fraught power dynamic between an employer and prospective candidate. Even in markets where talent is in high demand, job interviews place candidates in a vulnerable situation. Being expected to perform on demand only adds to the candidate’s stress and anxiety, and makes for a suboptimal candidate experience. This Twitter exchange between my friends Ryan and Jared touches on this…
As Jared notes later in the thread, design exercises introduce cultural bias, too:
Shown to be highly biased against women and people of color, especially those who come from cultures where questioning or interrupting authoritative strangers is seen as social insubordination.
— Jared Spool (@jmspool) May 10, 2018
What about take-home exercises?
This is often the response to my ranty diatribes against design exercises. What if they’re take-home? Then people have all the time they need, and it’s the pressure cooker of performing-on-demand is.
Beyond the obvious problem, that’s still at the root of all of my issues with design exercises (for the people in the back: THEY ARE ARTIFICIAL CONSTRUCTS THAT DON’T REFLECT HOW DESIGN ACTUALLY HAPPENS), they introduce new issues… Namely, now you’re asking this person to do unpaid work. Young people with savings (i.e., don’t need money) and free time will be able to put a lot more effort into take-home exercises than, say, a single parent whose at-home time is focused on their children, and can only do “homework” after the kids asleep and when they’re likely exhausted.
Recognizing this, some companies do offer to pay people for taking the time to do a take-home exercise (which can help defray costs like child care), and that’s better than not doing so, but even better…? No exercises. Because you don’t need them. Because they add nothing to the recruiting and hiring process that can’t be figured out through thoughtful, experienced-based interviews, a savvy portfolio review, and speaking with people the candidate has worked with.
Org Design for Design Orgs 
I’m a design professor, and when companies give our students take home design exercises, they have to stop working on their studio work in order to focus on the exercise. It detracts from what they are in school to do, disrupting team projects and solar research, as they put time into work that they can’t use in other places and for which they are not paid. Students put more time into it and the companies suggest that they should, but of course they do. They want their best shot at getting a good job.Moreover, companies reuse their prompts, so it isn’t like the companies are benefiting from the exercises either.
LikeLiked by 2 people
Design interviews definitely shouldn’t be taking cues from engineering interviews, since we engineers aren’t sure that interview coding challenges are worthwhile. The whole industry is collectively assuming (and hoping) that there’s a strong correlation between designing an algorithm in an interview and employee effectiveness, but there’s really not a lot of good data on it.
LikeLike
Interesting thoughts. Peter, how do you recommend a candidate respond when asked to do a homework design challenge, if they are still interested in the job?
LikeLike
Given the advent of “designers” (and design being so multi-faceted and so diverse) in the world, and the proliferation of “free” design work, artwork, UI/UX templates, and more, it is becoming increasingly difficult and soon an impossible task to determine someone’s hard to design skills purely from a portfolio. The questions that arise are; who worked on this? Was it a team or just yourself? If so, which part did you design? So on and so forth … to name a few, this is layered with potential issues and offense. So in the interest of understanding, someone’s hard skills, i.e. attention to detail, basic design language, structure, form, and function thinking; what is the harm in issuing a well-thought-out single user story to a potential candidate as a design exercise, at tops maybe 4/5 mobile screens or flows? In my experience serious candidates have gone out of their way to roll up their sleeves and pull out their toolkit who then say ‘bring it, it would be my pleasure’. It shows enthusiasm and love for what they do and a great filter to determine the good, the bad, and the ugly. And this is met with greater enthusiasm by the employer. Who follows up with interviews, that cover cultural fit, personality, and more; that leads to great ROI and excellent retention. The sum of the exercise (alone) is greater than the sum of the entire interview process by magnitudes.
Furthermore, yes “Design Is Not Engineering”, and Engineering is not Design either, no Engineer would be asked to design UI/UX, it simply isn’t the best use of their time or their discipline. Context and relativity are important. We are not asking designers to engineer or code something, we are asking designers to design. Why should engineers be given exercises and designers not? The argument here that designers need to consider vast inputs such as business etc is an oversimplification of what current forward-thinking tech companies and startups are doing in the real world. Exemplary engineers need to consider the same volume of input as do designers and often the same ones and sometimes different ones! Just as in design, there are many different possible variables for output as there are engineering, it really depends on what the specific user problem is. So context and relativity are important. I do find this specific argument a generalization, somewhat outdated, and perhaps even “waterfall” thinking.
Ultimately, an exercise whether it is for a designer or engineer is only one evaluation criteria and a single data point among multiple other interview steps, that include human connections, understanding, and agreements.
Writing isn’t the best form of expression for dialogue, thank you for your understanding in advance.
p.s. I’m a Designer of 20+ years 🙂
LikeLiked by 1 person
Aren’t you simply asking them to become a UI designer, pushing pixels to create a few screens when there’s no actual users to research and no mvp to test? Seems flawed if you’re looking for a full stack Product Designer
LikeLike
I understand your thinking Asim, but if that the only way for companies and hiring managers to decide then they haven’t grasp how to conduct well interviews and portfolio review. You can really know who the “serious” designer are by actually having a conversation about their work and design process. Other professions don’t ask their candidates for test and believe in their skills and abilities. On another note, hire character , skills can be teached.
LikeLike
Hi Peter! I agree with you overall. But I wonder how you would address Asim’s point, which is it is very easy to use templates to make your portfolio look polished. Can you give some specifics on the kinds of thoughtful, probing questions you ask around a portfolio that provide the kind of data you need to feel comfortable with a designer’s skill set?
LikeLike
I think just have to be honest and ask the designer about the process and how they came to those solutions in the design. I think hiring managers are taking the easy way out and now asking the right questions to access those skills.
For example, think of other professions and how they would ask those questions
I use to be a teacher and for interviews I never had to do a test to prove my skills. What they did was ask thoughtful questions about education philosophy and scenarios based questions on classroom management if I was a “phoney” I wouldn’t know how to answer that but because my years of experience and I can answer the same time as designer! Learn to ask better questions.
LikeLike
100% agree with the OP.
I’ve done a few creative tests in the past year and none of them have led to a job. The briefs were vague and judging by the language in them, written by company heads who have little grasp of the creative process. They all encourage use of creative license but then respond with ‘yeah but it’s not the style we really wanted’.
The logic behind these make no sense; how does a single hypothetical scenario inform one of how the candidate will handle the multitude of varied tasks they’ll be working on as an employee? I wish I could say I got something out of doing these tests, like experience but none of these have advanced my skill whatsoever. All I ever got out of these is disappointment from the subsequent rejections.
It’s a flawed practice and a complete waste of time on both ends.
LikeLike