You will learn how to analyze and improve customer experience, create products and provide services that customers need, implement service innovations
How do we measure design quality? I heard this question in all the companies where I worked. It wasn’t just the designers who asked it. Answers range from “we know when we see” to decisions based on analysis based on design principles or detailed and thorough product and design reviews. Other suggested solutions include surveys such as NPS (Net Promoter Score), CSAT (Customer Satisfaction Score), or individual user surveys such as “How much do you like our product?”
Each of these solutions has a number of problems. Internal evaluations can be skewed by intrinsic motives or blind spots. Surveys may fail to highlight design quality or may add unnecessary burden to the user.
Several years ago, I led a design and research team at a company that was just getting caught up in John Doerr’s philosophy and OKRs. We measured everything, including the quality of the design.
The scoring engine we ended up with was an extended set of usability and product heuristics based on Jacob Nielsen’s 10 UI usability heuristics. We concluded that while the design team may be biased in evaluating their own work, it is clear that designers need to know what “good design” is, as criticism is an important part of our practice.
Several colleagues and I developed a set of heuristics that fit our overall product design structure: design thinking, UX and UI design. An additional benefit was that we looked at the qualifications and skills of designers in similar teams, so when assessing quality, we could easily answer the question “where in this project did the designer demonstrate strengths or capabilities?”
To evaluate the quality of the design, we got together in a small group after the launch of a new product and gave a quick overview of the work done. We simply answered “yes” or “no” to each question, and applied the formula to determine the percentage of quality score (100 – yes, 0 – no).
Although this did not give an accurate result, we recorded resonant options from project to project and were able to give feedback to designers, as well as provide information about quality to non-designers.
This is a heuristic.
State of the system
A clear task
Recognize, not remember
Note that they have been adapted to our understanding of design quality and may vary depending on the context. For example, when shaping design principles, we focused on how people learn to use a product.
While this approach worked for us, we admit that it cannot be scaled outside of our small team. For example, a team of hundreds of designers submitting designs daily is unlikely to be able to manually check them. However, as designers learn to moderate their work, we believe that this approach will enhance design quality – which will help measure product success and performance – for both the company and our users.
Thanks to jennie § yip and Casey Callow for help!