Formative Vs Summative : The User Testing Battle
Ashutosh Chandra | 16th August 2020
Steve Jobs once argued that, “Design is not what it looks and feels like, but how it works.” Formative and summative evaluations are both a form of evaluative usability research. UX Consulting agency conducts them to understand how a product performs compared to benchmarks in the product’s history. Unlike summative, formative is also considered to be an exploratory research method. We use it to gain insight into the subject the product deals with and assists in problem identification and solving. Regardless of the research in formative or summative testing, the goal is to identify and resolve usability issues. Resolving these issues improve a product’s design.
In this blog post, we will understand the definition, difference, and purpose of formative and summative usability research. So, let’s begin.
What Is Formative Usability Testing?
Generally, we perform formative usability evaluation to find and fix problems with an existing product in order to make it more usable. We do it early with simulations and early working prototypes of designs and to explore whether usability objectives are obtainable or not. Formative evaluations tell us what is not usable about an existing product, or, what is broken. The purpose is to discover insights and shape the design direction. Unlike summative, formative testing can and should be done multiple times in a product design cycle. The goal is to assist in steering the design on the right path by discovering usability issues. Then fixing them in the next iteration.
An important aspect of formative evaluation is that the audience for the observations and recommendations is the project team itself. They improve the design of the product or service and refine the development specifications. Results are sometimes less formal compared to summative evaluation, as suits the needs of designers, developers, project managers, and other project participants.
Also, Read | Strategies To Improve A/B Testing Results With UX Testing
What Is Summative Usability Testing?
Summative user testing provides an indirect assessment of the usability of the design. In this situation, we give a task to a group of testers, and based on their performance, we measure the usability quotient of the design elements.
Summative user testing is done after the product has been launched in the market. Conducted by a group of more than ten testers, the data collected are more about the quantity than the quality of the design (as in, what is the percentage of the users who could use the product without running into a roadblock?), and that is why another name for summative user testing is quantitative user testing.
Benefits of formative evaluation vs. summative evaluation
Formative is the way to go when we have to iterate on a product without releasing it. It requires much less rigor than a summative evaluation. It intends to finish rapidly, and in succession in an iterative process. For a formative study, the only audience should be the design team or other stakeholders. It is simply less expensive and less time-consuming than a summative evaluation.
We should perform formative research early and often in the product development cycle. We need at least four to five participants to perform the research.
Some of the examples of formative methods are heuristic evaluation, usability testing, and interviews, contextual analysis.
You might also be interested in reading – A/B Testing In Digital Designs | Benefits, Risks And Steps
Difference Between Formative and Summative Testing
When we speak of usability testing, most people may be unaware of the two types of testing available. This can lead to misunderstandings in discussion and briefings. But the important thing to understand is the distinction between their purposes.
Formative testing is a testing that forms and shapes a design for a user interface. Whereas, the summative testing is about determining the issues, problems, or benefits of an existing or prototype design. Not only the objectives of these two types of research are different. If we observe, the construction of the methodology is in differing ways using separate techniques and devices.
For example:
Moderated and Remote testing
In the formative testing, it is crucial to observe and understand the participants thought processes. Then, their actions resulting from them. The tester moderates and observes the testing in real-time. This records how the user deals with the system and what issues they encounter. It excludes the unmoderated and especially remote unmoderated testing from the equation.
Whereas summative testing is more about measuring and counting. Some of the examples are, recording the time it takes to understand and finish the tasks, counting the number of clicks, pauses, or errors. In these testings, the volume and speed of tasks undertaken are more important than the quality of the observation or narrative.
Conclusion
Even though both testings are widely different, formative and summative testing are complementary to each other. Formative testing offers a direct view of why the design is failing. Whereas the summative testing offers rather solid, yet indirect insight into the reasons for design failure. We need both kinds of user testing to make the design flawless. Contact our UX UI design services for more assistance.
Image source: Google Images
