A guest post written by David Parmelee
Scoping a software, web, or app project is not just a once-and-done exercise
Whether you create your product for external clients, consumers, or internal users, your users will eventually see some version of your product. Maybe that’s a wireframe, a clickable prototype, an interactive prototype with dummy data, or a live product. New feature requests and change requests at any stage of development, especially after launch, are a fact of life for most products that have users.
Sooner or later, change requests from different users will start to contradict each other, or become too difficult to implement. This may make you rethink your product’s purpose. Product teams, engineering teams, and other stakeholders need ways to make sense of that.
Meetings: A Common Internal Solution
The traditional way is to have a meeting that involves experts from each team, including some lead or senior developers who can weigh in with estimates. These meetings are expensive. A weekly 1-hour meeting with 10 people at it would cost your organization 10 person-hours in meeting time alone, plus overhead.
According to game developer Dan Cook, these meetings have 3 other consequences:
- Bugs (or feature requests) end up on the same bug list with little to no prioritization. The developers may choose to fix bugs based on level of effort instead of importance.
- The people who are in the meeting get in the way of the process. They choose work items based on their own preferences or based on a desire to just finish the meeting.
- The work items which are actually most important are not addressed soon enough because they receive too low a priority.
From a UX standpoint, these consequences mean the following:
- Poor prioritization of UX bugs may mean that your users will keep complaining about the same problems.
- When you do discuss UX bugs, committees, politics, and/or user-prescribed solutions may drive the decision on what fix to apply, which could lead to implementing the wrong fix now and receiving a related UX bug report later.
- Even if UX issues in your product cost you real conversions and money, they might stay in your backlog for a long time.
Involving Users
Usability studies allow you to put your product, your competitors’ products, or a work in progress in front of users in your target audience. That lets you observe their behavior and note their feedback – both verbal and written.
Written questions at the end of a session allow your participants to think about your product as a whole. Commonly, they may describe what they liked, what they found frustrating, and what they would change if they could.
Formerly reserved for dedicated lab setups, usability studies have become much more feasible for any product team thanks to remote testing panels and videoconferencing tools. Panel sites do much of the heavy lifting for you. For example, UserTesting tracks task time and satisfaction questionnaire ratings.
Get the secrets to gaining customer satisfaction and loyalty
Reporting UI or UX bugs
Inevitably, usability studies will find problems with a product and ideas for enhancements. Usability, UI, or UX issues can come up during a study session based on what participants do, say, or write. Bad ratings in post-study questionnaires, and poor user reception to your product in general, often stem from these issues.
Naturally, these bugs and enhancements belong in your issue tracking system along with the work items that you created on your own.
As with reporting, reproduction steps for a crash, or any other type of bug, make sure that the problem statement is a concise and accurate description of a problem that a tester encountered. For example, “Was confused about links” is too open-ended. “Clicked on link to Products instead of Solutions” shows a more specific IA problem that may be fixed much faster.
Include screenshots and video clips in your usability bug reports. Video clips let you see the primary source of the problem. They also help sell the need to fix the problem to others in your company. For example, “Here’s a user who wouldn’t convert (and make us money) because he didn’t see our Add to Cart button.”
Learn how to improve your website’s user experience
Prioritizing UI/UX Bugs And New Feature Ideas
There isn’t one standard list to determine other kinds of bugs or new features’ severity and priority, unless you count those that may always be in use within a particular kind of development methodology. There aren’t any that work across all of software engineering either.
Similarly, there isn’t one standard list for usability issues’ severity or priority. It’s better to find a system that works well for someone else and stick with it.
Jeff Sauro has written comprehensively about scales for prioritizing UI and UX issues. Considering a wide range of scales that other usability experts have used, Sauro created a simple 3-point scale, which is specifically for issues found in a usability study:
- 1 = minor (causes hesitation/irritation)
- 2 = moderate (causes delays / more irritation and makes some users fail their tasks)
- 3 = critical (causes extreme irritation and makes any user fail their tasks).
Among other scales, Sauro describes one from Jeff Rubin, which suits itself better to issues that users encounter outside a study setting:
- Irritant (intermittent or cosmetic)
- Moderate (somewhat of a headache to work around)
- Severe (severely limits user’s ability to use the product)
- Unusable (users won’t want to use a particular part of the product).
Dan Cook’s User Pain system yields a single number which accounts for bug type (ranging from data loss to documentation issues), priority (ranging from “it breaks today’s build” to a mere nuisance), and likelihood (percentage of all users affected).
How about new feature ideas? Several products, including Trello, Tricider, and Aha, let users vote on feature ideas.
However, these scales and others like them assume that all users are the same. They fail to consider differences in behavior, differences in technical or domain expertise, and the frequency that a problem occurs. Why is that bad?
- Freemium products typically have a majority of their users on a free plan. Involving these users in voting risks you designing your product for people who will never pay for it.
- It fails to account for people who are not really part of your intended target audience. For example, an online community shouldn’t listen to suggestions from the people who want to harm it.
- It assumes that the users you need to design for the most will be the most involved in voting on new features.
Participants in a usability study or users of your live product will often map onto one or more of your product’s user personas. If the same UX issue or feature idea comes from more than one participant, take it more seriously. Similarly, investigate further if you observe a usability problem or receive a feature idea from someone like your primary or secondary personas.
Measuring Your Success
Prioritization systems intend to give your team a score to determine which bugs to fix. Dan Cook suggested sorting your bugs by User Pain so that you can see the most important ones first and avoid falling prey to preferences and politics.
While prioritization scales rarely account for implementation effort, they provide some insight into the value portion of the value/feasibility matrix. As such, they give your team an ideal to which you can aspire.
As Dan Cook concludes, your team can adopt this policy: not shipping your product unless all bugs above a certain User Pain threshold have been fixed. As you work through your backlog, you can slowly raise this threshold for future releases. Extending that to usability and user experience issues and new feature requests, this system helps your team measure the definite improvement of your product over time.
Learn more about user research and designing from data
This article is adapted from content in UX for Development Shops: Declutter Your Interface, Design from Data, & Increase User Adoption & Retention. You may buy this ebook at https://davidp.io/ebook.
David Parmelee owns Thrill & Create, a user experience consultancy in Maryland. Also bringing deep experience in software development, he now helps development teams increase user retention in their products. His ebook, UX for Development Shops, helps development agencies and developers achieve better business results by focusing on and involving their target users. David publishes a video series, More Than an Interface (>UI), and created Teardowns with a Twist, an innovative way to evaluate existing products. Learn more about David at DavidP.io.
