Your EdTech product has a unique and constant challenge: It must meet the ever-evolving needs of your users. And UX research that includes frequent user testing is critical to identifying those needs.
Your research can’t provide meaningful insights without the participation of the right users. Just who the “right” users are depends on the goals of the testing cycle. Unfortunately, product testing sessions tend to lean heavily on the involvement of the same users over and over again: power users. Power users know your product well and use it to its full potential.
But if your product team intends to design features in a way that can be understood by all, relying on power users can, ironically, disempower your product. So you must reach beyond the readily available, highly visible power users and elicit testing feedback from new, casual, and non-users.
3 Problems With Relying on Power Users in Testing Sessions
EdTech companies tend to use and re-use a small base of test users. It makes sense. Why wouldn’t you recruit dedicated, experienced users who are able to devote time to product testing?
So yes, power users bring a lot to the table — confidence, skill, and the willingness to share their opinions and test beta features. You ought to value their feedback; after all, their loyalty helps fuel your product’s success. However, you put your product’s effectiveness at risk by drawing conclusions based on their behavior and preferences alone.
1. Power Users Can Be Squeaky Wheels
Power users are involved. They’re eager to sign up for your email newsletter and will contribute to formal and informal product conversations. Strong communication about your product (with customer service, for example) can be very helpful — to a point.
Their reports and requests shouldn’t automatically be turned into work orders though. You must never lose sight of the fact that power users can push for features and functions that may not serve your broad user base well.
2. Power Users Are Biased
Power users are already extremely familiar with your product, so you need to be mindful that potential loss aversion will surface in testing. They know what product features they personally want to keep. But just because a power user likes a feature doesn’t automatically make it worth keeping.
This inevitable bias can keep power users from being able to provide useful insight to your research team.
3. Power Users Aren’t Product Experts
While power users may have a deep understanding of your product, they are not actually product experts. They don’t know what’s best for your product in general; they only have ideas about what’s ideal for their personal use. Sure, due to their skill and familiarity, they can provide a unique perspective during testing sessions. But it’s a mistake to think their feedback is more significant or more insightful than that of other users.
Tap into the Potential of New and Casual Users
The loyalty of power users can give your product an edge. But if you prioritize the opinions of those already loyal to your product, you’re missing the opportunity to turn new and casual users into your next generation of power users.
New users are using your product for the first time. Their fresh eyes and experiences can show you where users without “training” might struggle. Their onboarding failure or success is especially revealing; you’ll discover if your product introduces the right information at the right time. If they run into friction at this all-important step, they may not be able to complete tasks or seek the help they need to do so. Power users, on the other hand, know direct routes and shortcuts for task completion.
Casual users may still be in the process of learning how your product works. They may not be confident about how your product may improve or adapt to fit their needs, so they probably aren’t engaging with customer support as frequently as power users. Casual users tend to accept their product experience “as is.” It’s a low stakes relationship, especially for students who will likely move away from your product as soon as the semester is over.
Ultimately, including new and casual users in testing sessions needn’t be difficult. Your customer support team can reach out and ask users if they’re willing to participate. You can also elicit user feedback at key touch points in their user journey. Keeping record of users’ persona types will allow you to utilize them later on in further research — and perhaps earn their loyalty.
Find Ways to Engage With Non-Users
EdTech product users who currently don’t use your product can provide unique and powerful insights. Once you’ve recruited them, you can leverage their user experience behaviors and mental models.
Filling out your user roster with a pool of non-users is easier said than done. To enlist non-users to participate in your UX research initiatives, you can:
- Utilize your sales pipeline to engage potential non-user instructors.
- Encourage students you have an active relationship with to reach out to their friends to participate in user research. Offering honorariums incentivizes this as well. No method is as effective as word of mouth.
- Consider online user testing platforms to automatically gain access to a wide pool of non-users. Pre-selected user panels can apply demographic filters to diversify your user panel’s attributes quickly.
Cast a Wide Net When Selecting Users To Test
Power users are a great resource and asset. You can count on them to choose, critique, and recommend your product. But it’s important to remember they may have attachments to features and functions that may not be best for most users.
Testing with new and casual users is essential to your product’s ability to meet the needs of your mainstream users. Non-users can shed light on their allegiances to other products. As a group, all of these types can help keep your product from bias and blind spots.
Listening to a diverse user base in testing can also reveal opportunities in the competitive market — and keep your product on the cutting edge for years to come.
