As someone who always dreaded "market research" and data, I have to say that over the years I've become much more appreciative of marketing metrics. Most importantly, measuring consumer behavior, online and offline, has radically improved my marketing results. Enter the concept of user testing. In today's guest post Jason Thai explains user testing and the more common pitfalls associated with it. User-Testing Overview
Anytime a web-based businessreleases a new feature, several issues pop up, keeping product managers tossingand turning at night:
· 1. Arethere any glaring bugs with the new feature?
· 2. How willcustomers view the new feature?
· 3. Did wedesign the feature in the most customer-friendly way?
· 4. Does thenew feature impact other unidentified areas in the site?
· 5. And mostimportantly, how will the feature impact overall conversion?
If only there was a way totest your products before launch… Luckily,several companies have built fantastic user-testing platforms, providing companies with a pool of unbiasedrespondents to provide feedback as if they were real customers. A lot of the sites offer great features suchas video walk-throughs with step-by-step user interaction. Another great aspect of user testing is that thereare few constraints on what you can test. A few of the tests we’ve run include:
· Testingpreferences for before and after webpage designs
· Testingthe ease of our order flow process
· Testingthe usability of new customer features
Also, user-testing isrelatively inexpensive, making it a no-brainer for identifying pitfalls priorto product launch.
What Happens When User-Testing Results GoAgainst the Grain?
However, are the results ofuser-testing always gospel? We, atFoxTranslate, a certified translation service ran into that exact issue. A few months ago, we redesigned our homepage. We felt that the redesign more effectivelyconveyed our key service features and more professionally segmented the content.
Current Homepage | Redesigned Homepage |
 |  |
To validate our intuition, weran user-testing to compare the two sites, expecting our redesigned homepage tobe the clear champion. Unfortunately,the results of user-testing suggested otherwise.
User-Testing Results
If you needed a document translated, which page would you more likely use? | Common Themes for Picking the Current Page | ||
| “The current homepage feels more credible and has a more professional look” “The current homepage feels more secure” “The current homepage is much easier to navigate and gather information” “The current homepage has a much better layout” |
The results showed that usersclearly preferred the current homepage, meaning it might be time to head backto the drawing board.
Call it stubbornness, but weproceeded to test the redesign on our actual customers using Google websiteoptimizer. Google website optimizer tested theeffectiveness of our new homepage by sending half our customers to the oldhomepage and half to the new homepage. Fromthere, website optimizer calculated conversion rate (# of transaction made /site visitors) and revenue generated, allowing us to gauge which page lead tomore transactions.
Results of Current vs. Redesigned Homepage Test
Redesigned Homepage Conversion Rate and Transaction Size Improvement |
 |
Over the course of a month, ournew homepage generated 17% more revenue vs. the current homepage on similar customervisits. 6% of the increase was due tomore converting visitors and 11% of the increase was due to people spendingmore per transactions.
Our Learnings
So, why didn’t the actualresults align with the results from user testing? Sometimes, it’s hard to replicate yourcustomers – Regardless of the results; we are still advocates of user-testing. However, sometimes, it’s hard to findeveryday people to replicate your customers. We’re not selling laundry detergent, so generic user panels can onlytell us so much because they don’t exactly know what your customers care about.
About the Author: Jason Thai is a marketing manager for FoxTranslate, specializing in document translation of business,legal, immigration and academic documents in over 30 different languages.