UX practitioners survey results
Posted 10. Červen 2010on:
As promised I am going to accomplish my UXCampEurope Berlin 2010 participation. For those who missed the whole thing, I prepared an online survey to analyze which tools and methods that are used in UX industry do the real practitioners use and consider useful.
I tried to attract the participants by sharing the URL of the survey by giving the event attendees a physical notice (and I would like to thank Sigy for the „offline tweet idea“).
The survey listed 19 user experience tools/methods in 5 categories:
- Qualitative user research
- Offline surveys/questionnaires
- Online surveys
- Remote user testing
- Full-scale user testing
- Quantitative user research
- Eye tracking
- Mouse tracking/screen recording
- Web analytics
- A-B/Multivariate testing
- Information Architecture/Labeling
- Card sorting/tree testing
- Search keywords research/Text-ads testing
- Mind maps
- Wireframing and Prototyping
- LoFi wireframes/mockups
- HiFi wireframes
- Non-functional prototypes
- Full-functional prototypes
- Interaction Design/Scenarios
- Component/Widget libraries
- Pageflows/Activity diagrams
and the participants were asked to answer how often they use such a tool and how useful do they think it is:
Usage frequency:1-Never, 2-Rarely, 3-Regularly, 4-Often, 5-Every timeUsefulness:1-Worthless, 2-Informative, 3-Helpful, 4-Insightful, 5-Priceless
I hoped for more participants to be able to share significant results. After one week there are only 20 people who completed the form and I feel that I owe them at least some results so I will try to formulate a few hypotheses based merely on qualitative data.
There were 7 participants from Germany, 6 from Czech Republic, 3 from UK, 2 from Denmark, 1 from France and 1 from Spain.
Most of them stated their job title to be User Experience Designer, others are UX consultant, Analyst, Concept Developer, Information Architect and student. One participant was concerned that people in marketing industry might use these tools less often, but i don’t have enough data to support or confute this idea.
So lets start with the numbers. I normalized the data, so it can be read easier.
These numbers show how the participants perceive the listed tools/methods in relation to each other on the scale from 0 (the least used and useful tool) to 100 (the most frequent and appreciated one).
Qualitative user research
I have to admit that these results don’t surprise me a lot. Almost everyone has read Steve Krug’s „Don’t Make Me Think!“ and we all agree that usability is the core attribute of every software product. I think the optimism around remote user testing is interesting although the participants haven’t used it a lot. It may be caused by the fact that these tools had a pretty extensive representation during this year’s UXCampEurope.
Participants also mentioned etnography research, (non)directed interviews, focus groups and other sorts of sessions with the target audience.
Quantitative user research
On the other hand web analytics are widely adopted. The pessimism about its usefulness may be caused by the information overload that hasn’t been successfully tackled yet by any software vendor providing on the market. There is also a positive anticipation around a-b/multivariate testing. The small penetration of mouse tracking and screen recording tools is something that surprised me, because fancy heat-maps produced by these tools are demanded and well received by clients.
I admit that this category was the most vaguely defined. But it seems that card sorting and mind maps are also pretty widespread and seen as satisfactory in general.
The participants also used task analysis and affinity diagrams. We can also argue wether mental models or cognitive walk-troughs belong into this category or should rather live within qualitative user research.
Wireframing and Prototyping
These few questions are in fact only another dimension in the area. I was interested if perfection is widely required in all stages of concept development. We can argue that it depends highly on the size of the projects, but I can see no trend in relying on perfect wireframes or prototypes so the concept phase overlaps with the product development.
In the last category I was pretty surprised by the wide adoption of personas and storyboards, which might be caused by the grouping of the tools. Participants also mentioned swimlanes and object modeling in this category.
These results are not intended to be a full-scale statistically significant research as I didn’t pay much attention to recruiting participants or data cleanup. I just wanted to share an idea and provoke a discussion about the industry and the UX practitioners feelings about its overall maturity.
I am not saying that I managed the whole thing well, but I would really like to hear what you think about the topic rather than the survey itself.