Mission 08 · Completed · Recommendations delivered
UX DesignEvaluation
A structured usability evaluation of a board-game shopping and design website — expert review, then moderated and unmoderated testing, then recommendations built from the data rather than from opinion.
Project snapshot
- Role
- User Experience Designer
- Project type
- Usability evaluation · e-commerce and design website
- Status
- Findings and recommendations delivered
Tools & methods
- Expert review
- Test scenario design
- Moderated usability testing
- Unmoderated usability testing
- Data analysis
Context
The subject was a website where people both shop for board games and design their own — two quite different tasks sharing one interface. That combination is where usability problems tend to hide, because a navigation model that suits browsing a catalogue rarely suits a creation tool.
The work was evaluation rather than redesign: establish what was actually wrong, with evidence, before anyone proposed a fix.
The problem with evaluating by opinion
Everyone has a theory
Any stakeholder can name three things they dislike about a website. Without a method, the loudest theory wins rather than the most common problem.
Experts see different things to users
An expert review surfaces violations of known principles. It does not reliably predict where real users will actually get stuck — which is why it is a starting point, not a conclusion.
Watching changes behaviour
Moderated sessions produce rich explanation but a slightly performed version of the task. Unmoderated sessions produce natural behaviour with no commentary. Each covers the other's blind spot.
My role
User Experience Designer. I was responsible for analysing the site, identifying problems, designing both the moderated and unmoderated tests, collecting and analysing the data, and advising the recommendations that came out of it.
That is the full evaluation loop — designing the instrument, running it, and being accountable for what the numbers were taken to mean.
Responsibilities
- Analysed the website against usability principles
- Identified candidate problems through expert review
- Designed expert test scenarios
- Designed moderated usability tests
- Designed unmoderated usability tests
- Collected test data
- Analysed the results
- Advised improvement recommendations based on the findings
Method
Three stages, deliberately sequenced so each one narrows what the next has to investigate.
- Step 01Expert reviewA structured pass over the site against usability principles, producing a set of candidate problems and the scenarios worth putting in front of real users. This is the cheap stage, and its job is to make the expensive stage efficient.
- Step 02Moderated testingSessions with a facilitator present, where participants work through the scenarios and can be asked why they did what they did. Small numbers, high explanatory value — this is where you learn the reason behind a failure.
- Step 03Unmoderated testingParticipants complete tasks alone, in their own environment, with no facilitator shaping the interaction. Lower explanatory value, more natural behaviour, and the check on whether a moderated finding was an artefact of being watched.
- Step 04Data analysisResults from all three sources compared, so a problem confirmed by more than one method carries more weight than a problem one participant hit once.
- Step 05RecommendationsImprovement recommendations derived from the findings, tied back to the evidence that motivated them rather than presented as preferences.
Why this design of study
Expert review first, not instead
Running it first turns a broad site into a short list of scenarios. Treating it as the whole evaluation would have delivered a list of principle violations with no evidence that any of them cost a real user anything.
Both moderated and unmoderated
One tells you why, the other tells you whether it happens when nobody is looking. Running only one leaves a known gap in what the study can claim.
Scenarios over free exploration
Task scenarios make sessions comparable. Free exploration produces interesting anecdotes that cannot be counted.
Recommendations bound to findings
Every recommendation traces to something observed, which is what makes a UX report arguable on evidence rather than on taste.
Reflection
The most useful habit this project built is separating the question "is this wrong?" from "does this cost anyone anything?" Expert review answers the first cheaply. Only testing answers the second, and a surprising number of principle violations turn out to cost nothing at all.
It also shaped how I work as an analyst now. The instinct to run a moderated and an unmoderated version of the same question — to ask both why and whether — is the same instinct behind using A/B testing to settle a contested product decision rather than arguing it in a meeting.