Disclaimer:
Content still under NDA is Censored
A group effort completed for the Ministry of Labour, I was tasked with testing and finding UX and UI issues for a confidential project as part of a joint IXD (Interaction Design) and MOL (Ministry of Labour) effort. Due to the project still being confidential the content presented will be on the tools and process utilized rather than the product.
Working in a group of three we began analysis, focused on getting our tester's hands on the product. Discussing with the professor and going over the product's purpose we knew the client's biggest concern was the UX side of the product. With our purpose in mind we then got to work creating numerous documents in preparation for our tests including guides on roles for each other during the testing, recruiting procedures, NDA forms and many more. Through this process we came to fully understand the extent of our goal and how to achieve it.
Pre-Testing Documents
With the many papers needed for our tests finalized we went about recruiting from the target audience outlined by the client utilizing our previously established recruitment procedures. Tests were held in private rented rooms in which we prepared a laptop as well as a secondary screen displaying the screen capture in order to more easily observe the process without disturbing the tester's process. Using a team of one moderator whose job was to directly interact with the tester, one time keeper and one note taker in the end we were able to schedule four one hour tests for the client. During these four tests roles were swapped between us.
Testing Setup
With our four tests concluded the next big step for our group was organizing and finding meaning within the massive amount of collected data (four hours of video and written answers). Reviewing the footage we turned each video into extensive written transcripts which we then used in combination with notes and the video to find statements made by users. Each problem encountered by a user was then inserted into a Google Sheet under the user's assigned number, the stage it occurred in and the time slot in the accompanying video. This stage allowed us to gauge more accurately the user's response to their experiences in each section.
Tester Quotes Organized
With this multi-page sheet finished the next step was to find and identify trends among the user's reactions. Organizing issues into general or global issues and those localized into specific sections we were able to paint a clearer image of user issues within each section. Using censored videos of the problem and the user's reaction to reinforce the findings. Through this process we were able to present specific problems multiple user's had on specific stages of the test while also pointing the client to the exact section. Within the many statements made by users we were also able to give the client direct recommendations from the users for each of the problems encountered. Due to our extensive charting of quotes we were also able to present videos to the client of each quote.
Issue Slide
We also wanted to provide more data for our Client and so created an in depth analysis of the time user's spent on each section of the test as well as their answers for post-test questionnaires. All of this data was charted utilizing info supported by the time keeper's document as well as the videos, brought together to form this Excel document. Using this data we were also able to chart the Mean, standard deviation, median and a confidence interval within the Excel sheet.
Excel Data Tables
Utilizing this Excel sheet we were then able to present the user with charts for ease of understanding. Show casing the mean time for each section, the mean time average, confidence interval data for each section's average completion time and a chart regarding user responses to the post-test questionnaire.
Charted Data Sets
All of this info was then presented to a Client in person having been assembled into a quick and understandable presentation as well as a finalized report containing even further details. The Client was pleased with the data we'd collected as well as the supportive material for each of the problems identified.
