Resy Website Usability Testing
For this project I was tasked to find a website and perform both a Heuristic Evaluation and a Cognitive Walkthrough in order to discover poteintal usability issues. Then I was tasked to perform a full usability test on the site and report my findings and recommendations.
Solo Project
Tools Used: Figma, Adobe Photoshop, Google Forms, Zoom
Timeline: 4 weeks
Channel: Website
What is Resy?
Resy is an American online restaurant reservation site that helps users in booking a restaurant reservation in over 200 cities. The site is designed such that it helps the user to decide on a restaurant quickly by giving recommendations and by classifying the restaurants as top rated and newly established. The site also features additional functions such as event reservations and guides. Resy also helps user with dining choices by offering a fully realized article section that has detailed up to date opinions on the latest dining and food trends. The site is mainly used by people who would like to make prior reservations in restaurants
Why choose Resy?
Resy is a fully functional site that is more than capable of performing it’s primary function of allowing the user to make reservations. However, upon competing heuristic and cognitive evaluations, the site was observed as having some fundamental design problems. I wanted to do this assignment on a website that wasn’t boken but had subtle problems that the average company may not notice.
Heuristic Evaluation Discoveries
Issues with consistency & standards with how buttons/selectors are colored and placed throughout the site
Problems with recognition rather than recall in that some key links and filters were obscured from the user
Limited flexibility and efficiency of use particularly with the site’s search functionality
Cognitive Walkthrough Discoveries
3 tasks were created to walkthrough how a user would access:
Primary Function (Make a standard reservation)
Search for a specific restaurant in a different city
Access a Secondary Function like reading a curated article from the guides section
The walkthrough determined that the average new user would most likely fail tasks 2 and 3 due to poor link noticeability and task/goal association.
Goals and Objectives
Based on the data from the Heuristic Evaluation and Cognitive Walkthrough I was able to formulate what my usability testing goals and objectives would be.
Determine if the Resy app complies with usability standards
Evaluate the website’s navigation/design helps or hinders the user experience
Gauge how users interact with the site to see what features are under used
Gather user feedback on desired features or improvements they would like to see in the app
Research Questions
This study will strive to answer these questions:
How long does it take for the users to complete each task?
Was the “happy path” to completing the tasks clear to the users?
Are there any roadblocks to the user reaching their goal?
How does the user feel about the navigation on this site?
What does the user think about the number of steps to accomplish a task?
What are the major pain points during the process of going through a task?
Is the information provided on each page easily understandable and allows you to select the option that will lead you to your desired task?
Screening for Participants
The next step in the process was to build an online screener to filter out particpants. The biggest thing I wanted to filter out were people who had used the Resy website before. I wantedto main focus of this testing to be from a new user perpective.
Evaluation Measures
Tasks for each participant will be evaluated for both quantitative and qualitative data. This information will be gathered through pre/post questionnaires and scenario based usability testing.
Quantitative Evaluation
Users will be evaluated on if they can pass each task. This is either a pass or fail Evaluation.
Users' time to complete each task will be recorded. Once the user completes the task or runs out of time, the time will be recorded.
Users will be asked how each task felt in terms of complexity, flow, and expected results. These will be gauged by a 1 to 5 likert scale.
Users will be asked if they found the overall site easy to use or complex to use. These will be gauged by a 1 to 5 likert scale.
Users' path to success or failure will be recorded. Which path the user takes to get to either a successful result or failed attempt will be documented.
Qualitative Evaluation
Users will be given an opportunity to submit their own feedback on “what they would change” to make the site better. This question will be an open ended question to help catch any key feedback that the user might have.
User’s previous experience with reservation systems will be documented. This can be with dining or a similar experience other than dining so long as it doesn’t include Resy specific experence.
Common user roadblocks to task success will be collected and analyzed. If a user becomes lost or confused the path or area which has caused the confusion will be documented
Scenarios & Tasks
Scenarios and tasks were created to test the suspected problem areas of the heuristic and cognitive evaluations.
Testing Areas
Primary Function (Making a restaurant reservation)
Secondary Functions (Events, Guides, and Articles)
Interface Navigation
Filter Testing
Search Functions
Each scenario was built around each task as to provide the user with a level of context needed to preform said task. Having this context in a brief scenario greatly helped the user understand why they were doing the task and helped frame the situation so they could think about what they would do.
Data To Be Collected For All tasks:
The evaluator will collect the users time to complete the task either pass or fail. The time will start once the scenario and task have been read and the user begins to move their mouse. The path the user takes will be documented as well even if the user takes the incorrect route. Any roadblocks encountered that delay or confuse the user should be documented. Any task that has a run time of over 4 minutes will be considered a timeout and the participant will be asked to move on to the next task. Lastly if the user passes or fails the task will be documented.
Scenarios & Tasks List
Scenario: You are planning a trip to visit your friends that live in Austin Texas. On Thursday you want to book a table for 3 at 7:00pm at your choice of restaurant.
Task: Book a table in Austin Texas for this Thursday for 3 people at 7:00pm. You can pick any restaurant so long as it is in Austin.
Happy Path: Homepage>Location>Austin>(user selects restaurant)>Thursday>3 Guests>7pm>Reserve
Conditions for Success: The task will be considered successful if the user picks a restaurant in Austin Texas and makes the reservation for Thursday for 3 people at 7pm. The task is complete when the user clicks on the reserve button.
Task 1
Scenario: Your friends want to go out to dinner this Friday around 8pm. Friday nights are often busy in Chicago so you want to make a table reservation at one of the popular restaurants.
Task: Book a table for this Friday at 8pm for 4 people at a popular restaurant.
Happy Path: Homepage>Top Rated>(Any Restaurant)>Friday>4 Guests>8pm>Reserve
Conditions for Success: The task will be considered successful if the user picks a restaurant in the top rated category and makes the reservation for Friday for 4 people at 8pm. The task is complete when the user clicks on the reserve button.
Task 2
Scenario: You and your friends want to see what special events are happening this weekend in Chicago.
Task: Find what special events are happening this weekend and book the event for 3 people. You may pick whichever event you want as long as it is for 3 people and is in Chicago.
Happy Path: Homepage>Search>Events>(user picks event)>3 Guests>Book Now
Conditions for Success: The task will be considered successful if the user picks any event in Chicago and makes the reservation for 3 people. The task is complete when the user clicks on the Book Now button.
Task 3
Scenario: One of your friends heard you were curious about Italian food in Chicago and suggested that you check out Resy’s guide on that topic.
Task: Find Resy’s guide on Italian food cravings in Chicago.
Happy Path: Homepage>Guides>The Resy Guide for When You’re Craving Italian Food in Chicago
Conditions for Success: The task will be considered successful if the user finds the guides page and selects the article “The Resy Guide for When You’re Craving Italian Food in Chicago”. If the user finds another guide about Italian food in Chicago that would be acceptable as long as it was found in the guides page.
Task 4
Scenario: One of your friends wants to grab steak near where you live in Downtown Northbrook Chicago. You just moved to this area so you use Resy to find the nearest steak restaurant and book a table for 2. Any day or time is fine.
Task: Find a steak restaurant that is in Downtown Northbrook and book a table for 2 people. Any day or time is fine.
Happy Path: Homepage>search>Steak Northbrook>Kamehachi>2 people>any day/time>Reserve
Conditions for Success: This task will be considered successful ifthe user finds the only steak restaurant in Downtown Northbrook which is “Kamehachi Northbrook” and makes a reservation for 2 people. Any day of time is fine.
Task 5
Custom Task Sheets
Originally this project was supposed to be for a small team of 4 to 5 people. Unfortunately, some complications arose with scheduling that made this project a solo project instead. Adpating to this development, I made my own task sheets so I could take notes and document all aspects of testing without needing to flip through packages or notes. I condensed everything into an all-in-one task sheet for each task so I knew what I was looking for and could make quick annotations.
Other Testing Materials
In addition to the custom task sheets, I also made a consent form, pre/post questionnaires, and a script .
Pilot Test
Once all the testing materials were completed, the next step was to run a pilot test to see if there would be any unforeseen issues with the usability test.
Overall, the pilot test went well but there were a few areas that needed attention
Tasks 1 & 2 needed to be swapped. The location filtering task should be the first task as the volunteer may not be from the local area of Chicago. By swapping the tasks the location gets tested first so there isn’t an extra step or confusion in the task.
Each task should be posted in the zoom chat for clarity. The volunteer would often forget task details such as times and number of people that needed to be selected. Posting these details in the chat will give future tests more precision.
The hyper specific location in task 4 should be changed. The volunteer was able to find the target location too quickly due to it being part of the search engine’s default restaurants listed. This location was changed to make it where the user will have to use the full search function to find the target restaurant.
Some non-UX volunteers should be recruited. The pilot test went much faster than expected. The assumption is that because the volunteer was a UX student they were able to spot things more quickly. The main test will include non-UX volunteers to see if there is a disparity in the data.
Note: these changes were already made in the testing materials in the previous sections.
Testing Results
Requirements of this particular assignment called for 5 test subjects. I wanted to use a split of UX/Non-UX volunteers so I recruited 3 students for the DePaul University Research Pool and 2 volunteers from outside the school. All volunteers were able to pass the screener and showed various reservation experience apart from Resy.
Overall, testing went well with no problems occurring during the actual test. The results for each task can be seen below.
Key Findings Summary
During testing 21/25 tasks passed and 4 were failures due to timeout/incorrect selection
For the 25 tasks, testers encountered roadblocks in 14 of them
The happy path was only followed 52% of the time across all tasks
Very limited filter use was observed consisting of only 5 uses out of 25 tasks
Task time and difficultly did not seem to be affected by the user’s UX background
Users had the most difficulties in completing tasks 4 and 5 which were designed to test secondary functions and search bar filtering. These tasks were also the ones that the highest roadblocks during testing.
Many users encountered problems while attempting to find the guide section which could only be accessed for the footer on the homepage. When the users could not find the proper link they opted to use the search bar. The search bar does not include article or guide results, which further confused and delayed testers from reaching their goal.
Initial assumptions on whether or not a participants UX knowledge would bias the testing were not apparent. The sample size for both groups was small but there wasn’t any discernable difference in time to complete tasks or roadblocks encountered for either group.
Task 1
Task 2
Task 3
Task 4
Task 5
Pre/Post Questionaire Results
5/5 of the testers had no prior knowledge or use of Resy.com prior to testing
3/5 testers have used some form of restaurant reservation system before (not Resy)
4/5 testers said they would recommend Resy to a friend
Better search functions/filters was the most requested change to the site
The word “busy” was used to describe Resy by volunteers in every testing session
Findings & Recommendations
The primary objective of Resy is to give the user the ability to make a restaurant reservations from their website. Based on testing the site succeeds in this primary function. However, the site’s secondary functions, general navigation, and consistency need attention. Tasks 4 and 5 show that users would get lost or confused when they could not find access to their goal due to easily missed links and an inadequate search function.
Findings & Recommendations Summary
Based on observations and post questionnaire feedback the following recommendations were made:
Redesign of the homepage is needed (Severity 3: Major issue)
Links in the Footer need to be reorganized (Severity 3: Major issue)
The search & filter functions need to be changed (Severity 2: Minor issue)
The Guides page needs a search function (Severity 2: Minor issue)
Interface items such as buttons and selectors need consistency (Severity 1: Cosmetic)
Redesign The Homepage (Severity 3: Major Problem)
The current homepage hierarchy is poorly organized with information overload being a key contributor. Links are not organized at the top of the page and some secondary functions can only be accessed from the footer. There are many links that don’t help the user accomplish or further their goals that are thrown in with the main links. The core user functions of Resy should be the most prominent and should be positioned in such a way that the user sees and understands their importance a glance. Additionally, the font is very small and filters are barely noticeable (only 1/5 testers used them). Lastly, the location filter doesn’t specify a location outside the city name. It should include the state so the user is certain of the location.
Modify Links on the Footer (Severity 3: Major Problem)
In tandem with redesigning the homepage, the footer needs to have its links redistributed. The current footer houses some of the website’s only links to secondary user functions such as Guides, Events, and Stories. These are fully thought-out pages that have a great amount of content that user would be interested in but they are relegated to the footer. These links should be featured on the top of the homepage in a new menu system. It should be easy to see and access as soon as the user lands on the site. These links were missed by 3/5 testers which resulted in the failure/timeout of task 4.
Improve The Search Function (Severity 2: Minor Problem)
The search function on Resy operates in one mode: find restaurants. While this does work to accomplish the primary goal of the website it hinders users at the same time. The search should include new filters and tags that allow the user to find more specific things. This could be a location within a city or type of meal like the steak restaurant in task 5. The user should be allowed to filter the search for both things at the same time. Testing showed that doing these searches individually did not yield the correct results leaving the user to scroll through results to try and find a match that fits both steak and a location.
Add Search Function to The Guides Page (Severity 2: Minor Problem)
Another design issue that leads to problems with task 4 was that users are unable to search within the guides page for the few testers that were able to find it. This forces the user to scroll through every article one by one until they see something that aligns to what they are looking for. During testing, 2/5 users attempted to use Ctrl+F to find the article they were tasked with but were unable to find it because the site kept loading new articles throwing the browser function off.
Make the Website Consistent (Severity 1: Cosmetic)
During testing, participants noticed a number of consistency issues with how the website displays buttons and selectors. Some of these were difficult to notice and caused some testers to almost fail the task that they were on. One example of this was in task 3, the tester was supposed select an event for 3 people. However, 2 testers almost didn’t select the correct number of people because the selector appears in a different color than what is shown in the rest of the site. To make things more difficult the color the selector is makes it blend in with the other interface items on the screen. Resy should look at all interface items and make sure that they are consistent throughout the website.
