Website review: Sephora US website footer
When you’re looking to learn how to improve the findability of content on your website, you don’t have test the whole thing in one gigantic study. Gaining insights doesn’t have to be a massive undertaking.
Online information architecture (IA) testing tools like Optimal Workshop’s Treejack and Optimal Sort, allow you to scale your research activities to suit your project, time and budget constraints. Instead of testing the whole shebang, you can test a small section of your website.
A little research is better than nothing and smaller scale tests are quite useful for gaining buy in from stakeholders and fit nicely into ‘test early, test often’ research programs. Like a travel-sized classic red lipstick, smaller studies still pack a punch! On that note — this website review is Sephora US!
Getting started
Website: Sephora US
Research scope: Footer links and related content pages
Not in scope: Everything else
Tools: Treejack and OptimalSort
A bit of context: For those of you who haven’t heard of Sephora, it’s a large beauty online retailer based in Europe, but also has country-specific websites.
Footers are an important and often overlooked part of online shopping websites. It’s often where we find essential information and answers to burning questions like ‘will this website post to my PO Box’? It contains content that people need to be able to find. For this website review, I decided to focus all my research efforts on the footer and run both a tree test and an online card sort.
The Sephora footer stood out to me because it’s quite large and it contains a lot of information that doesn’t appear anywhere else on the website. If it’s not working and people can’t find what they’re looking for, they may just go shop somewhere else. Running both simultaneously allows you to assess the part of the IA that you’re testing from multiple angles to gain a clear picture of where improvements could be made.
Setting up the study: Treejack
Task questions
I decided to include 6 task based questions for my tree test on Sephora’s footer. I wanted to test as many of the shopping related content areas of the footer as I could while still keeping the study small. The task based questions for this study were:
- You recently moved to California and would like to find out where your closest Sephora store is. Where would you go to find out?
- It’s holiday season and Aunt Rose has given you beauty products containing goat milk and unfortunately you’re allergic! You don’t want to hurt her feelings, so where would you go to find out about getting a refund directly from Sephora?
- You’ve heard through friends that Sephora has an amazing customer rewards program but you can’t remember what it’s called. Where would you go to find out more?
- You’re getting married in a few months and would like to try out a few different beauty looks before you decide on the one for the big day. Where would you go to book in a beauty consultation at Sephora?
- You’ve shopped in store with Sephora but never online and you’re wondering if they post to PO Boxes. Where would you go to find out if they do?
- You bought your mother’s birthday present online from Sephora last week and you’re wondering where your package is right now so you can figure out if it’s going to make it in time for her surprise party! Where would you go to locate your package?
Building the tree for testing
Building a tree is really easy! Since I was only testing the footer in this case, it was a matter of copying and pasting only those labels into a spreadsheet from the live website and then drilling down into each one to get the lower level labels. My tree ended up being 5 levels deep and while the footer appears on all pages, I used ‘Home’ as my starting node as that was the most logical choice. Any other page may have influenced participant responses.
Setting up the study: OptimalSort
Open, closed or hybrid?
Card sorts can be open, closed or hybrid which is a combination of the first two. For this test, I decided to do an open card sort. This means my participants were presented with a series of cards online through OptimalSort and were asked to sort them into groups that made sense to them and to give each group a name. I chose this option because I wanted to learn not only how participants were grouping the content but also what category names would resonate most with them. You can learn more about card sorting via our handy 101 guide.
Choosing the cards
This was a fairly easy process because they’re all sitting right there in the footer (see below).
Just by looking at the footer I had 28 cards — and for a card sort, it’s best to have between 30 and 60 cards. That said, I wasn’t about to throw random cards in just to get to 30! I had a peek at the next level down and noticed two labels that didn’t seem to appear anywhere else: ‘Color IQ’ and ‘Skincare IQ’. I decided to include them out of curiosity bringing my card total to 30.
Post-study questions
I decided to include the same two post-study questions for both tests in this study:
- How confident are you that you completed the task successfully?
- Do you have any comments or feedback about this task?
I included these questions as an opportunity for participants to share how they felt about the activity and provide additional context if they so wished. I asked these questions after they completed the test to avoid influencing their responses or making them feel like they were the ones being tested rather than the footer IA. I set them both to be non-mandatory questions with space for multiple line responses so participants could share as much or as little as they pleased.
Recruiting participants
For both tests in this study, I used the Optimal Workshop recruitment service. All I have to do is share the recruitment brief and the team takes care of the rest.
The same recruitment brief was used for both tests:
- 20 participants who identify as female
- 10 participants who identify as male all residing in the US
I chose this gender split because while Sephora is predominately aimed at women, they do have some men’s products and I also wanted to account for cases where people might be shopping for someone else. Let’s also not forget that gender isn’t exactly binary either! I chose not to specify an age range for my participants because #beautyisageless.
Results: Treejack
This Treejack tree test had a total of 36 participants and of that number, 32 completed all 6 tasks and 4 abandoned the activity. Abandonments are recorded when participants close the tab or window of the the test without submitting their responses.
Task results
Looking at the Overview tab in Treejack, I noticed that this test didn’t perform very well. Task 6 was the only task to achieve a success rate higher than 50% (it came in at 54%) and the highest overall score for a single task in this test was 4/10 and that also went to Task 6. The overall score for each task is an aggregate score of the data collected for that task and tells you at a glance how it performed. A good overall score for a task is 7 or higher. Low overall scores are not great but they do present an opportunity to learn how to make things better, so let’s take a look at what we got!
No participant in this study was able to locate the complete list of stores.
With a direct failure rate of 56%, an overall score of 1/10 and a gigantic scattered pietree, Task 1 highlighted some significant usability issues.
The information that I asked participants to find lives under:
About Sephora > About Sephora > Complete Store List
Double labeling aside, the action taken pietree visualization and the paths tabs in Treejack for this one both show that participants did a lot of wandering and backtracking in their hunt for the store location. There are many labels in the footer IA that could contain this type of content but don’t, and participants in this test visited them all to no avail. If, like 19 participants, you click on ‘Store Locations, Events & Classes’ you don’t find a list of stores, you find information on the types of instore services that Sephora offers.
No participant in this study was able to locate the complete list of stores.
It’s also interesting to note that in Task 4 when looking for an instore service, only 25% of participants were able to find it resulting in another large, scattered pietree and an overall score of just 2/10.
Coming back to Task 1, if you click on ‘Sephora Global Sites’ like 8 of my participants did, you will find a list of Sephora’s international websites. Given the sheer number of stores that Sephora has, based on this test, I think it would be useful for them to review some of those footer labels and the content that lives below.
Now, remember that I was only testing the footer in this study — at the very top of the page there is a store finding tool but all sections of the IA should be usable and this one needs a little work.
Other findings from this tree test:
- Only 47% of participants were able to locate the policy for returning a product that was received as a gift
- 91% of participants were unable to locate information on Sephora’s customer rewards program
- Only 44% of participants were able to find out if Sephora ship to PO Boxes
- 44% of participants were unable to find information regarding tracking a online order shipment
Overall, it’s not pretty but results like these should always be viewed in a positive light because they show us exactly where the problems are and how to fix them. In this case some simple label and structural changes could have a significant impact on overall usability of the footer.
Post-study questions for Treejack
When asked how confident they were in their responses, most participants indicated that they were quite comfortable with how they completed the study. Of the 30 responses received (remember it wasn’t mandatory) 17 were confident, 10 were on the fence and 3 were not at all confident. It’s interesting to note that one person indicated they had struggled with Task 1:
I’m 100% confident on tasks 2-6 and maybe 60% on task 1
When asked if they had any additional comments or feedback, most of the 28 responses I received indicated they did not, however there were two comments that stood out:
‘Kind of a confusing menu.’
‘You could probably find the answers sometimes quicker with less click throughs’.
Those comments are consistent with the results and while they’re not telling us anything new, it is helpful because it provides a different perspective in the form of a direct user quote.
Results: OptimalSort
This OptimalSort test had a total of 53 participants, and of that number, 36 sorted all 30 cards into an average of 4 groups and 17 abandoned the activity. While the average number of groups is consistent with the live website, I did find that abandonment rate a bit concerning given that there were only 30 cards.
Card sort results
Overall the results from this card sort are somewhat consistent with the current state of the Sephora US footer with some cards shuffling their way into different groups. Many of the category names provided by participants are also different to what’s on the current website.
OptimalSort’s Similarity Matrix is a great visualization tool for data clusters because not only can I see the clusters at a glance in the darker shades of blue and the higher numbers along the edge of the diagram, I can also view hard quantitative data on card pairings chosen by participants in this study.
For this study the Similarity Matrix showed a number of card grouping differences between it and the current state, including:
- Card labels ‘My Account’ and ‘Order Status’ currently live under ‘My Sephora’, however in this study participants grouped them with cards related to online shopping and shipping which currently live under ‘Help and FAQs’
- ‘Flash Unlimited Shipping’ which currently lives under ‘My Sephora’ was grouped with ‘Shipping’ by 72% of participants and ‘Online Ordering’ by 75%
- The top 3 pairings for ‘My Account’ were ‘Customer Service’ (66%), ‘Billing’ (63%) and ‘Returns and Exchanges’ (50%) — none of these pairings exist in the current state
- ‘About Sephora’ was grouped with ‘Sephora inside JC Penney’ 27 times
As for those two extra cards from ‘About Sephora’ that I included solely out of curiosity, ‘Color IQ’ and ‘Skincare IQ’, the Similarity Matrix showed that they were commonly grouped with ‘Play! By Sephora’ (a subscription box of samples) and ‘Beauty Insider’ (the customer rewards program). Makes perfect sense to me given that they are all unique products and services offered by Sephora!
Another interesting insight gathered during this card sort related to the category names the participants gave to their groups. Out of the 128 categories created by participants, only one was an exact match to an actual category on the live website. The Categories tab in OptimalSort shows what participants named their groups and despite the earlier mentioned shuffling around of some cards, many of the new category names were still quite similar to the current state.
The one that was an exact match was ‘About Sephora’ and alternate suggestions included ‘About’ and ‘About Us’. It’s not a big enough difference to warrant a change and the card groupings for this also closely matched reality so I would leave it as is. The category name that had the biggest change, however, was the one currently known as ‘Help and FAQs’. Alternates included: ‘Customer Information’, ‘Customer Service’, ‘Customer Support’, ‘Information’ and ‘How can we help you?’. Only one participant created a group called ‘Help’ and there were two others called ‘FAQs’. Participants in this study seemed to prefer more ‘Customer’ centric labeling.
Post-study questions for OptimalSort
Most participants felt quite confident and comfortable with their completed card sorts and Participant 51 gets an honourable mention here with this awesome response:
“Fairly since it is my choice on the grouping. It made sense to me.”
When asked if they had any comments or feedback, most participants indicated they enjoyed the activity and 3 mentioned that they would have preferred more background information on what I was testing and why. I deliberately left those details out so as not to influence their responses but I understand where they are coming from. It’s something that has to be carefully considered every time — you don’t want to be overly ambiguous but you also don’t want to give the game away.
When a website doesn’t test well in a tree test or throws out some interesting card sort curveballs, it’s not a bad thing at all. It’s a chance to learn from your users and tools like Treejack and OptimalSort allow you to pinpoint exactly where the issues are so you can resolve them with surgical precision and get back to being awesome!
Key findings and recommendations for Sephora
- Surface the ‘Complete Store List’ content under ‘Store Locations, Events & Classes’ to improve findability and match user expectations
- Alternatively consider moving the ‘Complete Store List’ label to the top layer of the footer and change the ‘Store Locations, Events & Classes’ label to ‘Instore Services, Events and Classes’ to solve multiple usability issues in one go
- Rethink current footer taxonomy — participants in this study grouped by topic rather than the current state groupings which lean towards a more company focussed way of thinking
- Lose the ‘Help & FAQs’ label and replace with a more customer-centric label such as ‘Customer Service’ or similar
