Auditing the IRS With Treejack
Every year, the date April 15th strikes dread into the hearts of red-blooded Americans everywhere. Yes, folks, it’s the only other certain thing in life, besides death; it’s Tax Day.
To shine a bit of light on an otherwise depressing topic, I thought it would be interesting to see how the public-facing website of the U.S. Internal Revenue Service would fare with Treejack. The IRS redesigned its site in the fall of 2013, and when I visited it to download tax forms in January I noticed that the front page was noticeably cleaner, promoting links to deeper content that seemed in high demand by consumers.
Besides a visual face lift, the horizontal global navigation appeared to be refocused on topic areas rather than audiences, probably in recognition that certain subjects – like credits & deductions – applies to more than one user population. This is a great move by the IRS, as it eliminates the requirement for a user to figure out what population she or he falls into. Or more likely, what population the user thinks the IRS thinks she or he falls into.
As the recent heathcare.gov fiasco has shown, even if a governmental agency or initiative is well-funded, building a website that the general public can navigate easily is a major undertaking. There is usually little or no budget dedicated to usability testing, even though the sites of all federally-funded agencies are required to adhere to Section 508 standards.
For these reasons and more, I thoroughly applaud any major website overhaul by a government agency and fully expected that I would be able to show proof that the decisions of the IRS’s design team were sound. Sadly, however, this turned out to not be the case.
Study Design
When you are testing an existing website’s architecture, sometimes it can be a bit of a guessing game when it comes to deciding which page level elements you should include as nodes in the tree, and how far down to drill. In the case of the IRS website, which doesn’t have a consistent layout from page to page, I decided to include the headers of major page sections, as well as any hyperlinks that seemed to have an emphasized visual treatment. These kinds of items are containers for content and use trigger words the same way a navigation menu does, so it makes sense to consider them as nodes.
After adding the global navigation I was 3 to 4 clicks deep into the site, and ended up with probably the largest tree I’ve yet tested with over 300 rows in Excel. Treejack’s super-easy import tool lets you set up and refine your website architecture in an external spreadsheet, which not only serves as a convenient way to pass the tree around for review by teammates and stakeholders, but also makes it easier to wrangle a large architecture.
I polled my social networks for typical tasks that visitors would try to accomplish at irs.gov, and from this I generated ten questions to ask survey respondents. We recruited 52 participants through Mechanical Turk and then turned the tables on the IRS by subjecting them to an audit.
Results
Overall impressions
Oh dear. I really wanted to be able to say that irs.gov isn’t another web-based federal nightmare, but unfortunately the results tell a different story.
We can see immediately from the Results Overview chart that people had major trouble with this architecture. Over 70% of the answers selected by participants weren’t correct and almost half the time they had to backtrack up the tree before finding an answer they were satisfied with.
Trigger words are key
Treejack’s biggest strength is its austerity. By boiling a website’s information architecture down to its core elements, it removes the distraction (and help) the visual design of a page may offer. As a result when a user has trouble locating the correct area of your site, you can pretty much guarantee that there’s something wrong with how you are labelling things.
Take the results for Question 7 for example, which was one of the three worst performing tasks in this study. The question asked, “Where would you find out if you qualify for free tax preparation?” and we can see right out of the gate 65% of responders headed down the wrong path. This kind of chaotic behavior by users tells me that they are looking for very specific trigger words (in this case probably free and/or tax preparation) and, when they don’t find them, try other paths that seem promising.
The correct answer lay three clicks down, within the “Individuals” subsection of the “Filing” page. Interestingly visitors were attracted to the “Filing & Payment” node in the Treejack test, which – had the users been on the actual IRS website – would have also been appealing because the word Free is so prominent.
Since I told Treejack to ask questions in random order, there’s no way to know whether or not responders had already discovered that there was a node called “freefile” within the “Filing & Payment” branch. freefile is the catchy collective name that the IRS uses to refer to approved tax preparation software, which allows qualifying individuals to file a return for free, but the only kind of free tax preparation to be found here is the kind you do yourself.
Using a “Catchall” Node
One trick I like to use when building trees to test is to include an (Other) node at the end of every single branch. I figure if the participant was adamant enough that the answer should lie somewhere in a given section but none of the available nodes seemed right, the “catchall” node would be a fall back option.
The usefulness of this tactic is illustrated by the results for Question 9, which asks, “Your spouse is a reserve member of the Armed Forces. Where would you find information on special tax benefits available to the military?” It’s hard to believe that the only place I found the answer to this question was deep within the “News & Events” section of the site, and not in the “Credits & Deductions” section where I, along with nearly 40% of participants, believed it should be. Responders indicated their conviction that the answer should lie in the “Individuals” section by choosing the (Other)
node in this area.
Use your audience’s preferred terms
It’s vitally important that a website’s architecture use labels that make immediate sense to the audience who will be using it, in order to prevent users from making mistakes or “overthinking”.
One example can be seen in Question 6, which asks, “Where would you go to obtain a tax transcript from 2012?” The official-sounding tax transcript is unfamiliar to most people, who may assume it’s the same thing as a copy of a previous tax return (it’s not). Judging from the results for this task, participants seemed to think the term “transcript” was something that professional tax preparers used, as evidenced by a preference for areas labeled “For Tax Pros” and “Tax Pros & Partners”.
Advice
What’s the much-maligned Internal Revenue Service to do, given the results of this test? The website undoubtedly made some usability advances as a result of last year’s redesign, where we can see that the global navigation was reworked to be more category-based rather than role-based. It’s also apparent that an effort was made to minimize the use of specialized jargon which would be unfamiliar to the lay public, with the exception of tax “transcripts”.
Besides using clearer labels and trigger words, establishing a consistent layout from page to page will help. Treejack may remove the visual design for the purpose of testing a site’s information architecture, but a good page design is essential to showing what material on the page is most important. First- and second-level landing pages should be visually different from each other, and usage of wizard-type navigation patterns like the one found on the Refunds page should only be used when there is a sequence of events that happen in order.
Lastly, I want to emphasize the importance of user testing. Web-based tools like Optimal Workshop’s products make it fast and simple to test new designs and potential changes in a cost-effective manner, but really any time spent showing draft websites to end users can have great returns on investment.
This concludes my article on irs.gov. Speaking of returns, please excuse me while I go file mine.
Gianna is the senior UI/UX designer for a leading medical institution’s intranet team. She leads large design projects for clinical and operational departments, and helps author and implement enterprise-wide standards for web-based communication. She also designs and conducts user research studies and evangelizes for the human side of human-computer interaction.
Twitter: @giannalapin
Want more website reviews? Check out our Netflix review.
