Treejack Improvements: Bloopers and Deleted Scenes

Andrew Mayfield

We’ve recently overhauled the Optimal Workshop analysis tools across the board. It was a total rebuild project (ask me how I feel about rebuilds over a whisky sometime) with the aim of "Parity, but better".  I'm able to now say that we've done it. It's a dream come true, or a nightmare come to an end, depending how you look at it. We're publishing a separate post next week announcing what's new.

For now though, this is a short post with an apology, for we removed a feature from Treejack we'd thought would not be missed — and we were wrong.

Down with the spreadsheet! Long live the spreadsheet!

When Treejack was first built back in 2009, most results and analysis tools could be found in the downloadable spreadsheet. We knew this had all sorts of limitations, but it also gave researchers all sorts of flexibility when analyzing their tree testing results.

We recently finished doing what we've been intending to do all along, which is adding all available reports to Treejack itself. Researchers can now access First Clicks, Paths, and Destinations data alongside the usual suspects (Task results, Pietrees, Question responses, and so on).  We then thought it sensible to leave them out of the new “raw data” downloadable spreadsheet, as after all, they're summary reports, not raw data at all. And we started work on actual raw data exports, quite certain we'd improved things for everyone.

But alas, this wasn't the case. We found out that some of you felt like we'd pulled the rug out from under you. Blooper. Yes. I’m very sorry about that. It turns out that lots of people are using the downloadable spreadsheet for further analysis.

This is so obvious in hindsight. How researchers use the spreadsheet is something our user testing hasn't touched on much, and as it occurs in other applications (like Excel) it’s not in our analytics either. This is one of the shortcomings of our user testing routine, I guess. We don’t often get the opportunity to observe people in their own context, doing the things they do: taking their research right to conclusion, generating their own reports, answering a colleague's requests for more detail, or working on alternate presentation methods to suit their own special project needs.

We have taken all the feedback very seriously. As painful as it has been to effectively remove a feature and find ourselves in hot water with our beloved user research community, and as sorry as I am that we’ve caused such an inconvenience for a lot of people, I really do appreciate all the things we’ve learned about how folks are using Treejack. We’ve already made some small changes, and now have new ideas for making it even better in the future.

Today though, I'm happy to tell you that the full spreadsheet is back: it contains the First Clicks, Paths, and Destinations data that has been conspicuously missing this week.

As with everything though, we've had to completely rebuild the export mechanism for the new architecture and some things have changed. So here's a list of the differences you can expect to see in the new Treejack spreadsheet download (with detail for those of you who might need to adjust some macros):

  • It is now an .xlsx file, not an .xls
  • All tabs
    • Colors have changed. They now coordinate with what you see in other Treejack reports
    • Tabs are ordered more logically: First Clicks, Paths, Destinations
  • First Clicks tab
    • Added a column on the left for task numbers
  • Paths tab
    • Now distinguishes between direct/indirect failures and skips
    • Paths no longer begin with a superfluous space character
    • Added row at the top for task numbers
  • Destinations tab
    • Now distinguishes between direct/indirect failures and skips
    • "Overall" terminology has been changed to "Total", and totals now appear below their sub parts (direct/indirect)
    • Destination cells with no data will now appear blank, rather than zero
    • There are now averages for both “success/failure/skip” numbers and percentages
    • All the participant count figures have been clustered together and the percentages have been clustered together for easier reading
    • Surveys Completed cell has been split into separate count and percentage cells
    • An extra row for task numbers has been added above the task names

Thanks everyone.

Andrew Mayfield

Blogs you might also enjoy