Helping developers develop empathy at Atlassian

How can we as designers help our developers to help drive a great user experience?

You may have heard of Atlassian or used our products before — Jira, Confluence, Bitbucket or Stride, to name a few.

The Atlassian ecosystem is a collection of apps sold on our Marketplace, that end users can install onto our products that provide some extra functionality or integrate our products with other software tools. Similar to the way you can download Spotify from the App Store to access Spotify on your iPhone, the Marketplace lets you install InVision to Jira. That’s how I explain my job to my parents, anyway (sorry Mum).

We have thousands of apps on our Marketplace, for all sorts of use cases. These apps are mostly developed by third-party companies (not Atlassian!). There are entire companies that exist and make lots of money solely off developing and selling Atlassian apps. We refer to these companies as our “developers” or our “developer community”. Many of these companies are small, engineering-driven startups with little to no design resources. It’s therefore in Atlassian’s best interest to scale our own design practices across our developer community, starting with user research.

Sharing design best practices with our ecosystem

Nearly every quarter, we host events called App Weeks that provide developers with the opportunity to work directly with Atlassian. It’s also an opportunity for Atlassian to hear how developers are using our platform and get their feedback on our APIs. At one of these events in late 2016, I surveyed the developer community and asked how often they speak to their users.

From that survey, I found that none of our developers had a regular cadence for user research and the majority had never even spoken to a user before. These aren’t just lone developers in big companies, but entire startups, meaning in some cases no one in the company had ever spoken to a user. At Atlassian, we run user research every other week. Atlassian of course has many more resources; however I wanted to delve into this problem further and find out why exactly our developers aren’t speaking to users as much as they should.

These were the results: Why don’t Atlassian ecosystem developers talk to users?

Most people said they didn’t have time or didn’t think it was a priority, which was not particularly surprising to me. Some said they didn’t know how while others said they didn’t think it was worth it or had never considered it as a thing to do. The ecosystem design team came up with a mission to ensure that all developers who impact user experience are properly investing in the user.

At the first 2017 App Week, I introduced “User Research Day”. The purpose of User Research Day is to expose developers to the practice of user research and sell them on the value, so they can pick it up as part of their regular work. Since the kick off of User Research Day, I’ve run it four times at four different events — refining and improving it each time.

Kicking off User Research Day

So, what’s involved in this user research day?

Step 1: Pre-event prep

I gather a group of Atlassians (our term for people who work at Atlassian) to act as “test participants” for our developers. I use Atlassians for this for a few reasons. Firstly, it’s easier to organize than sourcing research participants from outside the company. More importantly, this is the first time most of these developers will have ever run a user test or customer interview, and it can be quite an intimidating thing to do. Using Atlassians makes the research process a bit more comfortable for developers while they’re learning the techniques.

I ask Atlassians from all over the company to participate. I specifically try to get people who aren’t in the ecosystem team so that I can eliminate the bias of knowing the technical background for how the app works. I ask people from HR, finance and legal to get business users, people in their first month at Atlassian to get new users, and people from product teams to get technical and experienced users. I put all of this in a spreadsheet and ask developers to pick a few test participants that represent their user base.

Step 2: The low-down

On the first day of App Week, I run a ‘How to do user research 101’ crash course on the basics of running user testing and customer interview sessions. I didn’t do this the first time I ran User Research Day and I got feedback from my Atlassian test participants that developers asked leading questions, justified decisions they had made, and a few other research faux pas. Bringing in this crash course helped to give them some guidance and teach them best practices.

The third time I ran User Research Day I decided to share a template we use internally at Atlassian to plan and structure our user testing sessions. After bringing this in I got feedback from test participants that they were impressed with the professional standard of how the sessions were run.

Step 3: Review testing plans

In the lead up to User Research Day, I offer to review developers’ test plans and provide feedback, or act as a practice participant.

Step 4: Wednesday of App Week is User Research Day!

On the day of the session, I run all the logistics. I organize people, find a time that both parties can meet, book a space for them, and organize the incentive (“Atlassians don’t really need an incentive, as they’re more than happy to help. However we’re trying to simulate a real user research session, plus it’s a nice thank you).

I do all the leg work because I’m trying to lower the barrier of entry into user research for these developers as much as possible. All they have to do is plan the questions they’re going to ask and show up.

Step 5: Follow up

After the sessions, I follow up with developers. I ask what they learned, what insights they identified and needed to interpret, and what actions to take next.

Step 6: Tips for running research in the wild

Finally, I share some advice for ways in which developers can continue user research back in their own organizations. We talk about different research methods and when to use them, how often they should do research, how much research to do and what sort of things are appropriate to test.

Developers are particularly interested in knowing how to find customers in their own cities — something that’s challenging even for us at Atlassian.

Feedback and results

At our most recent App Week, we stumbled across something that unintentionally helped our developers with their own user research. My colleagues and I were running user testing sessions at App Week to collect feedback on some of our own work.

Afterwards, a few developers told me that participating in our research and seeing how we conducted the sessions helped them to understand how to run their own. I thought this was particularly interesting because most of these developers had participated in our research before; however, now that they were thinking about running their own sessions, they were picking up on the techniques I had told them about, and seeing them performed in a real scenario.

“It was probably the best thing we did. It helped so so so so so so so very much.” – a two person startup, reflecting on their first experience speaking to customers.

It would be hypocritical of me to not ask for feedback on user research day from our developers. We get a ton of constructive feedback, and we’re constantly improving User Research Day and the design activities we run at App Weeks.

We also get some comments that are not so much feedback but rather expressions of appreciation: “It was probably the best thing we did. It helped so so so so so so so very much.” – a two person startup, reflecting on their first experience speaking to customers. For this company, seeing users struggle to understand their app experience was eye-opening.

Today, more of our developers are picking up user research as part of their regular work practice:

Do Atlassian developers talk to users?

This graph shows how the attitude towards user research has changed just over the last eight months. As you can see, we went from having the majority in the “Nope, never spoken to a user” category to now having some people frequently speaking to users.

What’s next?

As I mentioned, we’re always seeking to improve User Research Day and our design support to our developers. As per feedback from our last event, my focus for the next day will be around iteration. Developers are now asking for two user research days during App Week, one at the beginning of the week and one at the end. That way they can test their app, make changes, then test those changes. To me, two rounds of user testing in one week sounds pretty ambitious.

So my plan for the next event is to run User Research Day as we have been and then help developers organize another round of testing in their own cities and offices a week or two later. This is how the Atlassian ecosystem design team is influencing our developer community on the best practices of user research. Check out my previous blog for Optimal Workshop to learn more about other ways we’re influencing design and product best practices.