Validating personas using Chalkmark
Recently we used simple Chalkmark surveys to gather data to refine the Optimal Workshop personas.
While Chalkmark was never designed for this specific purpose, it worked well. We were able to test our ideas and develop new theories about how you all behave. Here is what we did.
- We developed theories about our personas.
- We segmented our database according to where users are in our customer journey map.
- We designed Chalkmark surveys and sent them to the relevant segments.
- We filtered results based on questionnaire responses to test for consistency.
Developing theories
After years of observing users (and being one ourselves), we thought we had a reasonable handle on the Optimal Workshop customer. But therein lies the advantage and the problem – self referential design! In order to get some objective facts, we developed two theories on what defines the customer interaction with us.
Firstly, we thought that the level of remote research experience you have impacts on how you interact with us. Secondly, we also thought that number of hours you spend on UX work changes how you would use our tools. These seemingly obvious theories have implications on where we spend our marketing and development efforts – but we needed data to validate our theories.
Segmenting our database
We analysed our database according to where each person was in the customer journey. We found three broad segments we were interested in and we set out to test our theories with each segment. (The segments were “the aware”, “the informed” and “the experienced”).
Chalkmark surveys
We designed a Chalkmark survey for each segment to see if our understanding matched with peoples’ perceptions of themselves. The surveys only had two tasks – one where the participant chooses which user profile they identify with the most, and secondly which step in the customer journey they were mostly in.
At the end of the Chalkmark survey, we also presented a short questionnaire that asked respondents to tell us how much time they spend on user experience, and how much remote user research they’ve done. All participants were invited by email, and each segment was given its own Chalkmark survey.
If you’re interested, you can take one of the persona surveys here.
Analysis of results
The results were pretty immediate – and thankfully validated our theories about you.
It was particularly useful to be able to filter by questionnaire response. For instance, for participants who spent less than 25% of their time on UX work, we could see which user profile they identified with most. Likewise for people who felt they could write a book on remote user research, we could see which step in the customer journey they were.
Most of you (unsurprisingly) spent most of your time on UX work and have a moderate level of remote research experience. We also learned a bunch of stuff about how much support each segment needs to progress to the next stage in the customer journey.
So what?
This simple method worked well for us as it was fast and leveraged existing knowledge. It also lends itself to being iterative – which prevents our personas from being a static creation. Of course, we wouldn’t rely on it exclusively to inform our personas but it was a helpful supplement to our research.
Chalkmark has always intrigued us for it’s versatility. Though it’s designed to be a “first click” testing tool, it clearly has applications in other areas also.
