How do you measure the impact of digital information on user behaviour? Evaluating how digital adds to charitable aims is more complex than conversion tracking. Kate Roberts says the trick is getting creative not reductive.
Every organisation has goals for its website, but some are more easily measurable than others. For most businesses, important website metrics include sales, customer acquisition and customer retention. They can use analytics to measure sign-ups, enquiries, ad clickthroughs and sales transactions, and even place a ‘value’ on each page according to how much its traffic contributes to revenue.
Charities, too, have clear conversion metrics around donations, newsletter subscriptions, campaign actions. For most non-profits, the main goal is to help people and to change lives and raising funds is a core part of enabling that but not the only metric that matters. The value of the website also lies in the information it provides. The user’s needs are met when they read and understand, not necessarily when they click through or submit.
A conversion is an action that you want users to take, but for charities, this action will often be taken offline. Shelter wants people to avoid eviction. Breast Cancer Care wants cancer sufferers to make informed choices about treatment. Carers UK wants carers to claim the benefits to which they’re entitled.
In Google Analytics, you measure conversions with goals. On an ecommerce website, a major goal would be reaching the confirmation page of the checkout process, signifying a sales conversion.
In ‘Practical Web Analytics’, Michael Beasley gives the following four-step guidance for choosing a goal:
- Define the overall purpose of the organisation. This is not “make money”. For example, eBay wants to empower people to sell goods online; the University of Michigan wants to educate people and conduct research.
- Consider how the website fits in with this purpose. The University of Michigan’s website doesn’t directly educate students, but it does attract new students, and supports current students by providing vital information.
- Ask what the organisation wants users to do on the website. The answers should be clearly defined tasks with clearly defined end points.
- Define the specific, measurable behaviour that shows that a user took that action. Usually it will be reaching a particular page or clicking a link.
Goals and conversions don’t have to be tied to sales (or of course, for charities, donations). For informational sites, there are several possible goals that could indicate a user’s interest in what the site offers:
- Downloading a report
- Making an online enquiry
- Posting on a forum
- Subscribing to a newsletter
- Watching a video
- Using a free tool like a calculator
- Sharing content on social media
However, for most non-profits, these are flawed and partial measures of success. Charities aim to help people by providing online content, but the effect is only evident offline. The conversion they want is a meaningful change in people’s lives. How do you measure this?
Analytics can only ever be part of the answer. To discover whether they are helping the people they want to help, charities must engage in other forms of research.
Exit surveys are a good way to quantify the user experience. Avinash Kaushik wrote that the “three greatest survey questions ever” are:
- What was the purpose of your visit to our website today?
- Were you able to complete your task today?
- If you were not able to complete your task today, why not?
These questions will certainly get some good data, and there’s something to be said for keeping a survey short and sweet – research has found that each additional page of questions increases the dropout rate by up to 5%.
But these questions measure user satisfaction, not outcomes. How can a survey evaluate the propensity of the user to take action based on what he or she has read or seen online?
Asking users to predict the future is notoriously unreliable. Many a survey has asked “Would you use this product?” and received an overwhelming ‘Yes’, only for the product to have no take-up after launch. Research participants may be eager to please, or simply find it impossible to imagine a hypothetical scenario.
But attitudinal questions can indicate the user’s propensity to take action. For example, asking how strongly they agree or disagree with statements like “This website helped me to understand my rights as a tenant” or “This website made me feel more able to approach my consultant about treatment”.
You could also ask people to leave their email address if they would be happy to participate in further research, and contact them in a few months to ask if they have taken the desired action.
Qualitative research can give further insight into the website’s effect on users. Diary studies are a longitudinal research method that records the user’s behaviour over time. Tools like Ethnio can intercept users coming to your website and ask if they’d be prepared for a researcher to telephone them for an interview shortly after their site visit.
Analytics are great, but for many non-profits they can only ever provide a small and blurry part of the picture. To really understand whether you’re changing people’s lives for the better, you have to get creative, not reductive.