The challenge:

Versus Arthritis Chatbot

Versus Arthritis (previously Arthritis Research UK) took the bold and forward-looking step of using Artificial Intelligence to reach more people who require information about arthritis conditions outside of their telephone helpline hours. Powered by IBM Watson, Versus Arthritis developed an Artificial Intelligence chatbot, the Arthritis Virtual Assistant in 2017.

As the Arthritis Virtual Assistant came to the end of its beta stage in 2018, Versus Arthritis wanted to understand how the initial user and interface experience was working for people using the chatbot. In particular they wanted to understand what people loved or were being frustrated by during their conversations with the chatbot.

Our approach:

Versus Arthritis engaged We Are MC2 to carry out thorough research of the user experience of the Arthritis Virtual Assistant, with the results and analysis being used to inform the next phase of the project’s growth.

During our discovery phase, we synthesised research including the results from four rounds of usability testing, and broader academic research into AI and virtual assistants. We also interviewed contacts at other charities that ran similar projects, as well as analysing the anonymised logs of real conversations that people had already had with the Arthritis Virtual Assistant.

As the Arthritis Virtual Assistant was still in beta, and the organisation was undergoing a re-branding exercise we placed a temporary intercept across the site, asking people to click through if they were interested in using a virtual assistant. This allowed us to assess interest in the product and to find out if there were any specific profiles of user that the Arthritis Virtual Assistant appealed to.

Finally, we carried out in-depth interviews with people who had used the Arthritis Virtual Assistant, to understand their needs for health information and advice, and their use of technology. We then observed them using the Arthritis Virtual Assistant in real-time to ask questions and identify if there were emotional responses to the chatbot, as well as any usability issues.

We observed a genuine propensity (even among the elderly) to engage with this kind of tool – *if* it answered their questions. In our report, we identified the most common user needs, and produced models for how people talk to the Arthritis Virtual Assistant and what they expected when it talked to them.

Our report enabled the Versus Arthritis team to adapt its content and user interface plans, to better serve the needs of the chatbots users. It also identified other areas the project could and should explore (like smart speaker integration and deeper personalisation) based on the user responses and our research.

The client’s view:

We launched the Arthritis Virtual Assistant in beta in summer 2017. Since then it has become a key part of our organisational strategy to reach people with arthritis at scale and ensure that no one is living in musculoskeletal pain without access to support and advice. Our aim was to develop a long term strategy and digital roadmap for the virtual assistant that aligned to our users’ needs. We had a lot of unanswered questions about who our current users were, or could be in the future and we felt this insight was crucial to moving forward with the virtual assistant.

Here’s some of what we wanted to know:
Who are our current users and how does this relate to our organisational audience segmentation?
What are their attitudes to AI and technology?
How could the AVA best support them?

We carried out a tender with a number of excellent UX agencies and We Are MC2 rose to the top of the pile. They approached the project with incredible professionalism and quickly applied their deep knowledge and experience of user needs, UX and research skills to every aspect of the project. Not only were they a dream to work with they also produced a truly excellent piece of research which delivered everything we needed to plan ahead with confidence. And the research they produced lives on; we return to it time and time again as our knowledge of the virtual assistant and our users grows and develops.

We wholeheartedly recommend We Are MC2 and look forward to working with them in the future.

What our clients say

  • Alison bought a huge breadth of strategic voluntary sector digital experience. She provided a framework and rigour to ensure we challenged ourselves to deliver a programme that was truly transformational rather than just a step change. She saw connections where we saw discrete projects defined by internal structures.

    Brendan Paddy, Director of Communications, DEC

  • Alison brought a huge breadth of strategic voluntary sector digital experience. She provided a framework and rigour to ensure we challenged ourselves to deliver a programme that was truly transformational rather than just a step change.

    Brendan Paddy, Director of Communications, DEC

  • We Are MC2 is a dream consultancy for third sector clients. My experience has been great.

    Coral Stanion-Nazeri, Digital Communications Manager, Action For Children

  • I wanted to work collaboratively with a strategist who has the right mix of experience, who has the ability to apply creative thinking to complex challenges and the know how to drive this thinking through to action. My experience of working with Alison and We Are MC2 is that they tick all these boxes and then some.

    Ben Butler, Head of Digital, Drinkaware

  • Quite simply, the best digital consultant I have worked with.

    Anne Clifford, Marketing Director, The Children's Society