Brief

How can we design for young people to understand automated decisions?

Client

Projects By If

What we did

We conducted user research, an ideation workshop, low + high fidelity prototyping and concept testing.

In this project, we developed a fictional mental health chatbot, MoodJar.

Our goal behind Moodjar was to see:

• What young people would understand about automated decisions.

• How we could show the inner workings of a mental health chatbot, in ways that make automated decisions more understandable.

We used the insights generated from the user research to design a provotype which simulated conversation, focusing keenly on the user interface (UI) of the chatbot.

We wanted to see if it could assist in making automated decisions more explainable, and subsequently understandable.

Results

What we learned in this project speaks to the relationship young people want to have with systems that use AI.

The project is still in its early stages, and we need to keep testing these things, but there are some really interesting results.

The main one is that young people want certain boundaries respected if they are going to continue to positively engage with these technologies.

Young people believe and trust that AI is objective. They believe AI should not be making decisions which will have a big impact on their lives (though more decisions increasingly are), and they don’t want AI to try and develop a human like relationship with them.

With these lessons in mind, there is an opportunity to create new ways of designing that inform fairer and more explainable AI products and services.

The MoodJar prototypes were raw and utilitarian - they were useful for us to explore how to communicate automated decisions through different UI interactions. They were prototypes for testing, not to tell a story.

The IF team evolved the prototype into a number of isolated consent interaction patterns.

All the while keeping the political and practical thoughts about accountability, transparency and usability that underpinned the first versions of the prototype.