Generative AI as a research partner in design – some risks and benefits
Service design is a process that aims to develop customer-oriented services. To develop services, an understanding of the service's customers is often needed at the beginning, in which case different research methods are used to uncover these human experiences. When trying to expand your thinking by defining research questions, collecting data, and analysing, there are often steps that are repeated regardless of the case. Whether you choose qualitative or quantitative sampling as your method, you still repeat similar steps in each case, and in those areas of repetition it might be tempting to use generative AI.
Different artificial intelligence solutions can indeed be used for these repetitive steps. ChatGPT published by OpenAI has already gained a great reputation among information workers – and designers and researchers for that matter – in a rather short period of time. Despite often being referred to as artificial intelligence, ChatGPT is still not in fact intelligent at all in the true sense of the word. It is an generative language model that is optimised for dialogue. It’s not even optimised to search information nor provide factual insights. It literally guesses its answers – and it’s doing rather well these days. But not all generative AI are like ChatGPT, that is why I will talk about algorithms more generally than about AI.
In my work as a customer researcher and anthropologist, I have used e.g. ChatGPT in many different ways during the past year. In this article, I will share some useful tips as well as some critical observations.
Using AI in design research – who is doing the leg work?
In the research phase of service design, often also referred to as the Discovery phase, it is important to first define what question we are solving and only after that start to pay attention to the methods. If we approach the problem methodically, we are like a hammer to which everything looks like a nail. It is common to be method-oriented when the only research methods you are familiar with are questionnaires or interviews.
For this reason, it is important that a research expert is involved in the Discovery phase, since they have more experience on determining good research questions and planning methods that are the most useful ones to help the team uncover the insights they need.
Qualitative methods and artificial intelligence
Qualitative research aims to understand customers' experiences and feelings. It's a particularly useful approach when we have a lot of "why" questions. Why do our customers choose a competitor's product? Why do our customers choose us? Why do our customers act differently? What are our emerging markets? Quantitative methods are great when we want to understand scales. In this case, we may ask "how many of our customers" type questions.
I have used ChatGPT and Perplexity services, for example, in defining research questions, in which case I have a dialogue with the algorithm to understand: a) the customer's industry, b) the most important competitors, c) and the strategic competitive advantage that they have – or don’t have. Often these points of view are discussed with the customer anyway, but dialogues with a generative AI can offer understanding much better and faster than, for example, random googling can.
In addition, I can ask the generative AI to help me think of research questions for interviews, fieldwork and the like. It can help me with a list of the most important research questions when I provide it with a sufficient framework by contextual prompting:
perspective (e.g. anthropology, service and business design, marketing)
However, sometimes I face hallucinations, in other words false information. The information is also not always up-to-date, but is based on September 2021 (by the time I was writing this blog ChatGPT announced that GPT-4, the subscription based model, is basing its information until April 2023). Hallucinations are especially visible when the prompts I use are not precise enough. Sometimes it is a question of one key word missing, sometimes I need to change the entire prompt. This is where Perplexity's Co-pilot feature comes in handy, as it helps narrow down the prompt you're using to make the response as useful as possible (this feature requires a login, otherwise you can use Perplexity without an account). ChatGPT does not have such a feature (yet).
Generative AI algorithms have also crept into existing services, such as Teams, Google or, for example, the research tools you like to use. For example, survey research service Surveymonkey has developed automations based on artificial intelligence, which saves a lot of time in building surveys. It provides you automated answer types without manually selecting them, and this is a great example of those repetitive tasks that happen despite the case.
Another useful AI enabled feature is transcription services. It is possible to have notes of interviews or meetings automatically. The services can also offer content analysis using automatic thematic analysis, recognising patterns, emotions, themes and keywords, which speeds up the analysis phase remarkably – if it works correctly.
Quantitative methods and artificial intelligence
As mentioned above, qualitative methods are best when understanding scales. Let's imagine that in the research phase of design you want to understand how customers use a certain mobile application. Application data, such as usage times and the use of different functions, are used as your research data. Generative AI can help to analyse this data and identify, for example, use patterns in a more versatile way than traditional analytics. This data can help develop the application to better meet customer needs and expectations. Artificial intelligence can therefore learn to recognize certain patterns and connections from the collected data. This can help, for example, to predict future trends or behaviour patterns in both application development and other business development.
Visual analysis can recognize different elements from images, such as people, animals and objects – although there is often a large dispersion in different services. Visual analysis can be used, for example, when you want to understand a large amount of customer behaviour inside the store, in which case behavioural conclusions can be created from the data recorded by the cameras. This can help to understand customer behaviour and preferences in the same way as, for example, heatmaps have been used in website analytics to visualise user activity.
In quantitative research, the benefit of artificial intelligence is especially in its speed and efficiency. This can help, for example, to find connections between different variables and to perform statistical analyses with better quality and efficiency than with previously used technologies. For example, Tableau's Einstein Copilot helps to democratise the use of predictive data analytics via dialogue user interface, so you do not have to be an expert on analytics to dig deeper into the data.
Impact assessment is missing from the design process – we need to do better
Although artificial intelligence services can be useful in the research phase of design, it is important to remember that they do not yet replace human analysis or problem-solving ability. AI services can help collect and analyse data, but it's still important for humans to interpret and understand that data. In addition, it is important to note that artificial intelligence services are prone to errors and biases – but so are humans.
In order to avoid biases, the designer must understand the effects of AI biases, such as discriminatory structures and practices, so that he can develop non-discriminatory AI applications and evaluate their ethical use.
However, biases are not a result of technology – they are a result of people. In fact, very few design projects take into account the effect of biases on the final result.
When mitigating the effect of biases, we need to carefully determine which types of people we collaborate, co-create and interview with. Especially in public sector projects, the sample size should be larger than what is used in, for example, commercial projects.
Also, in design projects, time and money are not often allocated to the evaluation of the project's effects. In recent years, we have seen several examples, especially in the field of technology development, when the impact assessment has not been done and the developed service could have been used harmfully. Facebook's Cambridge Analytica disaster is a good example of this kind of vulnerability. Recent developments with generative AI are forcing us to rethink how we design and build products and services – and reimagine entirely new, better ways.
What do you outsource to technology?
One conclusion can be drawn from the examples above: artificial intelligence solutions can save time in the research phase of service design and they can positively affect the quality of the final results. Recent research results support this observation (see e.g. the MIT study).
But the most important question comes here: What are you actually trying to outsource to the machine?
With the help of artificial intelligence, I free up time at work for other things, but what are these other things actually?, Every time I've talked to people about this, I ask: "What would you do with that freed up time?" Usually people say, "I'd use it for learning." But would you really?
More free time usually leads to more work. It's like a pie eating contest where the prize is more pie.
If I use the free time to take on more and more client work, it can lead to burnout. Creative thought work also requires space to be and think. When we don't have time to think, stop, read or think, it leads to work that resembles factory work. Creative information work cannot be done in a factory-like process. So when you outsource tasks from the design process to technology, be mindful of how you use your time and energy.
Our Lead Researcher Anna Haverinen is a multi-faceted professional, excelling as a Design Researcher, Strategist, and Service Designer. Armed with a background in Anthropology and a Ph.D. in Digital Culture Studies, Anna specializes in unearthing profound human insights to drive transformative change. Notably, Anna has demonstrated her adeptness as both a team and research leader, successfully overseeing intricate projects on a large scale.
With a track record spanning 15 years, she has contributed her skills to diverse industries, with a particular focus on MVP and Discovery projects encompassing everything from branding to product design and encompassing foundational research to strategic exploration. In addition to the skills mentioned above, she’s an experienced public speaker, educator, and coach.