ATHENE: What was the aim of your study?

Katrin Hartwig: We wanted to investigate how social media users can best be supported in recognising disinformation or misinformation. It is not realistic to expect problematic content to be reliably and promptly removed. Our aim is therefore rather to help people learn to make informed decisions for themselves. In doing so, we examined both disinformation and misinformation – that is, false information that is spread unintentionally.

ATHENE: Why was a new study necessary?

Katrin Hartwig: Much of the research to date has focused on text-based content and platforms such as Twitter, now known as X,  or Facebook. By contrast, image and video content, for example, on TikTok, as well as voice messages have been less thoroughly investigated. Yet voice messages, in particular, play an important role in the spread of disinformation, for instance in Telegram channels. We therefore wanted to find out whether known countermeasures also work across different platforms and in various media formats.

ATHENE: How did you structure the study?

Katrin Hartwig: We compared various so-called interventions – that is, responses to disinformation – with one another. These included professional fact-checks, community notes and other approaches that provide users with specific guidance on what to look out for when assessing content. A total of 1,004 people took part in the study.

ATHENE: What sets your study apart from previous research?

Katrin Hartwig: Many previous studies assume that fact-checks or other interventions work flawlessly. However, that is unrealistic. Whether it’s people or AI systems, mistakes are inevitable. That is why we investigated how the effectiveness of the various approaches changes when they do not work perfectly.

ATHENE: And what were the findings? What did the study show when fact-checks and other warning labels made mistakes?

Katrin Hartwig: A key finding of our study was that the effectiveness and credibility of disinformation interventions depend very heavily on how reliably they work. Under ideal conditions, fact-checks and indicators in particular were shown to work well. Problematic content labelled with these warnings was rated as significantly less credible by the study participants. Community Notes also had an effect, but this was weaker.

However, as soon as we used flawed variants, so as soon as the interventions no longer worked reliably and occasionally produced incorrect assessments, their positive effect disappeared completely. Our key finding is therefore that the quality and accuracy of the intervention measures are crucial to their effectiveness.

ATHENE: Does the platform itself also play a role?

Katrin Hartwig: Surprisingly, our study found that the platform itself, whether it was TikTok, Telegram or X, had little influence on the effectiveness of the various interventions. These generally worked in a similar way across all platforms. So it was not the platform itself that was decisive, but whether the respective measure worked reliably or not. In practical terms, this means that established measures such as fact-checks or AI-supported warnings have so far only built trust when users have the impression that the labels are entirely accurate. However, we must find ways to deal with the inevitable errors.

ATHENE: Can the findings be put into practice straight away?

Katrin Hartwig: We have already presented our study findings at ACM CHI, and platforms certainly follow such research findings closely. But of course these companies also have their own interests, so we do not expect scientific findings to be implemented automatically.

ATHENE: Are there any alternative solutions that users could employ regardless of the platform?

Katrin Hartwig: In principle, yes. Browser extensions or smartphone apps, for example, are conceivable. We have already developed a simple demonstrator: users could forward a TikTok video to an app and receive an assessment of the content’s trustworthiness there.

ATHENE: What will your next research focus be?

Katrin Hartwig: So far, we’ve worked a lot with young people. In future, we’d like to focus more on older people, as they’re considered particularly vulnerable to disinformation, for example on Facebook. We see this as an important area of research for the coming years.

show all news