Notes from a fantastic talk by Senior Professor Rainer Bromme called “Fraudulent results and failed replications: The effect of flawed research on the public’s trust in science”. Bromme is a Senior Professor (like Professor Emeritus) in Psychology at the University of Münster and he was speaking at the COMPASS Seminar Series.

The University of Auckland brought Senior Professor Rainer Bromme over through the Distinguished Visitor award scheme. He is being hosted by Professor Gavin Brown and will be giving several other talks that are well worth checking out. The COMPASS Seminar series is also really worth including in your seminar roster and our nibbles are rather good. (I may be biased here.)

Liza’s one sentence summary

The fear that trust in experts is eroding has been rising since Brexit and Trump, and while Bromme shows many positive results in surveys on trust in scientists, he also demonstrates that the epistemic climate needs to be researched further and extended beyond just the idea of ‘trust’.

Kinds of problems which might impact on public trust

• Intentional deception
  • fabrication and falsification
    • autism and vaccines
  • plagiarism (two former German secretaries of state - defense and science/education)
• Failed replications
  • questionable research practice
  • weakness of theory and/or methods and/or data of the original study OR of the replication itself
  • issue under study is not consistent across time, culture, contexts

While there is no evidence that intentional deception is decreasing, there has been an increase in retractions in recent years. [http://www.nature.com/news/2011/111005/full/478026a.html]

The V&A Study in Sweden (2014)

Study investigated confidence in research and experts, also asked broken down by specific disciplines.

There was only a small decrease in trust in science. Trust in science and scientists is not related to media coverage of misconduct.

Among those who pay attention to media and have a higher education we only see a slight decrease in trust. Those who don’t pay attention to media are the ones for whom trust is decreasing. Education and political attitudes are very important.

Nielsen 2014 report for Aotearoa New Zealand

Report on public attitudes towards science & technology

Main takeaway that Rainer mentioned: High and positive appreciation of science, though trust not asked about directly.

Good summary from the Science Media Centre here.

Epistemic climate

Based on Swedish, NZ and other surveys presented, trust seems mostly okay.

Rainer asks: Does this mean everything is fine?

Answer: We should talk about the climate, not just the weather.

Trust = weather, but there is also the epistemic climate.

Example: Climate gate allegations (which weren’t true) impacted trust among those who were already leery.

Which questions can be answered by science, and which are to be left for personal values? Knowledge vs belief.

This reminded me of Yuval Noah Harari’s discussion of science and religion in Homo Deus. Check out the chapter called “The Odd Couple”. Harari begins by talking about how science is about factual statements and religions are about ethical judgements. But he continues “In truth, it is not always easy to separate ethical judgements from factual statements. Religions have the nagging tendency to turn factual statemetns into ethical judgements”. One example he gives is the factual statement (a statement that can be considered through the lens of science, has a truth value) “God wrote the Bible” frequently flows into the imperative “you ought to believe God wrote the Bible”. Further discussion of eternal souls, whether the American nation is sacred and Sam Harris’ views on ethics ensue. This book is a must read as is Sapiens.

Survey data on public trust in science measures a ‘weather parameter’. Further research must use are range of approaches to get the full picture.

Measuring Laypeople’s Trust in Experts in a Digital Age: The Muenster Epistemic Trustworthiness Inventory

Work Rainer has been involved in. Exploratory and confirmatory factor analysis looking at expertise, integrity and benevolence, key ideas in trust research.

Link to paper.

2017 Germany Survey on Trust in Science Wissenschaftsbarometer

People trust in science because of the expertise of scientists, but distrust arises from the belief that scientists are too heavily reliant on funders.

“Twelve per cent of respondents say that they do not trust science and research. Every second person expresses trust. The remaining 37 per cent are indecisive.”

Check out the full report.

Wissenschaftsbarometer is my new favourite word.

Effects of overhearing a controversy among scientists

Assumptions:

• Who says it matters - who discloses new evidence (Jensen 2008)
• What is said matters - which kind of evidence?

The studies below presented participants with either scientific or ethical criticism, offered either by the author or another expert, on a science blog.

1. Disclose your flaws! Admission positively affects the perceived trustworthiness of an expert science blogger
2. Evoking vigilance: Would you (dis)trust a scientist who discusses ethical implications of research in a science blog?

Scientific criticism: Integrity and benevolence are rated higher if the responsible science blogger criticises them self. Expertise is considered slightly lower whether they criticise them self or someone else does. Moral of the story: If criticism is due, do it yourself! You get the bonus of increased assessments of your integrity and benevolence.

A piece of advice I heard once that captures this well: F*ck up, fess up, fix up. I think this was via my stepdad, something his father said perhaps. I have had to repeat this mantra to myself more than once, and it helps.

Ethical aspects: No effect on expertise. Better integrity and benevolence if the blogger comments on their own work.

These judgements don’t require subject knowledge. It is an example of a heuristic. Pragmatic expectations are formed using conversation related cues.

I’ve been reading Thinking, Fast and Slow by Daniel Kahneman and I think there is a great connection here. We’re substituting an easier question for a harder question. The question really is “is this person an content expert?”, which requires you to have a level of mastery yourself, but “does this person behave/sound like an expert?” doesn’t require this and is an easier question for us to answer. If you haven’t read Thinking, Fast and Slow yet…why not? Do it.

An aside

A bit off topic, but this also made me think a bit of the Dr Fox Effect, the idea that “seductive” lecturers, even when not delivering good content, would receive higher ratings from students. Behaving in a way that seems like you are a good teacher might cause students to rate you as a good teacher, even if you’re not delivering the content that should be an important part of their assessment of you. The findings in this case were, unsurprisingly to me, that combining the most seductive lecturing styles with high content levels seemed to lead to higher achievement and students ratings.

Questions

‘Supply side’: scientists should be quick to criticise their own work when it is merited. What should be do about the ‘demand side’, the skills and knowledge of citizens to engage with and judge scientific research?

• The goal of science education should be to develop the “competent outsider” (See “Outside the pipeline: Reimagining science education for nonscientists” - paywall journal link, free article that summarises some ideas ), not the “incompetent insider”.
  • On current science ed practice: “it is not clear that these campaigns have enhanced people’s ability to function in a world where conflicting health advice clutters the Internet, research is filtered through political screens, and the media strips context from scientific claims.”
  • This article really supports the efforts and changes we’ve been making in Stats 10x to develop our students! And of course, we are both seductive and high content.
• Scientific literacy for citizenship is different from scientific literacy for being a scientist.
• How does science establish trustworthiness?
  • Gavin made a connection to IB ‘Theory of Knowledge’ unit
  • “Learning how to know”
• Who has paid for the study a statistically significant predictor of success of drug trial.

Links

• List of Rainer’s publications http://orcid.org/0000-0002-6123-7269/print
• Retraction watch blog http://www.retractionwatch.com

Book recommendations

• Thinking, Fast and Slow by Daniel Kahneman
• Homo Deus by Yuval Noah Harari
• Sapiens also by Yuval Noah Harari

Other talks Rainer is giving

• October 11: Philosophy seminar 4pm Wednesday 11 Oct, Arts 2, 18 Symonds St (207-501)
• October 19: Between certainty and doubt - What citizens (need to) know in order to cope with science based expert knowledge, public lecture
• October 25: Te Pūnaha Matatini Lecture: The ingredients of informed trust, public lecture at Auckland Museum