October 6, 2022

Measuring Disinformation Risk on TV News Programming: A Demonstration Study of Three U.S. Networks

Disinformation has been a growing area of study in light of the increasingly widespread harm it has caused. With ever-widening access to information comes increased risk of accessing mis- or disinformation. However, identifying and quantifying this risk is complex and challenging, and few viable frameworks exist. 

Effectively identifying disinformation requires a more nuanced analysis than simply evaluating whether a statement or assertion is true or false. The Global Disinformation Index (GDI) defines disinformation as adversarial narratives targeting at-risk individuals, groups or institutions that lead to real-world harm. This definition transcends many of the semantic arguments and provides a framework for understanding the broader universe of disinformation techniques. 

Based on this definition, GDI has developed a number of measurement approaches that evaluate disinformation risk on open-web news websites. However, as disinformation does not only exist online, GDI is now applying its measurement approach to other forms of media.

This paper presents an initial attempt to measure the level of risk of disinformation posed by mainstream cable television news programmes in the United States.

Methodology

GDI’s disinformation risk rating methodology measures the risk that a news or information source will disinform its readers or viewers, based on both the observed behaviour of the media outlet and the policies and practices in place to mitigate risk. 

This paper demonstrates that the GDI disinformation risk rating methodology can be successfully adapted to TV news programming. Adapting the methodology for TV has involved: 

  • Updating the indicators used to detect disinformation risk to be applicable to TV programming
  • Developing a method of sourcing raw data for review (i.e., anonymised, plain-text transcripts of TV content)
  • Training a research team to conduct content reviews and gather additional data on the journalistic operations of each TV programme
  • Calculating risk ratings that apply specifically to TV news

The study was conducted in mid-2021; as such, the study sample was defined based on network programming as of May 2021. This iteration of the project focused on cable news; future, more comprehensive projects might cover a greater range of networks. 

The study includes programmes from Fox News, MSNBC and CNN. Ratings data from 2021 show that these three networks were the most widely watched cable news networks in the U.S., and among the top five cable networks overall. Fox News had the greatest number of views by a significant margin, at an estimated 2.39 million, followed by MSNBC (1.54 million) and CNN (1.11 million).

Full details of the methodology, results and potential future research can be found within this report.