Wikipedia:Wikipedia Signpost/2023-02-04/Recent research

Recent research

Wikipedia's "moderate yet systematic" liberal citation bias

By Tilman Bayer

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

English Wikipedia's news citations found to have "moderate yet systematic" liberal bias

"Distribution of Wikipedia’s news media citation political polarization scores using Kernel Density Estimates (KDE). Negative: liberal; positive: conservative." (Figure 2 from the paper)

A preprint titled "Polarization and reliability of news sources in Wikipedia" finds

[...] a moderate yet systematic liberal polarization in the [English Wikipedia's] selection of news media sources. We also show that this effect is not mitigated by controlling for news media factual reliability."

The study is based on a dataset of 30 million citations extracted in 2020, which the second author and others have already examined from different angles in other research publications (cf. our previous coverage: "6.7% of Wikipedia articles cite at least one academic journal article with DOI", "How Wikipedia keeps up with COVID-19 research", "A Map of Science in Wikipedia").

As with research examining other kinds of bias (like gender, language or geography), studying political bias involves the non-trivial problem of defining a "neutral" baseline against which to compare Wikipedia's content. For example, in a series of earlier papers that (among other results) found Wikipedia to be "more slanted towards Democratic views" than Britannica, although its "bias was moving from left to right", Greenstein and Zhu used the United States Congressional Record as a kind of gold standard of unbiased language. (Of course, this opened them up to the question whether the spectrum of opinions present among US federal lawmakers is an appropriate baseline for an international encyclopedia, even if their analysis was focused on articles related to US politics.) A 2017 paper studied both political and gender bias by comparing Wikipedia's coverage of topics to that of "political periodicals geared toward either liberal or conservative ideologies" (e.g. Mother Jones vs. National Review), and women's vs. men's magazines, respectively (see our earlier coverage: "English Wikipedia biased against conservative and female topics, at least when compared to US magazines").

The present study relies on a different source that has since become available:

To estimate the political polarization of Wikipedia citations, we use the Media Bias Monitor. This system collects demographic data about the Facebook followers of 20,448 distinct news media outlets [...]. These data include political leanings, gender, age, income, ethnicity and national identity. For political leanings, the Facebook Audience API provides five levels: Very Conservative, Conservative, Moderate, Liberal, Very Liberal. To measure the political leaning of an outlet, MBM firstly finds the fraction of readers having different political leanings, and then multiply the fraction for each category with the following values: very liberal (–2), liberal (–1), moderate (0), conservative (1), and very conservative (2). The sum of such scores provides a single polarization score for the outlet, ranging between –2 and 2, where a negative score indicates that a media outlet is read more by a liberal leaning audience, while a positive score indicates a conservative leaning audience. In the original paper, MBM is compared to alternative approaches used to infer the political leanings of news media outlets, finding that this method highly correlates with most alternatives."

Matching domain names between MBM and the "Wikipedia Citations" dataset, the study finds that

"The average Wikipedia citation polarization score (red line) is -0.51 (median -0.52) [on the aforementioned MBM scale from -2 (very liberal) to 2 (very conservative)], therefore leaning towards liberal. The bulk of citations also falls between the range -1 and 0."

"Distribution of Wikipedia citation political polarization scores for the top 10 WikiProjects" (figure from the paper)

Breaking down polarization ratings by ORES article topic areas, "we cannot see differences among macro topics". This "general trend" was also found for the top 10 (sub-)topic areas and the top 10 Wikiprojects, although with "minor shifts [...]. For example, the topic sports has a higher conservative-leaning fraction of citations, all the while maintaining a liberal-leaning skew. The WikiProjects Politics and India are more liberal-leaning than the average, instead. Taken together, these results confirm that the overall trend towards liberal political polarization is not specific to some areas of Wikipedia, but seems to be widespread across topics and WikiProjects."

"Distribution of Wikipedia’s news media citation reliability scores" according to Media Bias/Fact Check (figure 1 from the paper)

Motivating their second research question, the authors "speculate that editors may introduce political polarization in their sources in order to prioritise reliable ones" (which might remind one of Stephen Colbert's dictum "Reality has a well-known liberal bias"). To test this hypothesis, they use the reliability ratings of Media Bias/Fact Check (but not that site's bias ratings). They note in passing that "that, while there are only 1467 citations rated as 'VERY LOW' [reliability], there remains a sizable fraction of citations to low or mixed reliability outlets" on English Wikipedia, as of 2020. (It might have been interesting to conduct the same analysis with the English Wikipedia's own reliability ratings that the community has compiled for numerous news sources at WP:RSP - where, ironically, "Media Bias/Fact Check" is itself currently rated as "generally unreliable, as it is self published", somewhat in contrast to the present paper and the peer-reviewed publication that it cites in justification of using MBFC.)

However, in a linear regression analysis (which also takes article topic and WikiProjects into account), the authors "cannot see a clear pattern emerge. While high reliability shows a liberal skew, very high reliability shows a conservative skew in turn. Mixed sources tend to be more liberal, while low and very low reliability ones tend to be more conservative." Overall, they conclude that "the case for a possible association between low reliability and conservative news outlets disappear[s]" in the end.

Briefly

• See the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.
• The Wikimedia Foundation's Research team is soliciting nominations for the "Wikimedia Foundation Research Award of the Year" 2022, to be submitted until February 6.

Other recent publications

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

"Political representation bias in DBpedia and Wikidata as a challenge for downstream processing"

From the abstract:

"Diversity Searcher is a tool originally developed to help analyse diversity in news media texts [...] We compare two data sources that Diversity Searcher has worked with – [the Wikipedia-based] DBpedia and Wikidata – with respect to their ontological coverage and diversity, and describe implications for the resulting analyses of text corpora. We describe a case study of the relative over- or underrepresentation of Belgian political parties between 1990 and 2020 in the English-language DBpedia, the Dutch-language DBpedia, and Wikidata [...]. In particular, we came across a staggering overrepresentation of the political right in the English-language DBpedia."

From the "Method" section:

"As a null hypothesis, a knowledge source represents a political constellation in an unbiased way if the relative number of politicians from a given party who are represented as an entity in a knowledge source [...] equals the relative number of this party in a relevant real-life context. [... We] consider “having a Wikipedia page” (etc.) as an important contributor to public visibility of a person and their party. [...] The baseline is then – relatively – easy to define: the shares of the vote or the number of seats of parties Y at times T in a given political body. We started by concentrating on the national parliament, the Chamber of People’s Representatives (Kamer van volksvertegenwoordigers, henceforth KVV) and used the number of seats at the beginning of a legislature. We also looked at the regional (Flemish) parliaments (Vlaams parlement, VP) [...]"

From the "Results and interpretation" section:

"These results not only confirm our first informal observation of over-representation of rightwing parties (especially the N-VA) in the English-language DBpedia, with a trend growing over time. (During these years, the N-VA’s share of the popular vote increased, but the DBpedia growth clearly exceeds the baseline growth.) Different biases seem to occur in the Dutch-language DBpedia: although on the whole comparatively similar to the baseline, this ontology seems to over-represent the main centrist party (CD&V). Wikidata, in contrast, gives a rather accurate picture of party shares in the national parliament. The French-language Walloon parties are (understandably, given the language focus) under-represented in the Dutch-language DBpedia. Both the overrepresentation of rightist and centrist parties in media coverage have been identified in earlier international research [...]"

"Assuming Good Faith Online"

In this legal essay, US legal scholar Eric Goldman (whom some Wikipedians might recall for his – later retracted – 2005 prediction of Wikipedia's demise due to volunteer burnout) contrasts Wikipedia's "Assume Good Faith" principle with current attempts by Internet regulators to rein in on user-generated content websites and Section 230 (see also this issue's "In the media").

Simulation of article disputes finds that "it is more important not to have intolerant editors than to have very tolerant ones"

From the abstract:

"[...] we focus on how the editors' attitudes, namely being broad-minded or stubborn, affect the consensus-building process in a model of Wikipedia. We further investigate how banning editors affects the speed with which conflicts or debates can be resolved. For the analysis, we use an agent-based opinion model developed to simulate different aspects of Wikipedia. We show that, in most cases, banning agents from editing an article slows down the consensus-building process, and increases the system’s relaxation time. We show further, and counterintuitively, that with large groups of 'extremists' who hold other than the central opinion, consensus can be reached faster and the article will be less biased."

From the "Conclusion" section:

"[..] for the consensus [to be achieved] it is more important not to have intolerant editors than to have very tolerant ones.

Our results indicate that consensus is reached extremely slowly if the bias of the article can be changed only by a small amount. To resolve the conflict faster, one must either increase the change of bias in one edit or the ratio of extremists. In general, the latter cannot be controlled deliberately, but the former can be influenced.

In Wikipedia, there is already a method aimed at resolving disputes of that sort. The solution is to move the disputed questions into a new section (or page) where they can be discussed freely. The new trend to move disputed parts of the article into the Criticism or Controversy sections [which is actually discouraged in a widely cited community essay] is a good way to handle this problem. Assigning [sensitive] arguments and opinions to a small section of the article that is much easier to modify makes the full article less disputed. Thus, tolerance towards the main article increases [...]"

See also our review of a related earlier paper involving one of the authors: "More newbies mean more conflict, but extreme tolerance can still achieve eternal peace".

"The Role of Local Content in Wikipedia: A Study on Reader and Editor Engagement"

From the abstract:

"About a quarter of each Wikipedia language edition is dedicated to representing 'local content', i.e. the corresponding cultural context —geographical places, historical events, political figures, among others—. To investigate the relevance of such content for users and communities, we present an analysis of reader and editor engagement in terms of pageviews and edits. The results, consistent across fifteen diverse language editions, show that these articles are more engaging for readers and especially for editors. The highest proportion of edits on cultural context content is generated by anonymous users, and also administrators engage proportionally more than plain registered editors [...]"

(cf. by some of the same authors: "The Wikipedia Diversity Observatory: A Project to Identify and Bridge Content Gaps in Wikipedia")

This paper is part of a 2021 monograph published on occasion of Wikipedia's 20th anniversary ("Wikipedia, veinte años de conocimiento libre"), which comprises various other research papers, most of which are in Spanish with an English abstract.

"Discussing the Past: The Production of Historical Knowledge on Wikipedia"

From the abstract:

"This study explores how historical knowledge is produced on Wikipedia. The project is based on multiple methodologies ranging from qualitative analysis of Wikipedia pages related to history, survey with Wikipedia editors, to quantitative analysis of participatory practices within the Wikipedia community. The main argument is that Wikipedia allows people to discuss the past, express their opinions and emotions about history and its significance in the present and the future through the portal of “talk” [pages] that Wikipedia provides [...].

This dissertation includes detailed examinations of the history of discussions at Talk:Atomic bombings of Hiroshima and Nagasaki, Talk:Vietnam War and Talk:September 11 attacks.

"Producing Historical Knowledge on Wikipedia"

This is an earlier paper by the dissertation's author. From the "Conclusion" section:

"Wikipedia’s capability of producing historical narratives, its self-critical character through the talk pages, and its open character are significant tools that should not be underestimated. The popularity of Wikipedia and, particularly, the popularity of the historical pages that are visited daily by a lot of people have to be studied and not be neglected as a kind of not “real history.” Wikipedia cannot change radically the historical scholarship but can bring the historian closer to the society."

InternetArchiveBot found to be over-eager in declaring links as "permanently dead" but late in archiving them

From the abstract:

"Broken external references on Wikipedia which lack archived copies are marked as 'permanently dead'. But, we find this term to be a misnomer, as many previously dysfunctional links work fine today. For links which do not work, it is rarely the case that no archived copies exist. Instead, we find that the current policy for determining which archived copies for an URL are not erroneous is too conservative, and many URLs are archived for the first time only after they no longer work."

References

Supplementary references and notes:

This page is a draft for the next issue of the Signpost. Below is some helpful code that will help you write and format a Signpost draft. If it's blank, you can fill out a template by copy-pasting this in and pressing 'publish changes': {{subst:Wikipedia:Wikipedia Signpost/Templates/Story-preload}}

Images and Galleries

Sidebar images To put an image in your article, use the following template (link): [[File:|center|300px|alt=TKTK]] O frabjous day. {{Wikipedia:Wikipedia Signpost/Templates/Filler image-v2 |image = |size = 300px |alt = TKTK |caption = |fullwidth = no }} This will create the file on the right. Keep the 300px in most cases. If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes. Inline images Placing {{Wikipedia:Wikipedia Signpost/Templates/Inline image |image = |size = 300px |align = center |alt = Placeholder alt text |caption = CAPTION }} (link) will instead create an inline image like below [[File:|300px|center|alt=Placeholder alt text]] CAPTION Galleries To create a gallery, use the following <gallery mode = packed | heights = 200px> |Caption for second image </gallery> to create

Quotes

Framed quotes “ Lorem ipsum dolor sit amet... ” — AUTHOR, SOURCE To insert a framed quote like the one on the right, use this template (link): {{Wikipedia:Wikipedia Signpost/Templates/Filler quote-v2 |1 = |author = |source = |fullwidth = }} If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes. Pull quotes To insert a pull quote like “ Lorem ipsum dolor sit amet... ” use this template (link): {{Wikipedia:Wikipedia Signpost/Templates/Quote |1 = |source = }} Long quotes To insert a long inline quote like The goose is on the loose! The geese are on the lease! — User:Oscar Wilde — Quotations Notes from the Underpoop use this template (link): {{Wikipedia:Wikipedia Signpost/Templates/block quote | text = | by = | source = | ts = | oldid = }}

Side frames

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. A caption Side frames help put content in sidebar vignettes. For instance, this one (link): {{Wikipedia:Wikipedia Signpost/Templates/Filler frame-v2 |1 = Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. |caption = A caption |fullwidth = no }} gives the frame on the right. This is useful when you want to insert non-standard images, quotes, graphs, and the like. Example − Graph/Charts Further information: Category:Graph, chart and plot templates A caption For example, to insert the {{Graph:Chart}} generated by {{Graph:Chart |width=250|height=100|type=line |x=1,2,3,4,5,6,7,8|y=10,12,6,14,2,10,7,9 }} in a frame, simple put the graph code in |1= {{Wikipedia:Wikipedia Signpost/Templates/Filler frame-v2 |1= {{Graph:Chart |width=250|height=100|type=line |x=1,2,3,4,5,6,7,8|y=10,12,6,14,2,10,7,9 }} |caption=A caption |fullwidth=no }} to get the framed Graph:Chart on the right. If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes.

Two-column vs full width styles

If you keep the 'normal' preloaded draft and work from there, you will be using the two-column style. This is perfectly fine in most cases and you don't need to do anything. However, every time you have a |fullwidth=no and change it to |fullwidth=yes (or vice-versa), the article will take that style from that point onwards (|fullwidth=yes → full width, |fullwidth=no → two-column). By default, omitting |fullwidth= is the same as putting |fullwidth=no and the article will have two columns after that. Again, this is perfectly fine in most cases, and you don't need to do anything. However, you can also fine-tune which style is used at which point in an article. To switch from two-column → full width style midway in an article, insert {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-end-v2}} {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-start-v2|fullwidth=yes}} where you want the switch to happen. To switch from full width → two-column style midway in an article, insert {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-end-v2}} {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-start-v2|fullwidth=no}} where you want the switch to happen.

Article series

To add a series of 'related articles' your article, use the following code Related articles Visual Editor Five, ten, and fifteen years ago 1 January 2023 VisualEditor, endowment, science, and news in brief 5 August 2015 HTTPS-only rollout completed, proposal to enable VisualEditor for new accounts 17 June 2015 VisualEditor and MediaWiki updates 29 April 2015 Security issue fixed; VisualEditor changes 4 February 2015 More articles Draft namespace, VisualEditor meetings 25 December 2013 Sockpuppet investigations, VisualEditor, Wikidata's birthday, and more 6 November 2013 Last call for Wiki Loves Monuments; Community–WMF tension over VisualEditor 25 September 2013 Making Wikipedia more accessible 4 September 2013 Gallery improvements launch on Wikipedia 28 August 2013 The VisualEditor Beta and the path to change 31 July 2013 Defining consensus; VisualEditor default state; expert and layperson terms in article titles 31 July 2013 VisualEditor in midst of game-changing deployment series 3 July 2013 VisualEditor will "change world history" 12 June 2013 VisualEditor, elections, bots, and more 12 June 2013 The Visual Editor: Where are we now, and where are we headed? 25 March 2013 Visual Editor "on schedule" for July rollout 18 March 2013 Looking ahead to 2013 7 January 2013 The new Visual Editor gets a bit more visual 10 December 2012 Second Visual Editor prototype launches 25 June 2012 Visual editor demo launched, hailed as "most important change to our user experience ... ever"; but elsewhere over-hasty deployments criticised 19 December 2011 Full steam ahead on Visual Editor, the avoidance of lock-breaking and 1.18b1 release 7 November 2011 {{Signpost series |type = sidebar-v2 |tag = VisualEditor |seriestitle = Visual Editor |fullwidth = no }} or {{Signpost series |type = sidebar-v2 |tag = VisualEditor |seriestitle = Visual Editor |fullwidth = yes }} will create the sidebar on the right. If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes. A partial list of valid |tag= parameters can be found at here and will decide the list of articles presented. |seriestitle= is the title that will appear below 'Related articles' in the box. Alternatively, you can use {{Signpost series |type = inline |tag = VisualEditor |tag_name = visual editor |tag_pretext = the }} at the end of an article to create For more Signpost coverage on the visual editor see our visual editor series. If you think a topic would make a good series, but you don't see a tag for it, or that all the articles in a series seem 'old', ask for help at the WT:NEWSROOM. Many more tags exist, but they haven't been documented yet.

Links and such

This department (Recent research): Next issue · Next next issue · Search for previous Arbitration report: Arbitration · Arbcom · Archives · Cases · Enforcements Discussion report: News and notes: In the media: Google News · TechMEME By the way, the template that you're reading right now is {{Editnotices/Group/Wikipedia:Wikipedia Signpost/Next issue}} (edit). A list of the preload templates for Signpost articles can be found here.

← Previous "Recent research"

In this issue

4 February 2023

From the editor

News and notes

Section 230

Disinformation report

Special report

In the media

Op-Ed

Opinion

Recent research

WikiProject report

Tips and tricks

Featured content

Traffic report

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try .

Ech. So they compared an international encyclopedia to American publications and claim it's bad we don't perfectly align with them? Most of the Anglosphere is left of America. Adam Cuerden ^(talk)_{Has about 8.2% of all FPs. Currently celebrating his 600^th FP!} 16:23, 7 February 2023 (UTC)

• Yeah a bit silly of a story - of course stuff will be more left-leaning, that's most of the academic world LegalSmeagolian (talk) 21:10, 7 February 2023 (UTC)

Most of the Anglosphere is the United States of America. Somers-all-the-time (talk) 04:36, 8 February 2023 (UTC)

2/3rds of the Anglosphere is a majority, but not so much of one that Wikipedia would be expected to match American biases. Frankly, the premise of the study is that American political biases are some objective standard that Wikipedia should be trying to emulate. Since that's not and has never been Wikipedia's goal, and given the weirdness of the dataset (Facebook user data and Media Bias Monitor?), it's questionable. Oddly enough, though, Media Bias Monitor itself rates Wikipedia as "least biased". Adam Cuerden ^(talk)_{Has about 8.2% of all FPs. Currently celebrating his 600^th FP!} 18:50, 9 February 2023 (UTC)

That InternetArchiveBot blurb is quite concerning. Any attempts to fix this issue? DFlhb (talk) 09:23, 9 February 2023 (UTC)

Keep up with The Signpost on Twitter, Facebook or Mastodon.

Home

About

Archives

Newsroom

Subscribe

Suggestions