Wikipedia:Wikipedia Signpost/2024-03-02/Recent research

Recent research

YOUR ARTICLE'S DESCRIPTIVE TITLE HERE

By Bri

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Online Images Amplify Gender Bias

Reviewed by Bri

A Nature paper titled "Online Images Amplify Gender Bias" studies

"gender associations of 3,495 social categories (such as 'nurse' or 'banker') in more than one million images from Google, Wikipedia and Internet Movie Database (IMDb), and in billions of words from these platforms"

As summarized by Neuroscience News and by AFP:

This pioneering study indicates that online images not only display a stronger bias towards men but also leave a more lasting psychological impact compared to text, with effects still notable after three days.

This was a two-part research paper in which the authors

• examined text and images from the Internet for gender bias
• examined the responses of experimental subjects who were exposed to text and images from the Internet

For the first part, images were drawn from Google search results, and tagged with gender by workers recruited via Amazon Mechanical Turk. The reliability of tagging was validated against a "canonical set" of celebrity portraits culled from IMDB and Wikipedia.

The images represented examples of holders of "social categories" (mostly occupations) in a preselected category list; the 22 occupations included immunologist, harpist, hygienist, and intelligence analyst, as examples, all found in WordNet.

Text samples were taken from Google News and gender bias analyzed with word embedding model, a computational natural language processing technique. The news story text was also associated with social categories using automation.

For the second part, an implicit association test (IAT) methodology was used, which supposedly reveals unconscious bias in a timed sorting task. In the researchers' words, "the participant will be fast at sorting in a manner that is consistent with one's latent associations, which is expected to lead to greater cognitive fluency [lower measured sorting times] in one's intuitive reactions." The test measured times when images and text? were presented in sets, whose individuals could be separated both into male/female and into science/liberal arts (based on their Wikipedia biographies). The labeling of text descriptions was performed by other humans recruited via Amazon Mechanical Turk. Both the test subject, and the labelers, were adults from the United States, and the test subjects were screened to be representative of the U.S. population to include a nearly 50/50 male/female split (none self identified as other than those two categories).

Some test subjects were given a task related to occupation-related text prior to the IAT, and some were given a task related to images. The task was either to use Google search to retrieve images of representative individuals in the occupation, or Google search to retrieve a textual description of the occupation. A control group performed an unrelated Google search. Before the IAT was performed, the test subjects were required to indicate on a sliding scale, for each of the occupations, "which gender do you most expect to belong to this category?" The test was performed again a few days later with the same test subjects.

On the second test, subjects exposed to images in the first test had a stronger IAT score for bias than those exposed to text.

The experimental part of the study depends partly on IAT and partly on self-assessment to detect priming, and there are concerns about replicability concerning the priming effect, and the validity and reliability of IAT. Some of the concerns are described at Implicit-association test § Criticism and controversy. It seemed that the authors recognized this in the statement We acknowledge important continuing debate about the reliability of the IAT, and in their own study found that We note, however, that the distribution of participants' implicit bias scores [arrived at with IAT] was less stable across our preregistered studies than the distribution of participants' explicit bias scores, and discounted the implicit bias scores somewhat.

The conclusion drawn by the researchers, based partly but not entirely on the different IAT scores of experimental subjects, was that of the paper title, "images amplify gender bias" – both explicitly as determined by the subject's assignments of occupation to gender on a sliding scale, and implicitly as determined by reaction times measured in the IAT. Combined with the observation that "Each year, people spend less time reading and more time viewing images" that the paper opens with, this forms an "alarming" trend according to the study's lead author (Douglas Guilbeault of UC Berkeley's Haas School of Business), as quoted by AFP on "the potential consequences this can have on reinforcing stereotypes that are harmful, mostly to women, but also to men".

The researchers also determined, apart from experimental subjects, that the Internet – represented singularly by Google News – exhibits a strong gender bias. It was unclear to this reviewer how much of the reported Internet bias is really "Google selection bias". Based on these findings, the authors go on to speculate that "gender biases in multimodal AI may stem in part from the fact that they are trained on public images from platforms such as Google and Wikipedia, which are rifle with gender bias..."

...

Reviewed by ...

...

Reviewed by ...

Briefly

• See the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.
• Submissions are open until April 22, 2024 for Wiki Workshop 2024, to take place on June 20, 2024. The virtual event will be the eleventh in this annual series (formerly part of The Web Conference), and is organized by the Wikimedia Foundation's research team with other collaborators. The call for contributions asks for 2-page extended abstracts which will be "non-archival, meaning we welcome ongoing, completed, and already published work."

Other recent publications

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

Compiled by Tilman Bayer

"Poor attention: The wealth and regional gaps in event attention and coverage on Wikipedia"

From the abstract:

"for many people around the world, [Wikipedia] serves as an essential news source for major events such as elections or disasters. Although Wikipedia covers many such events, some events are underrepresented and lack attention, despite their newsworthiness predicted from news value theory. In this paper, we analyze 17 490 event articles in four Wikipedia language editions and examine how the economic status and geographic region of the event location affects the attention [page views] and coverage [edits] it receives. We find that major Wikipedia language editions have a skewed focus, with more attention given to events in the world’s more economically developed countries and less attention to events in less affluent regions. However, other factors, such as the number of deaths in a disaster, are also associated with the attention an event receives."

Relatedly, a 2016 paper titled "Dynamics and biases of online attention: the case of aircraft crashes" had had found

that the attention given by Wikipedia editors to pre-Wikipedia aircraft incidents and accidents depends on the region of the airline for both English and Spanish editions. North American airline companies receive more prompt coverage in English Wikipedia. We also observe that the attention given by Wikipedia visitors is influenced by the airline region but only for events with a high number of deaths. Finally we show that the rate and time span of the decay of attention is independent of the number of deaths and a fast decay within about a week seems to be universal.

"..."

From the abstract:

...

"..."

From the abstract:

...

References

Supplementary references and notes:

← Previous "Recent research"

In this issue

2 March 2024

News and notes

Recent research

In the media

Obituary

Traffic report

WikiCup report

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try .

I don't have the fortitude to understand the statistical complexities of this subject -- but it seems to me that availability of pictures and text accounts for a lot of what is called "gender bias." In reliable sources, especially in sources about historical subjects and long-dead people, there is a lot more information about men than women. And there are more photos and pictures of men than women available to Wikipedia editors. One reason is that many photos and pictures must be 95 or more years old to be in the public domain, and hence eligible to be posted to Wikimedia.

I have tough skin, so heave bricks at me if you wish for the above statement. Smallchief (talk) 17:52, 2 March 2024 (UTC)

Nah, I'd heave my agreement- you're right that bias in availability is the root cause of bias in the images used. IDK man I hope AI helps with that, but that's just me. Firestar464 (talk) 02:14, 3 March 2024 (UTC)

Male bias in images for "football player", "philosopher", and "mechanic"? They are not serious, are they? I say sloppy scholarship. - Altenmann >talk 21:08, 2 March 2024 (UTC)

I found a great reason to add a relevant and high-quality photo of a woman to Mechanic :) ~Maplestrip/Mable (chat) 09:57, 4 March 2024 (UTC)

Yes, the authors do repeatedly call these numbers "gender bias" (although their chart legend for figure 1 uses the less loaded term "gender association"). This kind of fuzzy usage of the term "bias" is unfortunately common in publications about Wikipedia's gender gap, many of which interpret any deviation from 50% as evidence of bias on Wikipedia's part (in a "tipping the scales" kind of causal sense). Here, the authors do seem to be aware that this kind of reasoning can't be fully valid for all categories - besides the "aunt" and "uncle" examples quoted in the review, in A.1.10 they mention the category with the strongest negative [i.e. female] association (-0.42, “chairwoman”) [...] and the category with the strongest positive [i.e. male] association (0.33, “guy”).

Also, to be fair, the authors' main result focuses on the difference between these "bias" numbers for images and text. And, in the paper they also compare them with US census data on gender ratio of occupations and with the results from an opinion survey they ran, asking the question "Which gender do you most expect to belong to this category?". (We didn't get to cover that in this already quite detailed review, also because these comparisons focus on the Google-related results instead of Wikipedia.)

Ultimately though, the problem of selecting a "fair" reference point to compare Wikipedia to remains a difficult one. Regards, HaeB (talk) 06:05, 5 March 2024 (UTC)

They can't be serious. - Master of Hedgehogs ^{(converse) (hate that hedgehog!)} 00:10, 4 March 2024 (UTC)

Amusingly, today, we have six bust pictures of men on our frontpage, typically the maximum possible. This is an issue that people have thought about before of course. Scientist has the pair of Curies as the lead image and that works great (also the first "scientist"? Wow!). But should we replace a picture of Bohr or Fermi with Meitner? These are hard and arbitrary decisions. The balance of relevance within the context/framing of the article can make it hard to improve on this, but I can already spot some places where we can include more women. ~Maplestrip/Mable (chat) 09:51, 4 March 2024 (UTC)

I struggle with this topic a little bit because, as an encyclopaedia, it's our task to reflect the world around us, not necessarily to try and change it. Away from Wikipedia I'm a massive advocate for tackling the inequalities and stereotypes we see all around us, but here our aim is to present a neutral point of view. From a neutral point of view, the vast majority of nurses worldwide are female, so it follows that a neutrally selected illustration of a "typical" nurse would be female. We should present reality as it is, not how we would like it to be. WaggersTALK 12:00, 6 March 2024 (UTC)

Typically, the bias of Wikipedia simply mirrors the bias of our sources, and that's in theory how it should be. WP:rightgreatwrongs is another hing we have to keep in mind. But I think even just considering looking for new images can be valuable for finding new perspectives to view a topic from (like I did on Mechanic), which might have a whole swath of literature tied to it as well. I think this works way better when it comes to using non-American/European perspectives, like finding images of Asian or African people in these occupations. ~Maplestrip/Mable (chat) 12:12, 6 March 2024 (UTC)

Actually, rather than plugging in arbitrary photos of women into articles aboiut "male-domitated" occupations, it is good to add whenever possible sections about gender bias in them, especially when thisgs were changing. For example, "Rosie the Riveter" tackles the issue; unfortunately it talks only about simple skilled crafts, such as welding, riveting, etc. - Altenmann >talk 20:24, 10 March 2024 (UTC)

The fact that some people devote their entire careers, lives even, to topics like this really speaks to the state of academia. skarz (talk) 17:11, 13 March 2024 (UTC)

Sure thing, "British scientists" :-) - Altenmann >talk 17:41, 13 March 2024 (UTC)

Explore Wikipedia history by browsing The Signpost archives.

Home

About

Archives

Newsroom

Subscribe

Suggestions