Wikipedia:Wikipedia Signpost/2023-05-22/Recent research

Recent research

Create or curate, cooperate or compete? Game theory for Wikipedia editors

By Tilman Bayer

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Cooperative creators "lose ownership" of their Wikipedia contributions, but the community's NPOV governance should still be "kept limited"

"A game-theoretic analysis of Wikipedia’s peer production: The interplay between community’s governance and contributors’ interactions", published earlier this month in PLoS ONE, investigates what it calls fundamental but still unresolved questions about "the way in which governance shapes individual-level interactions" in peer production.

Specifically, the authors build a complex game theory model to answer the following two research questions about English Wikipedia, and validate its predictions empirically by examining the revision histories of 864 articles (up to 2012):

"RQ1: What is the optimal strategy for a contributor who is attempting to balance the costs and benefits of peer production? Namely, how many sentences within an article should be owned by a contributor that is characterized by a certain cooperative/competitive orientation and a particular creator/curator activity profile?

RQ2: How does the community governance mechanism, particularly the attempt to ensure a NPOV, affect the dynamics underlying Wikipedia’s co-production process?"

These concepts are operationalized as follows (among other variables used to construct the the quantitative model):

• "Owned" here refers to having authored text that persists in a Wikipedia article, surviving subsequent revisions (as opposed to ownership in the sense of contributors being entitled to a form of control over the content - a notion that, as the authors discuss, is explicitly discouraged by the community). An editor's utility - i.e. the benefit that they derive from contributing to Wikipedia - is modeled using both their individual "fractional ownership" and the communal "benefit derived from the co-production of high-quality articles".
• An editor's "creator/curator" orientation is quantitatively represented by a metric that assumes that "Creators are characterized by large-size edits [..]; curators’ edits are smaller".
• The "cooperative/competitive orientation" was quantified "based on [contributors'] role in the community (those closer to the community’s core are assumed to be more cooperative)." Based on previous research, editors were "organized in four strata, reflecting contributors’ commitment and involvement within the community: unregistered members, registered members, privileged members (holding a special privilege), and core members (i.e. administrators)." The authors theorize that "the greater one’s rights and responsibilities within the Wikipedia community, the more one is considered to have a cooperative orientation. Specifically, we assigned the values of 0.1, 0.4, 0.6, and 0.9 to the different strata (e.g., an administrator is assigned w1 = 0.1, reflecting the least competitive and most cooperative orientation)."
• Lastly, an editor's cost is quantified as the sum of
  • a. "the effort expended in producing content and editing an article", where the authors "assume that changing an existing sentence requires less effort than originating a new sentence, such that the effort expended by a contributor is a linear function the contributor’s position in the creator-curator continuum" (obviously a simplification, considering that an editor's individual contribution may well vary between adding and changing content).
  • b. "the effort associated with participation in coordination and administrative work, such as editing the articles’ Talk Pages". Again, the authors model this solely based on the editor's overall "cooperative/competitive orientation", reasoning that "This cost element is more applicable for the cooperative contributors".
  • c. "the effort of complying with Wikipedia’s rules and policies, in particular, neutrality-enforcing policies. This effort is intended to regulate only the self-interest activities (i.e., attempting to 'own' article portions) and thus is more applicable to competitive contributors." It is modeled as being proportional to the number of "owned" sentences, multiplied by the editor's competitiveness factor and a variable representing the community's overall "level of neutrality enforcement".

Schema of the model's first level

The game theoretic model consists of two levels:

"[...] the first level models the interactions between individual contributors who seek both cooperative and competitive goals and the second level models governance of co-production as a Stackelberg (leader-follower) game between contributors and the communal neutrality-enforcing mechanisms."

A calculation of the model's Nash equilibrium (basically, a state where no individual "player" can improve their utility by making a unilateral change to their strategy) yields several rather complicated formulae (Theorems 1-4), from which the authors derive various overall conclusions, e.g. that

"[...] the contributor’s characteristics, or more specifically, the ratio between the contributor’s position on the creator-curator continuum and the contributor’s cooperative/competitive orientation is the factor that determines who ends up owning content. When this ratio is smaller than the group’s average, the contributor maintains ownership over portions of the article. Namely, under the governance mechanisms, the fractional content that is eventually owned by a contributor is higher for curators (i.e., with a typical small-size edit per sentence) with a competitive orientation (i.e., peripheral community members). In essence, creators with a cooperative orientation lose ownership of the article. This result was corroborated through [the] empirical analysis.

The authors explain that this means that

"[...]only those with a competitive orientation who choose to act as curators making small edits end up owning significant portion of the content. One might expect that the creators who contribute more content (and in the process exert more effort) would end up owning much of an article’s contents. In contrast, the results of our game-theoretic analysis implies that when competing over content ownership in the presence of Wikipedia’s governance to ensure neutrality, and when controlling for one’s cooperative/competitive-orientation, the creators of content who make on average large contributions would eventually not own any content."

The second "key result" from the game-theoretic analysis is that

"[...] excessive governance should be curtailed, by identifying and maintaining a permissible upper limit, beyond which it discourages contributors from making contributions to an article, bringing the co-production process to a halt. Furthermore, our empirical analysis suggests that a low level of governance is optimal for ensuring neutrality while maintaining articles’ comprehensiveness.

Based on this, the authors recommend that

[T]he community’s efforts to govern content creation and ensure neutrality, although essential for maintaining a balanced position, should be carefully monitored and kept limited. The reason is that when the 'tax' imposed on contributors in terms of complying with NPOV norms, policies and procedures is too high, it outweighs the benefits associated with content ownership, such that contributors stop competing for ownership (and in effect, co-production is stalled)."

See also previous coverage of related research by one of the authors (Ofer Arazy from the University of Haifa)

Briefly

• See the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.
• ...

Other recent publications

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

"Defying easy categorization: Wikipedia as primary, secondary and tertiary resource"

From the abstract:

"This article sets out to explore the different categories of source that Wikipedia could be defined as (primary, secondary or tertiary) alongside the varied ways in which Wikipedia is used, which defy easy categorization, exemplified by a broad-ranging literature review and focusing on the English language Wikipedia. It concludes that Wikipedia cannot easily be categorized in any information category but is defined instead by the ways it is used and interpreted by its users."

(We note with delight the mention of the Twitter feed associated with this research newsletter: "Serendipitous discoveries of relevant research were also made via the WikiResearch Twitter account @WikiResearch, the ‘Wiki-research-l’ mailing list and the Wikimedia Research biannual reports.")

"Conflict dynamics in collaborative knowledge production. A study of network gatekeeping on Wikipedia"

From the abstract and highlights:

"We study gatekeeping on Wikipedia by analyzing networks of deletions among editors. Factors considered to explain the emergence of these networks are editors’ information production capability, their range of activity, their communicative ties, their geographical location, and their past interactions. [...]

• Editors’ past interactions affect gatekeeping conflicts on Wikipedia.[...]
• In- and Outgroups based on editors’ real-world attributes influence gatekeeping."

"Understanding Open Collaboration of Wikipedia Good Articles with Factor Analysis"

From the abstract:

"This research aims at understanding the open collaboration involved in producing Wikipedia Good Articles (GA). [...] We propose an approach that first employs factor analysis to identify editing abilities [of contributors] and then uses these editing abilities scores to distinguish editors. Then, we generate sequence of editors participating in the work process to analyse the patterns of collaboration. Without loss of generality, we use GA of three Wikipedia categories covering two general topics and a science topic to demonstrate our approach. The result shows that we can successfully generate editor abilities and identify different types of editors. Then we observe the sequence of different editor involved in the creation process. For the three GA categories examined, we found that [...] highly scored content-shaping ability editors [tend to be] involved in the later stage of the collaboration process."

References

Supplementary references and notes:

This page is a draft for the next issue of the Signpost. Below is some helpful code that will help you write and format a Signpost draft. If it's blank, you can fill out a template by copy-pasting this in and pressing 'publish changes': {{subst:Wikipedia:Wikipedia Signpost/Templates/Story-preload}}

Images and Galleries

Sidebar images To put an image in your article, use the following template (link): [[File:|center|300px|alt=TKTK]] O frabjous day. {{Wikipedia:Wikipedia Signpost/Templates/Filler image-v2 |image = |size = 300px |alt = TKTK |caption = |fullwidth = no }} This will create the file on the right. Keep the 300px in most cases. If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes. Inline images Placing {{Wikipedia:Wikipedia Signpost/Templates/Inline image |image = |size = 300px |align = center |alt = Placeholder alt text |caption = CAPTION }} (link) will instead create an inline image like below [[File:|300px|center|alt=Placeholder alt text]] CAPTION Galleries To create a gallery, use the following <gallery mode = packed | heights = 200px> |Caption for second image </gallery> to create

Quotes

Framed quotes “ Lorem ipsum dolor sit amet... ” — AUTHOR, SOURCE To insert a framed quote like the one on the right, use this template (link): {{Wikipedia:Wikipedia Signpost/Templates/Filler quote-v2 |1 = |author = |source = |fullwidth = }} If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes. Pull quotes To insert a pull quote like “ Lorem ipsum dolor sit amet... ” use this template (link): {{Wikipedia:Wikipedia Signpost/Templates/Quote |1 = |source = }} Long quotes To insert a long inline quote like The goose is on the loose! The geese are on the lease! — User:Oscar Wilde — Quotations Notes from the Underpoop use this template (link): {{Wikipedia:Wikipedia Signpost/Templates/block quote | text = | by = | source = | ts = | oldid = }}

Side frames

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. A caption Side frames help put content in sidebar vignettes. For instance, this one (link): {{Wikipedia:Wikipedia Signpost/Templates/Filler frame-v2 |1 = Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. |caption = A caption |fullwidth = no }} gives the frame on the right. This is useful when you want to insert non-standard images, quotes, graphs, and the like. Example − Graph/Charts Further information: Category:Graph, chart and plot templates A caption For example, to insert the {{Graph:Chart}} generated by {{Graph:Chart |width=250|height=100|type=line |x=1,2,3,4,5,6,7,8|y=10,12,6,14,2,10,7,9 }} in a frame, simple put the graph code in |1= {{Wikipedia:Wikipedia Signpost/Templates/Filler frame-v2 |1= {{Graph:Chart |width=250|height=100|type=line |x=1,2,3,4,5,6,7,8|y=10,12,6,14,2,10,7,9 }} |caption=A caption |fullwidth=no }} to get the framed Graph:Chart on the right. If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes.

Two-column vs full width styles

If you keep the 'normal' preloaded draft and work from there, you will be using the two-column style. This is perfectly fine in most cases and you don't need to do anything. However, every time you have a |fullwidth=no and change it to |fullwidth=yes (or vice-versa), the article will take that style from that point onwards (|fullwidth=yes → full width, |fullwidth=no → two-column). By default, omitting |fullwidth= is the same as putting |fullwidth=no and the article will have two columns after that. Again, this is perfectly fine in most cases, and you don't need to do anything. However, you can also fine-tune which style is used at which point in an article. To switch from two-column → full width style midway in an article, insert {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-end-v2}} {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-start-v2|fullwidth=yes}} where you want the switch to happen. To switch from full width → two-column style midway in an article, insert {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-end-v2}} {{Wikipedia:Wikipedia Signpost/Templates/Signpost-block-start-v2|fullwidth=no}} where you want the switch to happen.

Article series

To add a series of 'related articles' your article, use the following code Related articles Visual Editor Five, ten, and fifteen years ago 1 January 2023 VisualEditor, endowment, science, and news in brief 5 August 2015 HTTPS-only rollout completed, proposal to enable VisualEditor for new accounts 17 June 2015 VisualEditor and MediaWiki updates 29 April 2015 Security issue fixed; VisualEditor changes 4 February 2015 More articles Draft namespace, VisualEditor meetings 25 December 2013 Sockpuppet investigations, VisualEditor, Wikidata's birthday, and more 6 November 2013 Last call for Wiki Loves Monuments; Community–WMF tension over VisualEditor 25 September 2013 Making Wikipedia more accessible 4 September 2013 Gallery improvements launch on Wikipedia 28 August 2013 The VisualEditor Beta and the path to change 31 July 2013 Defining consensus; VisualEditor default state; expert and layperson terms in article titles 31 July 2013 VisualEditor in midst of game-changing deployment series 3 July 2013 VisualEditor will "change world history" 12 June 2013 VisualEditor, elections, bots, and more 12 June 2013 The Visual Editor: Where are we now, and where are we headed? 25 March 2013 Visual Editor "on schedule" for July rollout 18 March 2013 Looking ahead to 2013 7 January 2013 The new Visual Editor gets a bit more visual 10 December 2012 Second Visual Editor prototype launches 25 June 2012 Visual editor demo launched, hailed as "most important change to our user experience ... ever"; but elsewhere over-hasty deployments criticised 19 December 2011 Full steam ahead on Visual Editor, the avoidance of lock-breaking and 1.18b1 release 7 November 2011 {{Signpost series |type = sidebar-v2 |tag = VisualEditor |seriestitle = Visual Editor |fullwidth = no }} or {{Signpost series |type = sidebar-v2 |tag = VisualEditor |seriestitle = Visual Editor |fullwidth = yes }} will create the sidebar on the right. If writing a 'full width' article, change |fullwidth=no to |fullwidth=yes. A partial list of valid |tag= parameters can be found at here and will decide the list of articles presented. |seriestitle= is the title that will appear below 'Related articles' in the box. Alternatively, you can use {{Signpost series |type = inline |tag = VisualEditor |tag_name = visual editor |tag_pretext = the }} at the end of an article to create For more Signpost coverage on the visual editor see our visual editor series. If you think a topic would make a good series, but you don't see a tag for it, or that all the articles in a series seem 'old', ask for help at the WT:NEWSROOM. Many more tags exist, but they haven't been documented yet.

Links and such

This department (Recent research): Next issue · Next next issue · Search for previous Arbitration report: Arbitration · Arbcom · Archives · Cases · Enforcements Discussion report: News and notes: In the media: Google News · TechMEME By the way, the template that you're reading right now is {{Editnotices/Group/Wikipedia:Wikipedia Signpost/Next issue}} (edit). A list of the preload templates for Signpost articles can be found here.

← Previous "Recent research"

In this issue

22 May 2023

News and notes

In the media

Arbitration report

Recent research

Featured content

Traffic report

WikiProject report

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try .

• those closer to the community’s core are assumed to be more cooperative – hmm... – Joe (talk) 09:08, 22 May 2023 (UTC)

• I would like to read the Gatekeeping article. Does anyone have a preprint? Sandizer (talk) 15:59, 22 May 2023 (UTC)

  @Sandizer: It's accessible via The Wikipedia Library. – Rhain ☔ (he/him) 23:37, 22 May 2023 (UTC)

• I feel like the game theory paper is interesting but it also feels like it could be empirically tested to see if its findings hold up in reality because like Joe Roe I'm not sure their statrting assumptions are correct. Best, Barkeep49 (talk) 16:35, 22 May 2023 (UTC)

  Well, the authors actually did extensive empirical testing, on a dataset spanning more than a decade of edits (as I already briefly mentioned in the review but should probably have expanded upon). From the abstract:

  We validate the model’s prediction through an empirical analysis, by studying the interactions of 219,811 distinct contributors that co-produced 864 Wikipedia articles over a decade. The analysis and empirical results suggest that the factor that determines who ends up owning content is the ratio between one’s cooperative/competitive orientation (estimated based on whether a core or peripheral community member) and the contributor’s creator/curator activity profile (proxied through average edit size per sentence). Namely, under the governance mechanisms, the fractional content that is eventually owned by a contributor is higher for curators that have a competitive orientation.

  But I guess what you and Joe are concerned about is rather what is often called construct validity - i.e., do these metrics defined in the paper really capture what we would commonly perceive as an editor's cooperativeness vs. competitiveness, say? And that's a valid question here. (Of course all models are wrong and it's virtually always possible to pick on some shortcoming where this kind of modeling doesn't fully reflect reality. One would need to ask whether and how much it affects the overall conclusions.)

  Relatedly, while the paper's data availability statement claims that "All relevant data are within the paper and its Supporting information files," the dataset provided there appears to be very incomplete. For example, it does not seem to contain the information necessary to calculate the creator/curator metric, and only contains data on 16,383 users instead of the aforementioned 219,811 (i.e. consists of 16,384 rows including the header, a suspiciously round number). It is also anonymized, meaning that one can't evaluate construct validity by inspecting the ratings for some concrete editors in the dataset manually.

  Regards, HaeB (talk) 18:16, 23 May 2023 (UTC)

• I wonder how good the definition of "owned" is. If I for example say "that's a fair enough copy edit which doesn't change meaning, does the other person "own" the sentence that says exactly the same thing as I added. Talpedia 16:18, 6 June 2023 (UTC)

Get the latest headlines on your user page — just add {{Signpost-subscription}}.

Home

About

Archives

Newsroom

Subscribe

Suggestions