Research recruitment: Difference between revisions

From Meta, a Wikimedia project coordination wiki
Content deleted Content added
mNo edit summary
No edit summary
Line 3: Line 3:
<!-- {{disputedpolicy}} -->
<!-- {{disputedpolicy}} -->


[[Academic studies about Wikipedia|Scholarly research of Wikipedia]] is useful for understanding the encyclopedia's content, readers, editors, history, current state, and future. These results also yield important knowledge applicable to other open content communities. In addition to driving scholarly knowledge of such systems, this work can also give results that can improve Wikipedia itself. Much valuable research cannot be done without Wikipedia community members who volunteer to participate in studies. This policy exists to further three goals:
[[en:Academic studies about Wikipedia|Scholarly research of Wikipedia]] is useful for understanding the encyclopedia's content, readers, editors, history, current state, and future. These results also yield important knowledge applicable to other open content communities. In addition to driving scholarly knowledge of such systems, this work can also give results that can improve Wikipedia itself. Much valuable research cannot be done without Wikipedia community members who volunteer to participate in studies. This policy exists to further three goals:
# To allow important research of Wikipedia and its users to take place.
# To allow important research of Wikipedia and its users to take place.
# To ensure that that individual community members' preferences on whether and how frequently they are invited to participate in research studies are respected.
# To ensure that that individual community members' preferences on whether and how frequently they are invited to participate in research studies are respected.
# To ensure that research studies are respectful of participating community members and community norms.
# To ensure that research studies are respectful of participating community members and community norms.


This policy describes how researchers can recruit members via talk page postings and creates the [[Wikipedia:Subject Recruitment Approvals Group|Subject Recruitment Approvals Group]] (SRAG), a public discussion group, to control who will be allowed to make those postings. SRAG receives applications from researchers to recruit editors to participate in research and manages the public consensus process which considers them. If the recruitment is approved, a bot owned by SRAG ([[User:SubjectRecruitmentBot|SubjectRecruitmentBot]]) contacts selected community members and invites them to participate. Each community member controls whether and how often the bot can contact him or her.
This policy describes how researchers can recruit members via talk page postings and creates the [[Subject Recruitment Approvals Group]] (SRAG), a public discussion group, to control who will be allowed to make those postings. SRAG receives applications from researchers to recruit editors to participate in research and manages the public consensus process which considers them. If the recruitment is approved, SOMETHING contacts selected community members and invites them to participate. Each community member controls whether and how often the bot can contact him or her (see [[en:User:SubjectRecruitmentBot]]).


==Scope==
==Scope==
This policy pertains to the activities of researchers within Wikimedia projects.
This policy pertains to the recruitment of Wikipedia users as subjects for research. Examples of these activities include [[statistical survey|surveys]], [[interview]]s, and [[experiment]]s.


==Researchers==
==Researchers==
For a more complete summary, see: [[en:User:EpochFail/What_are_these_researchers_doing_in_my_Wikipedia]]
This section gives a brief overview of who researchers are and why they are interested in studying Wikipedia and its editors.


Expanding articles is not the only way to contribute to Wikipedia. There is room for many different types of contribution: copy edits, content edits, template construction, vandal fighting, bot management, etc. Through the use of the scientific method, researchers contribute to Wikipedia by extended understanding about the community and bringing state-of-the-art technologies to editors.
===Who are they===
There are a wide variety of backgrounds from which people approach studying Wikipedia.
* Academics: the students, professors, and staff of colleges and universities
* Industry researchers: the staff of private companies
* Independents: individuals without an affiliation with an interest in studying Wikipedia
They are here to perform scientific analysis of Wikipedia and its users and, most often, intend to publish the results of their work in [[academic publishing|academic publications]].

===What do they do===
In the past, research in Wikipedia has built an understanding of how Wikipedia works,<ref>{{cite conference
| url = http://www.viktoria.se/altchi/submissions/submission_edchi_1.pdf
| title = Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie
| first = Aniket
| last = Kittur
| coauthors = et al
| year = 2007
| conference = alt.CHI at Conference on Human Factors in Computing Systems
| conferenceurl = http://www.chi2007.org/submit/altchi.php
| accessdate = 2009-12-29
| quote =
}}</ref> how editors interact with each other,<ref>{{cite conference
| url = http://kittur.org/files/Kittur_2007_Wikipedia_CHI.pdf
| title = He Says, She Says: Conflict and Coordination in Wikipedia
| first = Aniket
| last = Kittur
| coauthors = et al
| year = 2007
| conference = Conference on Human Factors in Computing Systems
| conferenceurl = http://www.chi2007.org
| volume = 1
| booktitle = Proceedings of the SIGCHI conference on Human factors in computing systems
| publisher = ACM Press
| pages = 453&ndash;462
| isbn = 9781595935939
| doi = 10.1145/1240624.1240698
| accessdate = 2009-12-29
| quote =
}}</ref> what work is discarded and why,<ref>{{cite conference
| url = http://www.grouplens.org/node/416
| title = A jury of your peers: Quality experience and Ownership in Wikipedia
| first = Aaron
| last = Halfaker
| coauthors = et al
| year = 2009
| conference = International Symposium on Wikis
| conferenceurl = http://www.wikisym.org/ws2009/
| booktitle = Proceedings of the 5th International Symposium on Wikis and Open Collaboration
| publisher = ACM Press
| pages =
| isbn = 9781605587301
| doi = 10.1145/1641309.1641332
| accessdate = 2009-12-29
| quote =
}}</ref> how admins are chosen,<ref>{{cite conference
| url = http://www.thoughtcrumbs.com/publications/chi1364-burke.pdf
| title = Taking Up the Mop: Identifying Future Wikipedia Administrators
| first = Moira
| last = Burke
| coauthors = et al
| year = 2008
| conference = Conference on Human Factors in Computing Systems
| conferenceurl = http://www.chi2008.org/
| booktitle = CHI '08 extended abstracts on Human factors in computing systems
| publisher = ACM Press
| pages = 3441&ndash;3446
| isbn = 978160558012X
| doi = 10.1145/1358628.1358871
| accessdate = 2009-12-29
| quote =
}}</ref><ref>{{cite conference
| url = http://www.grouplens.org/node/415
| title = Wikipedians Are Born, Not Made
| first = Katherine
| last = Panciera
| coauthors = et al
| year = 2009
| conference = Conference on Supporting Group Work
| conferenceurl = http://www.group2009conference.com/
| booktitle = Proceedings of the ACM 2009 international conference on Supporting group work
| publisher = ACM Press
| pages = 51&ndash;60
| isbn = 9781605585000
| doi = 10.1145/1531674.1531682
| accessdate = 2009-12-29
| quote =
}}
</ref> and how to detect vandalism.<ref>{{cite conference
| url = http://web.media.mit.edu/~fviegas/papers/history_flow.pdf
| title = Studying Cooperation and Conflict between Authors with history flow Visualizations
| first = Fernanda
| last = Viégas
| authorlink = Fernanda Viégas
| coauthors = et al
| year = 2004
| conference = Conference on Human Factors in Computing Systems
| conferenceurl = http://www.sigchi.org/chi2004/
| booktitle = Proceedings of the SIGCHI conference on Human factors in computing systems
| publisher = ACM Press
| pages = 575&ndash;582
| isbn = 1581137028
| doi = 10.1145/985692.985765
| accessdate = 2009-12-29
| quote =
}}
</ref><ref>{{cite conference
| url = http://www.grouplens.org/node/113
| title = Creating, Destroying, and Restoring Value in Wikipedia
| first = Reid
| last = Priedhorsky
| coauthors = et al
| year = 2007
| conference = Conference on Supporting Group Work
| conferenceurl = http://www.acm.org/conferences/group/conferences/group07/
| booktitle = Proceedings of the 2007 international ACM conference on Supporting group work
| publisher = ACM Press
| pages = 259&ndash;268
| isbn = 9781595938459
| doi = 10.1145/1316624.1316663
| accessdate = 2009-12-29
| quote =
}}
</ref> This research serves to increase understanding in how Wikipedia works and to improve its functioning. Researchers approach understanding Wikipedia in a few different ways.
* [[Field experiment]]: Determines the limitations and strengths of Wikipedia's functionality or tests new functionality for editing, collaborating, navigating, etc., by developing and distributing modifications to Wikipedia's functionality. Field trials will usually need to recruit users under this policy.
* [[Surveys]] and [[interviews]]: Learns various aspects of editors and editing (e.g. demographics, motivations, activities) using pre-written forms or back-and-forth conversations. Requests for participation can be either general (i.e. random), or targeted to specific editors. Requests for participation require sending unsolicited messages, so surveys and interviews frequently require recruitment under this policy.
* [[Participant observation]]: Gains a close familiarity with the editing community by joining Wikipedia and doing the same work that Wikipedians do on a regular basis. They often casually converse with editors as a part of the collaborative editing process. Participant observers who do not send unsolicited requests do not usually fall under the SRAG's guidelines. However, they are recommended to disclose themselves as researchers and consult with the SRAG before starting their research project. In addition, participant observers who decide to interview or survey users need to recruit users under this policy.
* Offline analysis: Analyzes [[m:Data dumps|database snapshots]] of [[m:Data_dumps#What's available?|publicly available information]] to examine the history of encyclopedia construction. Offline analyses will seldom need to recruit users under this policy.

===Why are they here===
Wikipedia is an interesting medium for scientific research. It is one of the most visited websites on the internet, serving as an information resource to millions of users every day.<ref>
{{cite web
|url=http://www.comscore.com/press/release.asp?press=849
|title=694 Million People Currently Use the Internet Worldwide According To comScore Networks
|date=2006-05-04
|publisher=comScore
|accessdate=2007-12-16
|quote=Wikipedia has emerged as a site that continues to increase in popularity, both globally and in the U.S.}}
</ref> Scientists find it remarkable that an encyclopedia in which articles can be edited by anyone anonymously, and in which damage can only be repaired after it occurs, has quality comparable to traditional encyclopedias.<ref name="GilesJ2005Internet">
{{cite journal
| first = Jim
| last = Giles
| authorlink = Jim Giles (reporter)
| title = Internet encyclopedias go head to head
| journal = [[Nature (journal)|Nature]]
| volume = 438
| pages = 900&ndash;901
| month = December
| year = 2005
| url = http://www.nature.com/nature/journal/v438/n7070/full/438900a.html
| doi = 10.1038/438900a}}
</ref> They want to understand how the social dynamic of Wikipedia works.
Further, Wikipedia is one of the few examples of millions of people working together on a single project. The [[Wikimedia Foundation]] also supports the work of researchers by maintaining a public mailing list devoted to scholarly research of [[Wikimedia]] projects and releasing periodic [[Wikipedia:Database download|database snapshots]] for analysis.


==Recruitment of research participants ==
==Recruitment of research participants ==
Line 180: Line 31:
* The user is not a current or recent participant in one of the researchers' studies.
* The user is not a current or recent participant in one of the researchers' studies.


Community members who have received a recruitment message have no obligation to participate in the study. Users who do not wish to receive subject recruitment messages will be able to opt out via template {{tl|bots}} (for more information, see [[User:SubjectRecruitmentBot]]). Also, a bot-specific template allows for customization regarding the frequency or the number of studies to which a user would like to be recruited over a period of time.
Community members who have received a recruitment message have no obligation to participate in the study. Users who do not wish to receive subject recruitment messages will be able to opt out via template {{tl|bots}} (for more information, see [[en:User:SubjectRecruitmentBot]]). Also, a bot-specific template allows for customization regarding the frequency or the number of studies to which a user would like to be recruited over a period of time.


===Requirements for SRAG approval===
===Requirements for SRAG approval===
Line 198: Line 49:


==See also==
==See also==
*[[Wikipedia:Subject Recruitment Approvals Group]]
*[[Subject Recruitment Approvals Group]]
*[[Wikipedia:Subject Recruitment Approvals Group/Requests]]
*[[Subject Recruitment Approvals Group/Requests]]
*[[Wikipedia:WikiProject Research]]
*[[en:Wikipedia:WikiProject Research]]
*[[Wikipedia:Researching Wikipedia]]
*[[en:Wikipedia:Researching Wikipedia]]
*[[Wikipedia:Ethically researching Wikipedia]]
*[[en:Wikipedia:Ethically researching Wikipedia]]
*[[Wikipedia:Wikipedia in academic studies]]
*[[en:Wikipedia:Wikipedia in academic studies]]
*[[Wikipedia:Canvassing]]
*[[en:Wikipedia:Canvassing]]
*[[meta:Research]]
*[[Research]]


==References==
==References==

Revision as of 20:34, 25 February 2011

Scholarly research of Wikipedia is useful for understanding the encyclopedia's content, readers, editors, history, current state, and future. These results also yield important knowledge applicable to other open content communities. In addition to driving scholarly knowledge of such systems, this work can also give results that can improve Wikipedia itself. Much valuable research cannot be done without Wikipedia community members who volunteer to participate in studies. This policy exists to further three goals:

  1. To allow important research of Wikipedia and its users to take place.
  2. To ensure that that individual community members' preferences on whether and how frequently they are invited to participate in research studies are respected.
  3. To ensure that research studies are respectful of participating community members and community norms.

This policy describes how researchers can recruit members via talk page postings and creates the Subject Recruitment Approvals Group (SRAG), a public discussion group, to control who will be allowed to make those postings. SRAG receives applications from researchers to recruit editors to participate in research and manages the public consensus process which considers them. If the recruitment is approved, SOMETHING contacts selected community members and invites them to participate. Each community member controls whether and how often the bot can contact him or her (see en:User:SubjectRecruitmentBot).

Scope

This policy pertains to the activities of researchers within Wikimedia projects.

Researchers

For a more complete summary, see: en:User:EpochFail/What_are_these_researchers_doing_in_my_Wikipedia

Expanding articles is not the only way to contribute to Wikipedia. There is room for many different types of contribution: copy edits, content edits, template construction, vandal fighting, bot management, etc. Through the use of the scientific method, researchers contribute to Wikipedia by extended understanding about the community and bringing state-of-the-art technologies to editors.

Recruitment of research participants

Many types of interesting research are only possible with the ability to contact a random sample of community members to ask them to participate in a study. However, it is important to respect the wishes of Wikipedia community members on whether and how often they are contacted in this way. This section defines the Subject Recruitment Approvals Group to control recruitment message postings.

SRAG holds public discussions regarding applications to recruit community members. If a proposed research study finds consensus, SRAG will contact community members on behalf of the researchers.

As a rule of thumb, if a researcher wants to contact an individual with the intent of collecting data for use in a study, and that contact is unsolicited, he or she will need approval from SRAG. Specifically, a researcher must obtain SRAG approval to use tools within Wikipedia (e.g., talk page postings) to contact Wikipedia users when all of the following apply:

  • The communication would be the first by that researcher and the user (automated talk page postings by bot do not count as prior communication).
  • The message is not personal communication.
  • The message is not related directly to work on Wikipedia content, policy, or systems.
  • The user did not solicit the message.
  • The user is not a member of a WikiProject, work group, etc. managed by the researcher.
  • The user is not a current or recent participant in one of the researchers' studies.

Community members who have received a recruitment message have no obligation to participate in the study. Users who do not wish to receive subject recruitment messages will be able to opt out via template {{bots}} (for more information, see en:User:SubjectRecruitmentBot). Also, a bot-specific template allows for customization regarding the frequency or the number of studies to which a user would like to be recruited over a period of time.

Requirements for SRAG approval

Wikipedia and SRAG favor approving recruitment for studies that are of low risk to Wikipedia's editors and that are likely to benefit the community.

All approved recruitment must meet the following set of requirements:

  • The research must have merit, as deemed by the Wikipedia community. The benefits of performing the research will influence the approved sample size.
    • The research is designed to learn about communities like Wikipedia (wikis, open collaboration systems, mass collaboration, etc.)
    • The number of users approved for recruitment should be influenced by the potential value of the results of a study.
  • The nature of the study is adequately disclosed. All information that would be necessary for participants give informed consent to participate in a study has been made available.
    • Adequate information may not mean complete information. Many studies will require that the experimental design is not made public until after data gathering is completed due to the use of controls.
    • Subjects must know what they will need to do, what risks may exist, and how information about them will be stored and/or published.
  • The study will respect the preferences of community members and Wikipedia policy. The study will not unnecessarily make life difficult for community members – participating or not. The study must also respect the policies of Wikipedia and not cause undue harm to the system.
  • Results must be made freely available in a reasonable timespan. Specifically, this means that the results of a study must be either made freely available or submitted to a publication within two years of the start of recruitment.
    • If the results are submitted to a publication that requires a fee or membership to retrieve the results, the results (possibly rewritten to avoid copyright problems) must otherwise be made available for free within one month of the non-free publication.
    • It is possible to petition SRAG for extensions to these requirements.

See also

References