Wikimedia Fellowships/Project Ideas/A Digital Wiki Coach To Provide Focused Advice

← List of Project Ideas

Digital Wiki Coach To Provide Focused Advice

To get more community feedback I've made public my fellowship idea - submitted earlier with my fellowship application.

Abstract

Simone the Wiki Coach

Strategic WMF goals are framed in a social context and analysed. Two key dilemmas are considered within a game theoretic perspective and solutions are proposed to create an eusocial synthesis of concerns. The Media in the MediaWiki software's ecosystem is considered using McLuhan's tetradic method. The requirement are refined, taking form as Simone - a proactive autonomous agent. Her activity is illustrated in the scope of new user recruitment coaching. Driven by a goal to reform the community by promoting cooperative behaviour areas, Simone's overriding concern is to oversee good-faith editing without damaging the long-term user base of the project. The innovation of influencing editors not articles is explained. How Simone will scale and her long-term viability are then outlined.

Reframing the central questions

Editor loss, the gender gap, article quality improvment and low editor engagement are not isolated problems. They are interrelated and share three main factors.

• a defective media architecture — MediaWiki, extensions, gadgets, robot and tools.
• a piecemeal editorial policy — Many pages of policy, recommendations, essays, specialised jargon and actual individual interpretation.
• social and personal anarchy — WMF < chapters < developers < adminstrators < seasoned editors < novice editors < readers < indirect readers.

The chief contributing defect of MediaWiki is unplanned social scaling . The piecemeal development of governing of editorial policy "has led only to the use of violence in place of reason, and if not to its own abandonment, at any rate to that of its original blueprint" . The violance is deletion of good faith work by domain experts and novice editors. Both Social and personal anarchy, anarchronistic vestiges from the early days of the web which have consitently brought about the destruction of societies which have embraced them as values. With the rise of social networks and blogging these values have undergone a reversal — as identity has played a central role in promoting transparency in the Social age.

Of the three factors MediaWiki is the easiest to change. Editrial Policy, the rules by which wikis are goverened are the next most easily changed though it does require a wider much larger group of individuals. Social change is harder to produce as it involves large masses of people who are lagly unaware of the issues at hand. Piecemeal change in policy is often reactionary, populistic and counterproductive. However even at the social level change can be observed though it thousands of not millions of people who are the real stakeholder providing the bulk of funding, content and influence for WMF projects.

When problems are well posed thier solution is much easier to find?

1. How can the loss of active editor, due to conflict over editorial concerns, be reversed?
2. How can women be encouraged to play a greater role in wikipedia?
3. How can wiki increase user's engagment when greater involvement requires less editing and more drugery?
4. How can articles quality be improved with limited access to domain experts?

the solutions should therefore involve

1. Modify the media/editorial policy to cater for more editors.
   1. Find more editors.
   2. Try to keep each editor for longer.
2. Since there are virtual communities where women participate more then men - they should be emulated.
3. Reduce non-editorial tasks using technological solutions and produce new ways to engage all level of users.
4. Provide a mechanism to allow a domain experts contribution to get have greater weight.

The editor's dilemma: Choosing content over community

The editors dilemma embodies a conflict between two cross-cutting concerns, building content and building a community. An informal statement is: "in a wiki an editor has to choose between two evils: accepts new content of poor quality or reject the new editors who are the long-term future of the project". The incentive for quality content is clear and easy to quantify but the cost of losing a potential editor's is marginalised since it is shared by the whole community.

The tragedy here is that for a wiki as pointed out above the significant reuirement to the sucess of a crowdsourcing wiki is to have to be a large and active community. Luckily it is possible to untangle these crosscutting concerns and get the best of both worlds - quality content and new editors. To go ahead would inflict costs , risks and some impediments to the community but these would be ofset by the greater benefit. A wiki requires editors for many labour intensive tasks and so this rejection is not without a moral hazzard.

However the editors dillema is a strategic problem occuring over a long term - to adress it the tactical version must be resolved, this is the reviewers dillema.

The reviewer's dilemma: Choosing well-formed edits over quality edits

The reviewer's dilemma stems from the asymmetry of information between editor and reviewer. On the one hand is domain knowledge and on the other is the knowledge of policy. It is easier to write about what one know about, more difficult to review what you don't. In the absense of a formal review protocol a mandatory step in print encyclopedias — self-appointed reviewers are faced a few choices:

1. revert.
2. check the fact.
3. defer to another reviewer.

As the information assymetry increases the bias towards option one increases — reverting is a type of confirmation bias and considered by most reverters simlpy a conservative return to the status quo. To support such a choice any editorial defect is cited . Though in a xenophobic situations standards for rejection are very low and arbitrary - "gramatical errors" or "too many red links". The domain expert will only see the default in response to his co-operation. If it happens a few more times he will learn that his good faith work is not welcome, and tell his friends . The simple statement of this dilemma is at every edits machine and human reviewers must choose or reject domain based edits (and resolve human conflict) based only on their expertise of editorial policy . In almost all cases of good faith work — rejection of edits will occur due to editorial concerns and not due to domain problems. Wikipedia's growing popularity means its errors will often be repeated in print, and on-line.

Incentives for cooperation by design

Kindness

As articles mature, contribution by domain experts becomes the bottleneck for progress on quality. Studies on the evolution of cooperative behaviour using iterated games show that information asymmetry, anonymity, and lack of communication are the worst conditions for cooperation to evolve. This theoretical warning is directly mirrored in conditions in the field - the requirements that editors work in the isolation on new articles in their private spaces. Solutions comes in three main forms:

1. Improve incentive design - correction of the payoffs so that individuals see their benefits exceeds their costs in time, and work, coordination, and communication. So that good faith work remains a rational choice while still being able to address bad faith work effectively. The focus shifts to of cost of coordination, copy editing is put on the reviewer (for whom it is easy) and the cost of communication is shared.
   1. One example is to introduce a better review protocol for good faith work:
      1. 3-5 inline tags must be introduced into the article by the reviewer.
      2. Notice allowing editors a week to respond by indicate willingness to correct the problems.
      3. Notice requesting the editor demonstrate his domain expertise by taking a quiz. (could be fun within the scope of user engagement).
      4. The editor may request a peer review for a second opinion.
      5. If the edit is not illegal the un reverted a copy or link to the edit will be given to the editor in his user space.
      6. Reviewers who do not comply will lose his privileges for 2 weeks. (cooling off).
   2. Let experts and new users operate by paying forward
   3. Increasing visibility - article deletion effectively hides the users edits while reversion hides the editorial problems. Both these allow a moral hazard of reducing potential penalties to the reviewer.
2. improve game form
   1. Reducing anonymity - reputations cannot develop in a truely fully anonymous environment.
      1. Simone will assign users under coaching a unique avatar and advize how to link it to their signature.
      2. Simone will allow users to use a facebook id to identify.
   2. Simone will bear the cost of coordiating repeated encounters within smaller groups of individuals.
   3. Increasing visibility - article deletion effectively hides the users edits while reversion hides the editorial problems. Both these allow a moral hazard of reducing potential penalties to the reviewer.
   4. Using a social network identity would be even better to avoid deception.
   5. Repeated encounters between the same individuals are important to evolve from a reviewer's dilemma to a bargaining game.
3. diversify the population - an increase in circles of social and intellectual relatedness will improve capabilities of co-operation within each group.
   1. No more speedy deletion of for good faith work - speedy is unsuitable for good faith work. It send a bad messages to editors and reviewers.
   2. Restrict reviewing of non spam to categories which they have edited. This will coordinate reviewers and editors better and keep interactino within circles of greater relatednes.
   3. Invade the community with a new species of editor who can dominta though co-operation.

Some of these ideas would create a gap which is where Simone's comes into the picture. It will be possible for a reviewer to request it to intercede in such situations.

• Provide an example of software agent whose goal is positive social engineering rather than negative editorial automation.
• Not just increase the number and life span of editors but increase the domain expertise will reduce the high cost or reviewing edit.
• Reduce the need for subjective editorial decisions - which are usually handled badly by human expert.
• Create a minimal learning curve for acquiring editorial, social, technical and administrative skill.

I propose to provide positive alternatives to for instance templates for section and article under construction work easier use of problem tags, wider usage of incubation and requested article features. As these come to effect there would be far less need for a police state mentality and a return to the early meritocracy. I envision this happening socially in more and more places over time using technological means. This would make participating less stressful and more productive.

Certain of the early instigators of the more esoteric policies on Wikipedia have voiced regret once the protocols they innovated got wider and wider usage. Other policies solve localized problems by imposing unnecessary criteria on the whole community.

Wiki Evolution from Asocial to Eusocial

Biting newcomers

Wikipedians has been described as xenophobic in their attitude to new comers. Not surprisingly there Wikipedia jargon for this behavior is biting the newcomers. This roots of this asocial dynamic can be understood using simplified models like the reviewer's dilemma. This law of the jungle behaviour with its mad logic, brutish cunning and has evolved in countless encounters has. But this begs the question - could a new 'species' of Wikipedian invade and dominate the population? Considering that such invasion have been described by Sue Gardner in her address at Wikimania 2011.

To answer this question eloquently it is best to use a powerful social metaphor for the wiki and it is the beehive. Within the lens of beehive it becomes preeminently clear that a change of the asocial paradigm to the eusocial one is required. In the real world it means increasing userrelatedness and changing the form of conflict.

In a hive bees use chemical signatures to differentiate hive members from foreign intruders. In the Wikipedia settings behavioural signatures are used. These behavioral signatures have developed though iterated editing games in which the dominating strategy is tit for tat. To be recognised as community member, a newcomer must make a protracted commitment of time and effort.

Eusocial Communication

Like a real hive, a wiki requires constant policing to ensure that drones are not pursuing their own agenda at the cost of precious shared resources. In the hive these freeloaders once identified are restrained or destroyed and their progeny eaten. In the wiki sock puppets are unmasked using a check user and are then banned and their works systematically purged. The hive must deal with parasites (COI) and invaders (Paid Editors), and a degree of anarchy (ArbCom). Each generation is very closely related and the oldest, most senior maintains the status quo.

So one may argue that this is the natural order of things. New editors should be viewed with great suspicion -- Are they foreign agents? (Paid Editors) trying to subvert, or perhaps self-serving parasites? (COI). The default solution is one of extreme hazing - the old generations retaliate against individuals who might be suspected of hurting the group, until they learn to walk the line or depart. And the few who graduate will maintain the asocial dynamic. This is a form of total war — it is extremely expensive to wage and is uncommon in nature.

However such solutions are unnecessary and no longer acceptable — it is time to groom another breed of Wikipedians who default to settling differences sporting within a "limited war" paradigm rather than the picric outcome of todays "total war".

Understanding Media in MediaWiki

In the early dats of Wikipedia it was a simple technology, it lacked experienced editors or an editorial policy. It had no established hierarchies of influence or personal stakes. All these things are signs of progress, yet they present impediments to contribution. Consequently the threshold cost of contribution was low. However as the project matured it was modified piecemeal to address immediate concerns - without a guiding hand the changes have often been regreted in retorspect.

Marshal McLuhan in his seminal work understanding media and later text innovated the tetrad as a tool for analyzing the impact of new media. By provide such an analysis for Simone, it will become possible to better understand the grasp the scope of its impact. Next Simone wil be compared with some other media being utilised and developed today.

Tetradic Analysis

Simone Tetrad

• Enhances - review of good faith edits for: article quality, editorial standards; user recruitment, and coaching; group collaboration under peer review.
• Reverses - editorial complexity, community reduction, coordination cost, recruitment of new editors, policy complexity.
• Retrieves - organic editing , pay it forward editing shorter learning curve. reduced cost of contribution.
• Makes Obsolete - early seletion of new articles; subjective human assessment of editorial level for articles.

Simone and Normal Bots

Simone reuses existing bot framworks but will be different in two senses:

• It is aimed at altering user's editing behaviour and not to fix or revert thier edits.
• It is planned to preempt future problems rather than react to old ones.
• It is planned to rival human asessment of editorial issues.

Simone and Abuse Filter

Simone will also build on abuse filter but different in three senses:

• Abuse filter provides negative options to a few admins. Simone chooses from many positive choices for many many editors.
• It will provide many options in the cooperate category rather than provide one options in the default category.
• Abuse filter is inteded for abuse, but Simone is planned to cover a grey area of good faith work that borders on "abuse".

Simone and The Visual Editor

On the other hand MediaWiki developers are working to reduce the complexity of editing via the Visual Editor. While a visual editor will reduce the technical level of editing, but does not address any of the social issues discussesd above. This solution may have an unintended result of increasing new user hazing. This could happen because new ediors will be harder to assess on using technical signitures and will be judged more harshly on policy and stlye — which will appear to be more arbitrary.

Case Study: The Decline and Loss of Organic Editing

No other loss of functionality present a greater stumbling block than loss ofOrganic Editing . Organic editing is the ability to create sentence by sentence a new article in main space. Organic also refers to natural growth of the articles. Yet today anyone bold enough to attempt to do this will be censured by patrollers and adminsitratos at every save and informed that her work is not welcome. It will be nominated for speedy deleation on grounds of context, content or non notability. However notability is not an objective assesment, it is simply the harshest dictum to defend for a new editors. When the one editor complains they will be informed that thier article may be exiled to the user space. It could apply for a visa when it reaches maturity. Unfortunately userspace is not a place for natural collaboration. Your work get no exposure and other new collaborators are often reverted there.

So why was organic editing lost? As predicted by The number of edits in a day depends on existing article counts theb since or as the number of articles grows much faster then the number of patrolers in a given week. And this leaves out the growing complexity introduced by a rise in quality

• "Don't make many small edit - save them as one large edit".
• There are less red links being added by new editors
• I reverted your good faith edit because it had many red links.

Corrective Action

Whenever a page is edited it will be analysed and stored into the index it with a list of current issues. Searching the index will allow Simone a greater level of intelligence. Running strategic queries it will enable planned proactive action rather than a reactive interventions.

Simone's explainable activities would be driven by different causes. However strategies will assist develop full and order behaviour and where multiple options exist it will allow choosing the option which has the highest priority.

1. Simone objectively advises new users how they could best contribute. New users will, once again contributing with at a near zero cost - on the assumption of good faith work and that they would pay their dues once they have ascended up the learning curve. They would be more rapidly progress in skill and join rank as the next generation of our community - a generation that is less obsessed with unwritten rules, one which would not require wiki-lawyers.
2. Expand the program to bring articles systematically to featured levels by providing editors with expert advice.
3. As the Simone matures it could be directed by users to instruct users of policy infringement and by project to direct thier members to collaborate on hot spots.
4. To reach a far greater audience and higher levels of engagement a reward based game mode would be developed.

Rationale

The project is made of software elements unified into a bot I will refer to as Simone. It will either work as an external user or will be integrated into the editor. Work will begin with sensor modules which can assess a given revision's merit in various dimensions. These include a comparison with Gold Stadard and a Minimum standard.

technically these can range from simple rules to search wisdom of crowds ranking algorithms to more traditional statistical discriminative models.

• Next it will select the highest priority issue from its queue.
• The bot will summarize the top issue in a one line human readable report.
• Next it will offer advice on how to rectify the problem.
• Finally it will offer some alternatives and ask for feedback.

Illustration: advice sequence for new user

The following figures illustrate a sequence of advice notices that would be generated by Simone as it coaches Hildur, a new user from Iceland working on her first article on Kafka's story The Trial. The UI element shown is a talk page/user talk page based system that would be inserted by Simone.

new users coaching can start in several ways

1. proactive detection of an anonymous editor / new article by a registered user.
2. reactive intervention based on a biting incident sensor.
3. invocation from Afd - i.e. a new suspended delete sentence. Either correct the quality issue within a week or be deleted. Simone will be used to carry out and enforce the probation - if successful the probation will end by removing the probation notice. If no action is performed the article will be deleted/userified.
4. invocation by Huggle, Twinkle or a similar tool with Simone integration.

Step 1 - Join Wikipedia and apply for coaching to save your work

Current Goal: Bring the article to Threshold Level
Wikipedia Wants You! Hello stranger, thanks for helping out with this article. We always need more helpers	Your report card for: The Trial Current Status: The last submission has some editorial problems and since you are unregistered it is likely to deleted soon. If you are planing to further improve this entry you can save your work by improving it. If you are interested in advice on how to improve it to the threshold level of inclusion choose one of the options bellow. Article Status Aspect Overall Score Δ Threshold Context +5 20 Content +3 15 Notability +3 8 References +5 15 Style -3 0 Social 0 0 Your Mission — Sign up for coaching. Action Items Options Accept Reject Register for coaching accept reject To mark the page as "under construction" for a day accept reject I want to view a video on the policy and the advantages of coaching. accept reject Please don't bother me, I know what I'm doing. accept reject I've done all that I can. accept reject
( opting out | feedback | about )

Logs

Simone believes that one should always be testing. She is careful to make her logs useful for analysis using R as well as for keeping track of action that needs a followup. On creating the above message Simone will append to its log:

• <NewUserEvent>

<Source>1.234.44.215</Source>

<revId/>486369447<revId/>

<Sensor>unregistered edit</Sensor>

<Opt id="100"/><Opt id="101"/><Opt id="105"/> <Opt id="202"/><Opt id="230"/>

<Opt id="G20"/>

<NewUserEvent/>

This log includes:

• the user/i.p.
• a time stamp. (not shown)
• the revisionID which comes from https://en.wikipedia.org/w/api.php?action=query&prop=revisions&titles=The_Trial&format=xml
• the options used - for doing optimization of advice using the R stats package.
• the last option is a User Interface option for doing simple A/B testing R stats package to.

Notes

• Points out that there is a problem with the edit.
• Advises the use to join sign the user.
• Can allow measurement of interest in anonymous editing.
• Option 2 will place an organic editing template in the page with one day expiry. It will also place a notice on the talk page.
• This user element would be further customised for capturing new users based on stylometric analysis of their input. To consider gender and nationality.
• Most of the options are just hooks to get the user to sign up. For example I've done all that I can will inform her that she can do much more - Wikipedia has mistakes she can correct and so on.
• Depending on integration with MediaWiki
  • If in the submission loop the advice will expire immediately if ignored.
  • If it is just put into the IP user page - unregistered advice expires at is never clicked on would expire after 3 days and marked as a bounce.

Step 2 — Learn about context

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach Hildur, thanks for enrolling for wiki coaching.	Your report card for: The Trial Current Status: Your article is so short that it could be marked for speedy deletion under article A1 - which means that it does not provide enough context to justify its inclusion in. Your article: on The Trial has been temporarily protected from speedy deletion. It is a good start – but 50 words is considered too short for a Wikipedia article. Once you add at least another 100 words, it would qualify this article as a stub article. Article Status Aspect Overall Score Δ Threshold Context +5 20 Content +3 15 Notability +3 8 References +5 15 Style -3 0 Social 0 0 Your Mission — fix context Action Items Options Accept Reject you can expand the article - don't forget to include details or technical terminology. accept reject you can let add a template to protect it while you work accept reject you can move your article to your user space accept reject you can see an explanation video tutorial about context accept reject to dismiss this matter for now. accept reject
( opting out | feedback | about )

• There are many ways to improve context.
  • Adding info boxes
  • categorising
  • Writing sufficient details.
• Simone will be basing it context score on minimally unique word n-grams. This will quickly indicate if the article if covering something new or not. Even if there are no unique terms it will also use word vector cosines to identify the closest articles.

Step 3 — Learn To Wikify Text

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach You are making good progress, the edit brings the content score to 4/100. But you article is an orphan.	Your report card for: The Trial Current Status: speedy deletion A1 - your stub lacks context. To fix this all you need to do is add some internal links. We call this processes wikifying. Article Status Aspect Overall Score Δ Threshold Context +5 20 Content +3 15 Notability +3 8 References +5 15 Style -3 0 Social 0 0 Your Mission — Wikify some text Action Items Options Accept Reject to wikify a word use braces like this [[word]]. accept reject for a how to video tutorial about why it's important to wikify articles. accept reject to get a recommendation on what to wikify click here. accept reject to dismiss this matter for now. accept reject
( opting out | feedback | about )

Notes

• Syntax should be taught from simple to the more complex.
• If difficulties arise for engaging users with pedagogically significant advice or goals Simone could review a user's edit history to gauge the existing skill set.

Step 4 — Learn about Notability

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach great your content is now scoring 7/100	Your report card for: {{{ArticleName}}} Current Status: Your article has low notability and may be marked for speedy deletion under article A7 or A9 - which means that it does not indicate the importance of the subject to justify its inclusion. Your article: on The Trail has no references to external sources - add references and wikifi the text so you can better demonstrate what other articles your work touches on – these will quickly boost your score by increasing the visibility of your articles notability. Article Status Aspect Overall Score Δ Threshold Context +5 20 Content +3 15 Notability +3 8 References +5 15 Style -3 0 Social 0 0 Your Mission — demostrate the article importance for inclusion in an encyclopedia. Action Items Options Accept Reject you can expand the article accept reject you research and insert references from print sources. accept reject you research and insert references from online sources. accept reject you can watch a tutorial about notability and reliable source. accept reject to dismiss this matter for now. accept reject
( opting out | feedback | about )

note: at this stage we would like to teach the user an editorial skill set as well as a social one.

• the Editorial requirement that she must show that the article is important enough to be included in wikipedia. This will require more work, the less notable the article is

1. don't include it to better answer the issues of notability

However there are probably more than 5 options to fulfil this mission. If we are able to collect 20 options and track each options according to its success — Simone can optimise this advice. I.e. She will pick 5 options at random out of the 20, respecting each option's probability of success.

Step 5 — Learn to add categories

Current Goal: Bring My First Article to Threshold Level
Simone Wiki Coach great your content is now scoring 7/100	Your report card for: {{{ArticleName}}} Current Status: Your article has low notability and may be marked for speedy deletion under article A7 or A9 - which means that it does not indicate the importance of the subject to justify its inclusion. Your article: on The Trail has no references to external sources - add references and wikifi the text so you can better demonstrate what other articles your work touches on – these will quickly boost your score by increasing the visibility of your articles notability. Article Status Aspect Overall Score Δ Threshold Context +5 20 Content +3 15 Notability +3 8 References +5 15 Style -3 0 Social 0 0 Your Mission — demostrate the article importance for inclusion in an encyclopedia. Action Items Options Accept Reject you can use hot cat accept reject you can create a new category. accept reject you can use the category recommendation too accept reject you can watch a tutorial about categories accept reject you can to dismiss this matter for now. accept reject
( opting out | feedback | about )

• option number four would be supported by an automatic categorising tool

Step 6

Once threshold is reached the article will be safe to remain in main space. The user will have a positive payoff for her work in the form of a permanent contibution to wikipedia. But there is still much more that the user can still learn. By completing the new user's coaching she be able to create new articles without significant risks of deletion. Thus Simone will continue to instruct the user and cover more points such as

1. how to use her talk page.
2. how to use the user space sand box.
3. how to customise their user space for eusocial communication.
4. how to view requests for creation.
5. how to interact with reviewers using wiki love.
6. the new article creation wizard.
7. how to get a human coach
8. how to join project.
9. how to interact with templates
10. how to use Template:Inuse-section
11. how to join project.
12. how to improve their user page.
13. how to make tables
14. how to watch pages
15. how to insert images
16. how to delete old/broken references

the curriculum will include trails based on

• mediawiki software trail
  • e.g.how to use preferences to add some gadgets
  • how to master the rest of the wiki syntax
• wiki's policy trail
  • core editorial policies
  • recomended best practices (Simone good wikipedian training)
  • netiquete - socially accepted behaviour

Advice Strategies for Other Use Cases

Advising only on new users might seem to be a more focused approach but it is defficent in servral areas

• it will not mobilize sufficient human resources to reverse the underlying reasons for editor loss
• it will be restricted to situations where machine judgment is better than human Judgment.
• it will miss on some of the greatest benefits that Simone can provide - discounting coordination and communication costs.
• it will mean that experienced users will not be familier with the possiblity of invoking Simone to place new articles and users under probation.

I will present a some strategies for executive coaching, article improvement. Since there exist an very low activity admin coaching projects this is also a candiadte for intermidiate level of editors. It can stress finer points of policy.

Retention and Engagement for Seasoned Users

Loss of seasoned users is the flip side of new user recruitment. Experienced users may run out of information to contribute or they may end up being promoted into role that are less satisfying and experience burn out. Recent studies now provide deep knowledge of this problem. Studies show that under the right conditions these users can be retained longer within the community.

By adding a burn out sensor users who fall out of the fold can be contacted and offered to received executive editor coaching acknowledging, perhaps for the first time, their superior status. There is a cost of coordination of

Simone is goal and mission based. She will, based on feedback provide them with tasks. These will involve work that requires human touch and is therefore harder for Simone to do.

• Peer review in their area of editing.
• Peer review in their area's of expertise.
• Crowdsourcing of articles for importance and quality.
• AFC in their areas of expertise.
• Quality improvement missions.
• Translation missions based on criteria in the next section.
• Missions will be varied, including technical task and social tasks.

• executive coaching will complement the new user coaching program in places where Simone is unable to assist.
• report cards in executive coaching will be used to give a the tasks a game-like feel.

Quality improvement

Article quality improvement is the primary driver of Simone's behaviour. However if user have completed new user coaching they can either select another module or stay with the quality improvement module which will give feedback on editing progress.

Simone will still detect problems and direct users to fix them.

• Outstanding inline tags in the users previously edited articles.

• Quality improvment will be able to select Users who can translate sections from another language where the other language has a better article.
• report cards in Quality Improvment will also be used to give a the tasks a game-like feel. This include offering badges and barnstars for sucessful missions.

Simone's UI

Simone's U.I. is designed to expedite coaching activities:

• Simone develops a one on one relationship with the student.
• It monitor's the coachee's performance using the dash board
• It defines a specifc goal to work on.
• It breaks down the goal into missions which can be acomplished in a single editing session
• It provides feedback on previous activity.
• It engages the users by providing intervention feedback in the form of advice.

The UI used in the Ilustrations would be

• A/B optimized for greater usability.
• Streamlined to make better use of space.
• Adding quizz widgets to get a tighter feedback loops with even more engagement.
• Adding media widgets for audio/video/text
• Adding a IM chat widget for interaction with human coaches.

Floating UI

• Suplamented with a gadget to place the UI in a coaching tab to make access to coaching easier and more personal.
• Later a more dynamic Floathig UI may be developedabove and to the side of the editer/browse window.
  • This would allow Simone to inspect edits in real time as well as show videos to user's while they edit.

More so when it comes to web-based applications and even more so in the context of extending MediaWiki. MediaWiki is years behind normative content management systems, which provide simple UI widgets and an easy extension mechanism. The WMF claims to be open source, and constantly solicits help from volunteer developers. However in recent years the practice has become increasingly less open, there is little trust in community submissions and therefore little help from volunteer developers. Recent discussion on wiki-techL has highlighted that it can take over two years to review 3rd party plugins -- long enough for them to become obsolete.

I have made some paper UI design but the Template above represent a second iteration of these ideas. While they could be vastly improved and streamlined this should be done within the scope of the project itself. As mentioned above I would propose short development cycles with testing using real against real users and getting feed back from active coaches.

Like Simone might say there are 4 options regarding integration:

1. 1 Using a Message based U.I. and no integration.
2. 2 Minor MW and UI integration.
   • An event hook to allow providing advice during a Save page event.
3. 3 Major MW integration
   • Requires the 20% time of an experienced MW developer.

Provide floating UI elements within the edit page.

Provide Coaching NAmeSpace or Tab

1. 4 Full MW integration, highly customised A/B tested UI.

Integration into the core - requires 2 developers

• A MediWiki developer.
• An Information specialist.

It may be expidient to intergrate into Simone a capabilty for collection information using forms. This is part of a use case for collecting feedback in surveys. Another use case would be adminstering tests to verify a user's progress before a graduation event. If this is identified as important such a component will be intergrated into the UI.

Simone's Invocation API

Simone will also have an refferal mode. It would be possible to introduce a referral template into the page by a reviewer to invoke Simone's presence and ask her to present certain advice to an individual at a page. The reational for this is to allow quick integration of Simone into tools like Huggle, Twinkle etc. This will in effect introduce positive alternative into these tools. This will have the effect of changing the standard tactics from total war to limited war in the good faith cases.

Scaling

The project can scale in four dimensions:

Technological scaling - data based sensors

Simone's information architecture is designed to grow organically with the MediaWiki community it advises. Technological scaling of this sort involves an increasing discrimination capability (more points in space); and using feedback to fine tune a sensor's activation threshold (e.g. gaugeing the notability of a Soap opera episode article is not the same task as an article on a war.

This type of scaling requires creating sensors that become more refined as the wiki grows.

Simone's response to the above strategy is to provide smarter data driven sensors. The data for these will come from sampling the wiki so they will be normative rather than proscribing what may become unrealistic utopian criteria. By considering more data and by clustering this data to more levels it is possible for the same basic sensor to track changing dynamics as wikis evolve over time. Both positive and negative feedback mechanism can then be used to adjust the behaviour signatures which it detects .

The sensor developer's goal is Pareto efficiency. With a robust detection of Context, Content, Notability, References , Style and Social behavioral signatures in user edit and talk Simone will to preempting 20% of the most socially damaging reversion/deletion events which should dramatically reduce community oversight by about 70-80%.

The handling of remaining incidents would require human intervention and provide greater engagement. If not then once such a milestone is achieved the method can be refined by adding sensors and refining existing sensors to track the top 20% of remaining issues. This is likely since deeper underlying problems (such as cultural and gender gap issues) may be obscured by more visible issues.

Sensors could also be added to address additional (non editorial) concerns so long as the phenomena is discernible using techniques of discourse analysis. This simplest should involve retention of experienced editors.

In cases where a rules and pattern are too weak to detect issues (Such as neutrality) it will be necessary to produce a collection of good/bad edits and train models to discriminate their issues. Building some "Gold standards" will be one of the project's mile stone since it will involve community expertise and feedback.

Also if such issues become a priority the community will be requested to assist by tagging problematic revisions. Though this can be accelerated by earmarking a budget to outsource corpus development by outsourcing the work online using Amazon mechanical Turk at a token cost per tagged revision. This approach has been reported to be successful in developing an anti spam corpus.

Some initial sensors that would be developed:

• biting sensor - incidents of new user biting.
  • bite scale from 1-10.
  • biter's leader board to mitigate biting and identify delinquent biters.
• new user sensor
• new editor sensor
• new article sensor
• afd sensor
• speedy sensor
• article information sensor
  • determines quantitive the amount of information in this article.

Community Scaling – Advice

Sensing problems is not enough – Simone needs to offer useful advice. Since this will be the most visible aspect of the project it needs to be developed with great sensitivity to users and community acceptance. Advice defaults will provided but this will only be a starting point. Again community input will be requested on this subject. Suppose the community solicits our projects with 3 advice notices for a certain issue - say lack of Citations. Simone would systematical test these messages and score them based on their success (i.e. check how each advice correlates with the sensor score for Citations in following edits.) Tracking advice - and correlating using R in a DB will be another Project milestone.

Sample Advice for Context Issue

Probability	Advice
0.5	Write another 120 words
0.25	Add an info box template
0.0625	Add a category.
0.0625	Add the article skeleton using section titles.
0.0625	Add categories.
0.03125	Add project description in talk page.
0.03125	Add project stub.
0.03125	Add an interview link to a version in another language.

• Advice is offered in the scope of missions which are grouped under asingle goal.
• Goals are collected in strategies.

Sample Strategy for New Users

Sensor Definition	Goal	Mission
S1	Threshold	Provide Context
S2	Threshold	Provide Content
S3	Stub	Provide Content
S4	Stub	Notability
S5	Stub	Add project description in talk page.

Deployment and Scaling on Language

To speed Simone's' community acceptance, default advice will be provided to the initial project by the developer. This is perhaps the most labour intensive task -- primarily because Simone would like to support new user in their first thousand edits to bring them into relatedness with other wikipedians. To provide meaningful and engaging advice different editing issues would have to be explained in text and video.

Once Simone has stabilised through three or four initial development cycles it will be ready for its initial deployment. However Simone won't need all this media in order to go online. IT will co-operate with talented coaches to develop more advice for new situation. It will be done by issuing a "Request for Advice" - a crowd sourcing request to the community. This is a strategy to increase the number of stake holders involved in the project and who wish to see it succeed. The best scoring advice will periodically supplant the default advice and undergo translation. For example:

• a sandbox based tutorial
• a in-depth study on making references

Since the initial default advice will be in English, these will be Internationalized by translating using the resources of

TranslateWiki. This would involve:

• Making its sensors cross lingual.
• Translating its templates
• Translating its advice.

I have consulted with the WMF developer of TranslateWiki which handles the Internationalization and localization of MediaWiki and he recommended for the translation of the templates:

• https://www.mediawiki.org/wiki/Help:Extension:Translate
• https://www.mediawiki.org/wiki/Help:Extension:Translate/Page_translation_example

Scaling on Projects

For example Wiktionary has quite different editorial guide lines from Wikipedia as do wiki source and Wikibooks. Scaling Simone for work on new projects requires:

1. determining a socio-editorial policy using the deletion/reversion history and deriving concrete list of violation.
2. sending out a Request For Advice to the project's community.
3. beta testing the behaviour on a control group.

Thus Each project could adapt Simone to its needs.

Metrics

As already explained metrics are at the very heart of this projects - we call the sensors.

• Simone will generate detailed logs of its activity.
• Reporting will be generated from the logs.
• Simone will be configred initialy to have a control group which will not get the benefits of coaching.

Simone will provide periodic reports with summary statistics on

1. incident (selected sensors correlating to high level events).
   1. new user biting
   2. speedy nominations
   3. article improvment from C to B and other levels
   4. user projected activity based on recent edit history
2. incident reponse
   1. new user biting interventions.
   2. speedy interventions.
   3. user coaching enrolement.
   4. editor's leaving coaching.
3. long term change.
   1. new editor acclimatization to the community.
   2. editor under coaching mass action. The overall edit count * positive change due to these as seen by sensors.
   3. morbidity rates estimates for editors under coaching and the control group.

Long term - Sustainability

No claims about the long-term of the project only suggestions. I hope it is taken over by the WMF development team and more tightly integrated into MediaWiki whose shortcomings it addresses. However, since it is not the traditional model for robot based tools, the more realistic community process will be explained in the different roles of different stake holder involved.

Role of Developers

Today Bots developers are the most influential members of the community. Within the limited scope of thier projects they provide more agile solutions than the WMF. They are also better aligned to the community's needs because not only must they get approval to run their bots from committee, they must then respond to issues raised by users.

Simone will be created in short development cycles as a MediaWiki based project - using git for source control, labs for test deployment, Bugzilla for bugs, code review for reviewing external patches once these are provided, translate wiki for internationalization.

Bots are mostly constructed from a few exiting frameworks and thus there exist a possibility of a high mobility within the community if a better technology is introduced. It some areas, such as ranking articles for completion and importance, Simone would provide a significant technological edge. This suggests that there is potential for poolling resources with exiting robot operators.

She will be deployed and start collecting data early on and generating reports. Her nature is planed to be extension. This means that the community will be given freedom to contribute to this project.

To promote Simone amongst developers:

1. Her code will be available as open source.
2. Data analysis scripts will be provided.
3. Documentation will be provided on installation.
4. Evangelism at at Wikimedia, and hackathons and other conferences would boost this community.
5. After the end of the fellowship the volanteers will be found to take over and maintain the project.

Role of Administrators

The stake of administrators in running Simone - is that she will oversee editing in a the grey area between spam and good edits. This will provide a massive reduction in patrolling.

After undergoing the robot approval process Simone will be operated on Wikipedia. As usage will increase responsibility to operate it will be transferred to qualified Administrators.

Simone would be primarily controled by a configuration files. If required a web-based interface could be added too based on edit filter. Administrator will:

• fine tuning its sensors and their thresholds.
• adjusting corpuses for normative standards
• add remove or change strategies
• add remove or change defining goals, mission, and advice hierarchies.

Role of Policy Makers

There is a number of suggestions proposed to policy in this proposal. These changes will be performed informaly by Simone. As well tempered users invade the community these policies will gain weight and gradualy replace the existing practices.

A key assumption is that as Simone influence will be felt by policy makers. In this view when they decide to repel anti-social policies Simone will be reconfigured to fill in the resulting vacuum. This will allow to reform current policy to a shorter and clearer set of guidelines — more suitable for a larger more diverse society of contributors.

Role of Human Coaches & Domain Experts

Human Coaches will be asked to provide expert advice on coaching and help for debugging based on the open source doctrine of many eyes find bugs faster. Thiers stake in Simone is that it will make coaching more fun and productive. For humans the tasks will be less technological and more of a social activity.

Using a project page they will be asked to provide original and alternative texts for advice. They will be invited to suggest new missions, goals, strategies and sensors for current development cycles. Once these suggestions are provided they would be added into Simone as part of the next development cycle

These will be use by developers and administrators.

Domain experts can be paired with more seasoned editors to work as pairs. One contributing domain knowledge, the other providing wiki know how. This is another situation of changing the game from zero-sum to win-win rule.

Role of Projects within a MediaWiki instance

Specific project in Wikipedia such as biography of living persons have their own needs. Simone will be able to address these projects as a group, recommend new users to join the projects teach people the rules required in these and arrange collaboration on top priority articles. This coordination will allow a participating project to get boost in personal and to focus them on work in teams on the articles of greatest importance. This is something of a lost capability now that matching articles to individuals. Simone's user Soliciting module could be fine tuned from the start to training users in their areas of expertise in a socially acceptable way.

Role of The Researchers

Simone will provide reports on its activities. Its high level strategies would have better results of based around research carried by experts in this field. By providing better answers to the key theoretical questions it will be possible to define better goals and sensors and preemption strategies for Simone to pursue.

For example if there is more knowledge on conflict situation Simone could be instructed to preempt conflict ; existing user retention.

However, if researcher were to discover a better model of the editing and review game or a more suitable incentive paradigm these would be used by Simone to coach better editors for tommorow.

• life span in days/edits of editor generations.
• percentage of new articles deleted under each SD clauses.
• percentage of new articles deleted under other clauses.
• degree of communication before reversion event.
• time from first edit to becoming an administrator.
• persistance of unregistered editors

Community Role

The most important role for Simone is that of community interaction. As users work with her she will become wiser and able to give better advice. Maintenance of advice as Wikis' policy changes is one major role of the community.

Submitted by

Oren (talk) 13:43, 3 March 2012 (UTC)

Endorsements

This section is for endorsements by Wikimedia community volunteers. Please note that this is not a debate, vote, or poll, but is rather a space for volunteers to describe in detail why they think a project idea is of value. If you have concerns or questions rather than an endorsement to make, please use the idea Talk page. Endorsements by volunteers willing to work in collaboration with a fellowship recipient on a project are highly encouraged.

 Support, I believe implementing this idea is very helpful for giving a smooth start to new wiki editors, and for saving expert editors' time spent on coaching new editors. --Ciphers (talk) 05:12, 21 March 2012 (UTC)