View source for Wikimedia Fellowships/Project Ideas/A Digital Wiki Coach To Provide Focused Advice

{{process header
 | title    = Digital Wiki Coach To Provide Focused Advice
 | previous = ← [[Wikimedia_Fellowships/Process#Open_project_ideas|List of Project Ideas]]
}}


<!-- =Simone The Content “Coach” - An Autonomous Agent for Content improvement and User Retention= -->

To get more community feedback I've made public my fellowship idea - submitted earlier with my fellowship application.

==Abstract==
[[File:Wikipe-tan_coaching.png|thumb|150px|Simone the Wiki Coach<ref group="note">this and other avatars used for Simone are based on Wikipe category are taken from commons and were contributed by their respective creators.</ref>]]
Strategic WMF goals are framed in a social context and analysed. Two key dilemmas are considered within a game theoretic perspective and solutions are proposed to create an eusocial synthesis of concerns. The Media in the MediaWiki software's ecosystem is considered using McLuhan's tetradic method. The requirement are refined, taking form as '''Simone''' - a proactive autonomous agent. Her activity is illustrated in the scope of new user recruitment coaching. Driven by a goal to reform the community by promoting cooperative behaviour areas, Simone's overriding concern is to oversee good-faith editing without damaging the long-term user base of the project. The innovation of influencing editors not articles is explained. How Simone will scale and her long-term viability are then outlined.

==Reframing the central questions==
'''Editor loss''', the '''gender gap''', '''article quality''' improvment and low '''editor engagement''' are not isolated problems. They are interrelated and share three main factors.
*a defective media architecture — MediaWiki, extensions, gadgets, robot and tools.
*a piecemeal editorial policy — Many pages of policy, recommendations, essays, specialised jargon and actual individual interpretation.
*social and personal anarchy — WMF < chapters < developers < adminstrators < seasoned editors < novice editors < readers < indirect readers.

The chief contributing defect of MediaWiki is unplanned ''social scaling'' <ref name="Shiky">[http://www.shirky.com/writings/group_enemy.html A Group Is Its Own Worst Enemy] by [[w:Clay Shirky|Clay Shirky]]</ref>. The piecemeal development of governing of editorial policy '''"has led only to the use of violence in place of reason, and if not to its own abandonment, at any rate to that of its original blueprint"'''<ref name="Popper">[[w:The Open Society and Its Enemies|The Open Society and Its Enemies]], Vol 1 & Vol 2, P.161 [[w:Karl Popper|Karl Popper]]</ref> <ref>[http://www.youtube.com/watch?v=b2MuXzV0MJ8 Where Wikipedia has gone wrong, what we can do to bring it back on track] by Dror Kamir Wikimania 2011 Haifa</ref>. The violance is deletion of good faith work by domain experts and novice editors. Both Social and personal anarchy, anarchronistic vestiges from the early days of the web which have consitently brought about the destruction of societies which have embraced them as values. <ref name='Freeman70'>[http://flag.blackened.net/revolt/hist_texts/structurelessness.html The Tyranny of Structurelessness] by [[w:Jo Freeman|Jo Freeman]] (1970).</ref> With the rise of social networks and blogging these values have undergone a reversal — as identity has played a central role in promoting transparency in the Social age.

Of the three factors MediaWiki is the easiest to change. '''Editrial Policy''', the rules by which wikis are goverened are the next most easily changed though it does require a wider much larger group of individuals. Social change is harder to produce as it involves large masses of people who are lagly unaware of the issues at hand. Piecemeal change in policy is often reactionary, populistic and counterproductive<ref name='Freeman70'/>. However even at the social level change can be observed though it thousands of not millions of people who are the real stakeholder providing the bulk of funding, content and influence for WMF projects.

When problems are well posed thier solution is much easier to find?
# How can the loss of active editor, due to conflict over editorial concerns, be reversed?
# How can women be encouraged to play a greater role in wikipedia?
# How can wiki increase user's engagment when greater involvement requires less editing and more drugery?
# How can articles quality be improved with limited access to domain experts?
the solutions should therefore involve
# Modify the media/editorial policy to cater for more editors.
## Find more editors.
## Try to keep each editor for longer.
# Since there are virtual communities where women participate more then men - they should be emulated.
# Reduce non-editorial tasks using technological solutions and produce new ways to engage all level of users.
# Provide a mechanism to allow a domain experts contribution to get have greater weight.

===The editor's dilemma: Choosing content over community===
The editors dilemma embodies a conflict between two cross-cutting concerns, ''building content'' and ''building a community''. An informal statement is: "''in a wiki an editor has to choose between two evils: accepts new content of poor quality or reject the new editors who are the long-term future of the project''". The incentive for quality content is clear and easy to quantify but the cost of losing a potential editor's is marginalised since it is shared by the whole community.<ref group="note">Value in the form of new or improved articles is immediatly available for all. An increase in community size affects the community members indirectly though allocation of tasks and resources. New editors do not perform much communal tasks. Because each member only does as much work as they wish to they can personaly avoid the consequence of a reduced population. The brunt of the cost is paid by the community.</ref> 

The tragedy here is that for a wiki as pointed out above the significant reuirement to the sucess of a crowdsourcing wiki is to have to be a large and active community. Luckily it is possible to untangle these crosscutting concerns and get the best of both worlds - quality content and new editors. To go ahead would inflict costs , risks and some impediments to the community but these would be ofset by the greater benefit. A wiki requires editors for many labour intensive tasks and so this rejection is not without a moral hazzard.

However the editors dillema is a strategic problem occuring over a long term - to adress it the tactical version must be resolved, this is the reviewers dillema.

===The reviewer's dilemma:  Choosing well-formed edits over quality edits ===
The reviewer's dilemma stems from the asymmetry of information between editor and reviewer. On the one hand is domain knowledge and on the other is the knowledge of policy. It is easier to write about what one know about, more difficult to review what you don't. In the absense of a formal review protocol a mandatory step in print encyclopedias — self-appointed reviewers are faced a few choices:
#revert.
#check the fact.
#defer to another reviewer.
As the information assymetry increases the bias towards option one increases — reverting is a type of confirmation bias and considered by most reverters simlpy a conservative return to the status quo. To support such a choice any editorial defect is cited . Though in a xenophobic situations standards for rejection are very low and arbitrary - "gramatical errors" or "too many red links". The domain expert will only see the default in response to his co-operation. If it happens a few more times he will '''learn that his good faith work is not welcome, and tell his friends''' <ref>[http://vimeo.com/10699949 Gustave Flaubert Laughs at Wikipedia] '''Alan Shapiro'''  Critical Point of View Amsterdam</ref>.  The simple statement of this dilemma is at every edits machine and human reviewers must choose or reject domain based edits (and resolve human conflict) based only on their expertise of '''editorial policy'' '''. In almost all cases of good faith work — rejection of edits will occur due to editorial concerns and not due to domain problems. Wikipedia's growing popularity means its errors will often be repeated in print, and on-line.

===Incentives for cooperation by design===
[[File:Peace_dove.gif|150px|thumb|Kindness]]
As articles mature, contribution by domain experts becomes the bottleneck for progress on quality. Studies on the evolution of cooperative behaviour using iterated games show that '''information asymmetry''', '''anonymity''', and '''lack of communication''' are the worst conditions for cooperation to evolve. This theoretical warning is directly mirrored in conditions in the field - the requirements that editors work in the isolation on new articles in their private spaces. Solutions comes in three main forms:

#Improve '''incentive design''' - correction of the payoffs so that individuals see their benefits exceeds their costs in time, and work, coordination, and communication. So that good faith work remains a rational choice while still being able to address bad faith work effectively. The focus shifts to  of  cost of coordination, copy editing is put on the reviewer (for whom it is easy) and the cost of communication is shared.
##One example is to introduce a better<ref group="note">this protocol operates by 
*reducing information assymetry
*adjusting personal utility to favour cooperate rather then delete/defect</ref> review protocol for good faith work:
###3-5 [[w:wp:Responsible_tagging|inline tags]] must be introduced into the article by the reviewer.
###Notice allowing editors a week to respond by indicate willingness to correct the problems.
###Notice requesting the editor demonstrate his domain expertise by taking a quiz. (could be fun within the scope of user engagement).
###The editor may request a peer review for a second opinion.
###If the edit is not illegal the un reverted a copy or link to the edit will be given to the editor in his user space.
###Reviewers who do not comply will lose his privileges for 2 weeks. (cooling off).
##Let experts and new users operate  by paying forward<ref group="note" name="pay forward"/>
##''Increasing visibility'' - article deletion effectively hides the users edits while reversion hides the editorial problems. Both these allow a moral hazard of reducing potential penalties to the reviewer.
#improve '''game form'''
##''Reducing anonymity'' - reputations cannot develop in a truely fully anonymous environment.
###Simone will assign users under coaching a unique avatar and advize how to link it to their signature.
###Simone will allow users to use a facebook id to identify.
##Simone will bear the cost of coordiating ''repeated encounters'' within smaller groups of individuals. <ref group="note">Without research on the subject, my own informal tests indicate that coordination is the greatest stumbling block to cooperation. Followed by communication. </ref>
##''Increasing visibility'' - article deletion effectively hides the users edits while reversion hides the editorial problems. Both these allow a moral hazard of reducing potential penalties to the reviewer.
##Using a social network identity would be even better to avoid deception.
##''Repeated encounters'' between the same individuals are important to evolve from a reviewer's dilemma to a bargaining game.
#diversify '''the population ''' - an increase in circles of social and intellectual relatedness will improve capabilities of co-operation within each group.
##No more speedy deletion of for good faith work - speedy is unsuitable for good faith work. It send a bad messages to editors and reviewers.
##Restrict reviewing of non spam to categories which they have edited. This will coordinate reviewers and editors better and keep interactino within circles of greater relatednes.
##Invade the community with a new species of editor who can dominta though co-operation.
 
Some of these ideas would create a gap which is where Simone's comes into the picture. It will be possible for a reviewer to request it to intercede in such situations.
*Provide an example of software agent whose goal is positive social engineering rather than negative editorial automation.
*Not just increase the number and life span of editors but increase the domain expertise will reduce the high cost or reviewing edit.
*Reduce the need for subjective editorial decisions - which are usually handled badly<ref>[http://www.randomhouse.com/book/6795/super-crunchers-by-ian-ayres Super Crunchers] - '''Ian Ayres''' Random House</ref> by human expert.
*Create a minimal learning curve for acquiring editorial, social, technical and administrative skill.

I propose to provide positive alternatives to for instance templates for section and article under construction work easier use of problem tags, wider usage of incubation and requested article features. As these come to effect there would be far less need for a police state mentality and a return to the early meritocracy. I envision this happening socially in more and more places over time using technological means. This would make participating less stressful and more productive.

Certain of the early instigators of the more esoteric policies on Wikipedia have voiced regret once the protocols they innovated got wider and wider usage<ref group="note">Conflict of interest, Arbitration</ref>. Other policies<ref group="note"> No Original Research</ref> solve localized problems by imposing unnecessary criteria on the whole community.

===Wiki Evolution from Asocial to Eusocial===
[[file:Burne-Jones-le-Vampire.jpg|150px|thumb|''Biting'' newcomers]]
Wikipedians has been described as xenophobic in their attitude to new comers. Not surprisingly there Wikipedia jargon for this behavior is '''biting the newcomers'''. This roots of this asocial dynamic can be understood using simplified models like the reviewer's dilemma. This law of the jungle behaviour with its mad logic, brutish cunning and has evolved in countless encounters has. But this begs the question - could a new '' 'species' '' of Wikipedian invade and dominate the population? Considering that such invasion have been described by Sue Gardner in her address at Wikimania 2011.

To answer this question eloquently it is best to use a powerful social metaphor for the wiki and it is the beehive. Within the lens of beehive it becomes preeminently clear that a change of the asocial paradigm to the eusocial one is required. In the real world it means increasing user'''relatedness''' and changing the form of conflict.

In a hive bees use chemical signatures to differentiate hive members from foreign intruders. In the Wikipedia settings behavioural signatures are used. These '''behavioral signatures ''' have developed though  iterated ''editing games'' in which the dominating strategy is '''[[w:tit for tat|tit for tat]]'''<ref>The Evolution of Cooperation - ''Axelrod, Robert''  Basic Books, 1984 0-465-02121-2</ref>. To be recognised as community member, a newcomer must make a protracted commitment of time and effort.<ref group="note">One has to develop a user page, learn to respond to policy challenges laced with Jargon in like kind and develop an edit count.</ref>

[[File:Bee waggle dance.png|thumb|150px|Eusocial Communication]]
Like a real hive, a wiki requires constant policing to ensure that drones are not pursuing their own agenda at the cost of precious shared resources. In the hive these freeloaders once identified are restrained or destroyed and their progeny eaten. In the wiki ''sock puppets'' are unmasked using a ''check user'' and are then ''banned'' and their works systematically purged. The hive must deal with parasites (COI) and invaders (Paid Editors), and a degree of anarchy (ArbCom). Each generation is very closely related and the oldest, most senior maintains the status quo.

So one may argue that this is the natural order of things. New editors should be  viewed with great suspicion -- Are they foreign agents? (Paid Editors) trying to subvert, or perhaps self-serving parasites? (COI). The default solution is one of extreme hazing - the old generations retaliate against individuals who might be suspected of hurting the group, until they learn to walk the line or depart. And the few who graduate will maintain the asocial dynamic. This is a form of '''total war''' — it is extremely expensive to wage and is uncommon in nature.<ref name="logic">The Logic of Animal Conflict - ''J. Maynard Smith, G.R. Price'' Nature 246 November 1973</ref>

However such solutions are unnecessary and no longer acceptable — it is time to groom another breed of Wikipedians who default to settling differences sporting within a "limited war" paradigm rather than the picric outcome of todays "total war".

==Understanding Media in MediaWiki==
In the early dats of Wikipedia it was a simple technology, it lacked experienced editors or an editorial policy. It had no established hierarchies of influence or personal stakes. All these things are signs of progress, yet they present impediments to contribution. Consequently ''the threshold cost of contribution'' was low<ref group="note">where by cost I mean the amount of work time and learning required to successfully contribute an article</ref>. However as the project matured it was modified piecemeal to address immediate concerns - without a guiding hand the changes have often been regreted in retorspect.

Marshal McLuhan in his seminal work [[w:Understanding Media|understanding media]]<ref>[[w:Understanding Media|Understanding Media: The Extensions of Man]] — '''[[w:Marshall McLuhan|Marshall McLuhan]]''' McGraw Hill, NY, 1964;  ISBN 1-58423-073-8.</ref> and later text innovated the tetrad as a tool for analyzing the impact of new media. By provide such an analysis for Simone, it will become possible to better understand the grasp the scope of its impact. Next Simone wil be compared with some other media being utilised and developed today.

===Tetradic Analysis===
[[File:SimoneTetrad.svg|left|link=w:Tetrad of media effects|thumb|250px|Simone Tetrad]]
*Enhances - review of good faith edits for: article quality, editorial standards; user recruitment, and coaching; group collaboration under peer review.
*Reverses - editorial complexity, community reduction, coordination cost, recruitment of new editors, policy complexity.
*Retrieves - organic editing<ref name="Bruning"/> , pay it forward editing<ref group="note" name="pay forward">protected editing under a view that the editorial problems will be corrected by  editor as they gains experience. Paying forward means that the cost of the learning curve will be paid later to future editors. Pay it back means making they new editors pay for thier mistakes like reviewers had to</ref> shorter learning curve. reduced cost of contribution.
*Makes Obsolete - early seletion of new articles; subjective human assessment of editorial level for articles.

===Simone and Normal Bots===
Simone reuses existing bot framworks but will be different in two senses:
*It is aimed at altering user's editing behaviour and not to fix or revert thier edits.
*It is planned to preempt future problems rather than react to old ones.
*It is planned to rival human asessment of editorial issues.
===Simone and Abuse Filter===
Simone will also build on abuse filter but different in three senses:
*Abuse filter provides negative options to a few admins. Simone chooses from many positive choices for many many editors.
*It will provide many options in the cooperate category rather than provide one options in the default category.
*Abuse filter is inteded for abuse, but Simone is planned to cover a grey area of good faith work that borders on "abuse".

===Simone and The Visual Editor===
On the other hand MediaWiki developers are working to reduce the complexity of editing via the '''Visual Editor'''. While a visual editor will reduce the technical level of editing, but does not address any of the social issues discussesd above. This solution may have an unintended result of increasing new user hazing. This could happen because new ediors will be harder to assess on using technical signitures and will be judged more harshly on policy and stlye — which will appear to be more arbitrary.

===Case Study: The Decline and Loss of Organic Editing===
No other loss of functionality present a greater stumbling block than loss of'''Organic Editing''' <ref name="Bruning">[[w:User:Kim_Bruning/Lost_functionalities|Lost Functionalities]]</ref><ref>[http://www.youtube.com/watch?v=gagO-of1xIw Wikipedia reconsidered - Everything You Know About Wikipedia Is Wrong] by Mindspillage, Fluffernutter, Ironholds, Kim Bruning, James F. and Tom Morris. Wikimania 2011 Haifa</ref>. Organic editing is the ability to create sentence by sentence a new article in main space. Organic also refers to natural growth of the articles. Yet today anyone bold enough to attempt to do this will be censured by patrollers and adminsitratos at every save and informed that her work is not welcome. It will be nominated for speedy deleation on grounds of context, content or non notability. However notability is not an objective assesment, it is simply the harshest dictum to defend for a new editors. When the one editor complains they will be informed that thier article may be exiled to the user space. It could apply for a visa when it reaches maturity.
Unfortunately userspace is not a place for natural collaboration. Your work get no exposure and other new collaborators are often reverted there.

So why was organic editing lost? 
As predicted by <ref>E. Goldman. Wikipedia’s labor squeeze and its consequences. Journal of Telecommunications and High Technology Law, 8, 2009.</ref>
The number of edits in a day depends on existing article counts theb since  or as the number of articles grows much faster then the number of patrolers in a given week. And this leaves out the growing complexity introduced by a rise in quality
*"Don't make many small edit - save them as one large edit".<ref group="note">it is often explained as straining the MediaWiki software. But Media Wiki only store a limited history in the database - so this is in realty only are patrolling problem</ref>
*There are less red links being added by new editors
*I reverted your good faith edit because it had many red links.

=== Corrective Action===
Whenever a page is edited it will be analysed and stored into the index it with a list of current issues. Searching the index will allow '''Simone''' a greater level of intelligence. Running ''strategic'' queries it will enable planned proactive action rather than a reactive interventions. 

Simone's explainable activities would be driven by different causes. However strategies will assist develop full and order behaviour and where multiple options exist it will allow choosing the option which  has the highest priority.

# ''Simone'' objectively advises ''new'' users how they could best contribute. New users will, once again contributing with at a near zero cost - on the assumption of good faith work and that they would pay their ''dues'' once they have ascended up the learning curve. They would be more rapidly progress in skill and join rank as the next generation of our community - a generation that is less obsessed with unwritten rules, one which would not require ''wiki-lawyers''. 
# Expand the program to bring articles systematically to featured levels by providing editors with expert advice.
# As the ''Simone'' matures it could be directed <ref group="note">using an api; a tool in the toolbox or talk page templates</ref> by users to instruct users of policy infringement and by  project to direct thier members to collaborate on hot spots.
# To reach a far greater audience and higher levels of engagement a reward based game mode would be developed.

==Rationale==
The project is made of software elements unified into a bot I will refer to as '''Simone'''. 
It will either work as an external user or will be integrated into the editor. 
Work will begin with '''sensor modules''' which can assess a given revision's merit in various dimensions. These include a comparison with '''Gold Stadard''' and a '''Minimum standard'''. 
:technically these can range from simple rules to search wisdom of crowds ranking algorithms to more traditional  statistical discriminative models.
* Next it will select the highest priority issue from its queue.
* The bot will summarize the top issue in a one line human readable report. 
* Next it will offer advice on how to rectify the problem.
* Finally it will offer some alternatives and ask for feedback.

==Illustration: advice sequence for new user==
The following figures illustrate a sequence of advice notices that would be generated by Simone as it coaches Hildur, a new user from Iceland working on her first article on Kafka's story The Trial. The UI element shown is a talk page/user talk page based system that would be inserted by Simone.

new users coaching can start in several ways
#proactive detection of an anonymous editor / new article by a registered user.
#reactive intervention based on a '''biting incident''' sensor.
#invocation from Afd - i.e. a new '''suspended delete''' sentence. Either correct the quality issue within a week or be deleted. Simone will be used to carry out and enforce the probation - if successful the probation will end by removing the probation notice. If no action is performed the article will be deleted/userified.
#invocation by Huggle, Twinkle or a similar tool with Simone integration.

===Step 1 - Join Wikipedia and apply for coaching to save your work===
{{User:OrenBochman/UI
|mode=intro
|Goal=Bring the article to '''Threshold Level'''
|FollowUpMsg=Hello stranger, thanks for helping out with this article. We always need more helpers
|Status=The last submission has some editorial problems and since you are unregistered it is likely to deleted soon.
|StatusLong=If you are planing to further improve this entry you can save your work by improving it. If you are interested in advice on how to improve it to the threshold level of inclusion choose one of the options bellow.
|BadContext=1
|ArticleName=[[w:The Trial|The Trial]]
|Mission=Sign up for coaching.
|Option01=Register for coaching 
|Option02=To mark the page as "under construction" for a day
|Option03=I want to view a video on the policy and the advantages of coaching.
|Option04=Please don't bother me, I know what I'm doing.
|Option05=I've done all that I can.
}}

====Logs====
Simone believes that ''one should always be testing''. She is careful to make her logs useful for analysis using R as well as for keeping track of action that needs a followup. On creating the above message Simone will append to its log:
*<nowiki><NewUserEvent></nowiki>
::<nowiki><Source>1.234.44.215</Source></nowiki>
::<nowiki><revId/>486369447<revId/></nowiki>
::<nowiki><Sensor>unregistered edit</Sensor></nowiki>
::<nowiki><Opt id="100"/><Opt id="101"/><Opt id="105"/> <Opt id="202"/><Opt id="230"/></nowiki>
::<nowiki><Opt id="G20"/></nowiki>
:<nowiki><NewUserEvent/></nowiki>
This log includes:
* the user/i.p.
* a time stamp. (not shown)
* the revisionID which comes from https://en.wikipedia.org/w/api.php?action=query&prop=revisions&titles=The_Trial&format=xml
* the options used - for doing optimization of advice using the R stats package.
* the last option is a User Interface option for doing simple A/B testing R stats package to.
====Notes====
*Points out that there is a problem with the edit.
*Advises the use to join sign the user.
*Can allow measurement of interest in anonymous editing.
*Option 2 will place an organic editing template in the page with one day expiry. It will also place a notice on the talk page.
*This user element would be further customised for capturing new users based on stylometric analysis of their input. To consider gender and nationality.
*Most of the options are just hooks to get the user to sign up. For example I've done all that I can will inform her that she can do much more - Wikipedia has mistakes she can correct and so on.
*Depending on integration with MediaWiki
**If in the submission loop the advice will expire immediately if ignored.
**If it is just put into the IP user page - unregistered advice expires at is never clicked on would expire after 3 days and marked as a '''bounce'''.

===Step 2 — Learn about context ===
{{User:OrenBochman/UI
|Goal=Bring ''My First Article'' to '''Threshold Level'''
|FollowUpMsg= Hildur, thanks for enrolling for wiki coaching.
|Status=Your article is so short that it could be marked for speedy deletion under article [[A1]] - which means that it does not provide enough context to justify its inclusion in.
|StatusLong=Your article: on [[The Trial]] has been temporarily  protected from speedy deletion. It is a good start – but 50 words is considered too short for a Wikipedia article. Once you add at least another 100 words, it would qualify this article as a ''[[stub]]'' article.
|Mission=fix context
|ArticleName=[[w:The Trial|The Trial]]
|BadContext=1
|Option01=you can expand the article - don't forget to include details or technical terminology.
|Option02=you can let add a template to protect it while you work
|Option03=you can move your article to your user space
|Option04=you can see an ''explanation'' video tutorial about '''context'''
|Option05=to dismiss this matter for now.
}}

*There are many ways to improve context.
**Adding info boxes
**categorising
**Writing sufficient details.
*Simone will be basing it '''context score''' on minimally unique word n-grams. This will quickly indicate if  the article if covering something new or not. Even if there are no unique terms it will also use word vector cosines to identify the closest articles.

===Step 3 — Learn To Wikify Text ===
{{User:OrenBochman/UI
|Goal=Bring ''My First Article'' to '''Threshold Level'''
|BadContent=1
|FollowUpMsg=You are making good progress, the edit brings the content score to 4/100. But you article is an [[orphan]].
|Status=speedy deletion A1 - your stub lacks context.
|StatusLong=To fix this all you need to do is add some internal links. We call this processes wikifying.
|Mission=[[Wikify]] some text
|ArticleName=[[w:The Trial|The Trial]]
|Option01=to wikify a word use braces like this <nowiki>[[word]]</nowiki>.
|Option02=for a ''how to'' video tutorial about why it's important to wikify articles.
|Option03=to get a recommendation on what to wikify [[link assessment tool|click here]].
|Option04=to dismiss this matter for now.
}}

====Notes====
*Syntax should be taught from simple to the more complex.
*If difficulties arise for engaging users with pedagogically significant advice or goals Simone could review a user's edit history to gauge the existing skill set.

===Step 4 — Learn about Notability===
{{User:OrenBochman/UI
|Goal=Bring ''My First Article'' to '''Threshold Level'''
|FollowUpMsg=  great your content is now scoring 7/100
|Status=Your article has low notability and may be marked for speedy deletion under article [[A7]] or [[A9]] - which means that it does not indicate the importance of the subject to justify its inclusion.
|StatusLong=Your article: on [[The Trail]] has no references to external sources - add references and wikifi the text so you can better demonstrate what other articles your work touches on – these will quickly boost your score by increasing the visibility 
 of your articles notability.
|Mission=demostrate the article importance for inclusion in an encyclopedia.
|BadNotability=1
|Option01=you can expand the article
|Option02=you research and insert references from print sources.
|Option03=you research and insert references from online sources.
|Option04=you can watch a ''tutorial'' about notability and reliable source.
|Option05=to dismiss this matter for now.
}}

note: at this stage we would like to teach the user an editorial skill set as well as a social one.
*the Editorial requirement that she must show that the article is important enough to be included in wikipedia. This will require more work, the less notable the article is
# don't include it to better answer the issues of notability

However there are probably more than 5 options to fulfil this mission. If we are able to collect 20 options and track each options according to its success  — Simone can optimise this advice. I.e. She will pick 5 options at random out of the 20,  respecting each option's probability of success.

===Step 5 — Learn to add categories===
{{User:OrenBochman/UI
|Goal=Bring ''My First Article'' to '''Threshold Level'''
|FollowUpMsg=  great your content is now scoring 7/100
|Status=Your article has low notability and may be marked for speedy deletion under article [[A7]] or [[A9]] - which means that it does not indicate the importance of the subject to justify its inclusion.
|StatusLong=Your article: on [[The Trail]] has no references to external sources - add references and wikifi the text so you can better demonstrate what other articles your work touches on – these will quickly boost your score by increasing the visibility 
 of your articles notability.
|Mission=demostrate the article importance for inclusion in an encyclopedia.
|BadNotability=1
|Option01=you can use hot cat 
|Option02=you can create a new category.
|Option03=you can use the category recommendation too
|Option04=you can watch a ''tutorial'' about categories
|Option05=you can to dismiss this matter for now.
}}

* option number four would be supported by an automatic categorising tool

===Step 6===
Once threshold is reached the article will be safe to remain in main space. The user will have a positive payoff for her work in the form of a permanent contibution to wikipedia. But there is still much more that the user can still learn. By completing the new user's coaching she be able to create new articles without significant risks of deletion. Thus Simone will continue to instruct the user and cover more points such as
#how to use her talk page.
#how to use the user space sand box.
#how to customise their user space for eusocial communication.
#how to view requests for creation.
#how to interact with reviewers using wiki love.
#the new article creation wizard.
#how to get a human coach
#how to join project.
#how to interact with templates
#how to use Template:Inuse-section
#how to join project.
#how to improve their user page.
#how to make tables
#how to watch pages
#how to insert images
#how to delete old/broken references

the curriculum will include trails based on
*mediawiki software trail
**e.g.how to use preferences to add some gadgets
**how to master the rest of the wiki syntax
*wiki's policy trail
**core editorial policies
**recomended best practices (Simone good wikipedian training)
**netiquete - socially accepted behaviour

==Advice Strategies for Other Use Cases ==

Advising only on new users might seem to be a more focused approach but it is defficent in servral areas 
* it will not mobilize sufficient human resources to reverse the underlying reasons for editor loss
* it will be restricted to situations where machine judgment is better than human Judgment.
*it will miss on some of the greatest benefits that Simone can provide - discounting coordination and communication costs.
*it will mean that experienced users will not be familier with the possiblity of invoking Simone to place new articles and users under probation.

I will present a some strategies for executive coaching, article improvement. Since there exist an very low activity admin coaching projects this is also a candiadte for intermidiate level of editors. It can stress finer points of policy.

===Retention and Engagement for Seasoned Users===
Loss of seasoned users is the flip side of new user recruitment. Experienced users may run out of information to contribute or they may end up being ''promoted'' into role that are less satisfying and experience burn out. 
Recent studies now provide deep knowledge of this problem. Studies show that under the right conditions these users can be retained longer within the community.

By adding a '''burn out sensor''' users who fall out of the fold can be contacted and offered to received '''executive editor coaching''' acknowledging, perhaps for the first time, their superior status. There is a cost of coordination of 

Simone is goal and mission based. She will, based on feedback provide them with tasks. These will involve work that requires human touch and is therefore harder for Simone to do. 
* Peer review in their area of editing.
* Peer review in their area's of expertise.
* Crowdsourcing of articles for importance and quality<ref group="note">autamated quality assessment can be done faster and at a higher level of objectivity by machine. Importance is contextual and thus less suited for machine assessment. But capturing both it is possible to make use cross validation in a repeated '''article rating game''' in which editors get a reputation by self select though accurate ranking in a knowledge domain. Accordingly their votes on importance are boosted or discounted accordingly.</ref>. 
* AFC in their areas of expertise.
* Quality improvement missions.
* Translation missions based on criteria in the next section.
* Missions will be varied, including technical task and social tasks.

{{User:OrenBochman/UI
|Goal=join the ''MIB''
|mode=executive
|ArticleName=outstanding achievments
|FollowUpMsg= Do you have what it taks?
|Status=Your many contribution have singled you out as an outstanding community member.
|StatusLong= Why not take it to the next level? If you join the executive coaching you will be able to fulfill your potential and join the editor elite. I will give you some high impact missions.
|Mission=Score on five introductory missions to unlock your Executive Coaching Badge
|mode=executive
|Option01=tagging a mentored users' first article with inline problem tags.
|Option02=Correction of references to web sites that are broken.
|Option03=Translation missions based on criteria in the next section.
|Option04=Learn how to join executive coaching.
|Option05=Help being an article to featured level
}}

*executive coaching will complement the new user coaching program in places where Simone is unable to assist.
*report cards in executive coaching will be used to give a the tasks a game-like feel.

===Quality improvement===
Article quality improvement is the primary driver of Simone's behaviour. 
However if user have completed new user coaching they can either select another module or stay with the quality improvement module which will give feedback on editing progress.

Simone will still detect problems and direct users to fix them.
* Outstanding inline tags in the users previously edited articles.

{{User:OrenBochman/UI
|Goal=Quality Improvement
|mode=quality
|ArticleName=Quality improvement
|FollowUpMsg= Do you have what it takes?
|Status=It's time to do some work on improving article quality!
|StatusLong= Quality improvment is not only one of this wiki's strategic goal - it is a part of how community members view your contribution. If you help bring an article to '''good''' or '''featured''' will count in your favour in many situations.
|Mission=Select an article to improve and upgrade it from C to B level to receive a QI Badge.
|Option01=Place a section for improvment in a C level article.
|Option02=Introduce 3 new refrences in the article.
|Option03=Look at suggestions for article to improve.
|Option04=Learn more about Quality Improvement by watching a video.
|Option05=Help being an article to featured level
}}

*Quality improvment will be able to select Users who can translate sections from another language where the other language has a better article.
*report cards in Quality Improvment will also be used to give a the tasks a game-like feel. This include offering badges and barnstars for sucessful missions.

===Simone's UI===

Simone's U.I. is designed to expedite coaching activities:
*Simone develops a one on one relationship with the student.
*It monitor's the coachee's performance using the dash board
*It defines a specifc goal to work on.
*It breaks down the goal into missions which can be acomplished in a single editing session
*It provides feedback on previous activity.
*It engages the users by providing intervention feedback in the form of advice.

The UI used in the Ilustrations would be
*A/B optimized for greater usability.
*Streamlined to make better use of space.
*Adding quizz widgets to get a tighter feedback loops with even more engagement.
*Adding media widgets for audio/video/text
*Adding a IM chat widget for interaction with human coaches.

Floating UI
*Suplamented with a gadget to place the UI in a coaching tab to make access to coaching easier and more personal.
*Later a more dynamic Floathig UI may be developedabove and to the side of the editer/browse window.
**This would allow Simone to inspect edits in real time as well as show videos to user's while they edit.




More so when it comes to web-based applications and even more so in the context of extending MediaWiki. MediaWiki is years behind normative content management systems, which provide simple UI widgets and an easy extension mechanism. The WMF claims to be open source, and constantly solicits help from volunteer developers. However in recent years the practice has become increasingly less open, there is little trust in community submissions and therefore little help from volunteer developers. Recent discussion on wiki-techL has highlighted that it can take over two years to review 3rd party plugins -- long enough for them to become obsolete.

I have made some paper UI design but the Template above represent a second iteration of these ideas. While they could be vastly improved and streamlined this should be done within the scope of the project itself. As mentioned above I would propose short development cycles with testing using real against real users and getting  feed back from active coaches.

Like Simone might say there are 4 options regarding integration:
#1 Using a Message based U.I. and no integration.
#2 Minor MW and UI integration.
#* An event hook to allow providing advice during a Save page event.
#3 Major MW integration
#*Requires the '''20% time''' of an experienced MW developer.
:Provide floating UI elements within the edit page.
:Provide Coaching NAmeSpace or Tab
#4 Full MW integration, highly customised A/B tested UI.
:Integration into the core - requires 2 developers
:*A MediWiki developer.
:*An Information specialist.

It may be expidient to intergrate into Simone a capabilty for collection information using forms. This is part of a use case for collecting feedback in surveys. Another use case would be adminstering tests to verify a user's progress before a graduation event. If this is identified as important such a component will be intergrated into the UI.

===Simone's Invocation API===
Simone will also have an ''refferal mode''. It would be possible to introduce a referral template into the page by a reviewer to invoke Simone's presence and ask her to present certain advice to an individual at a page. The reational for this is to allow quick integration of Simone into tools like Huggle, Twinkle etc. This will in effect introduce positive alternative into these tools. This will have the effect of  changing the standard tactics from ''total war'' to  ''limited war'' in the good faith cases.

=Scaling=

The project can scale in four dimensions:

==Technological scaling - data based sensors==
'''Simone's''' information architecture is designed to grow organically with the MediaWiki community it advises. Technological scaling of this sort involves an increasing discrimination capability (more points in space); and using feedback to fine tune a sensor's activation threshold (e.g. gaugeing the notability of a Soap opera episode article is not the same task as an article on a war.

This type of scaling requires creating sensors that become more refined as the wiki grows. 

Simone's response to the above strategy is to provide smarter data driven sensors. The data for these will come from sampling the wiki so they will be normative rather than proscribing what may become unrealistic utopian criteria. By considering more data and by clustering this data to more levels it is possible for the same basic sensor to track changing dynamics as wikis evolve over time. Both positive and negative feedback mechanism can then be used to adjust the behaviour signatures which it detects .

The ''sensor developer's''  goal is [[w:Pareto efficiency|Pareto efficiency]]. With a robust detection of Context, Content, Notability, References , Style and Social behavioral signatures in user edit and talk '''Simone''' will to preempting 20% of the most socially damaging reversion/deletion events which should dramatically reduce community oversight by about 70-80%.

The handling of remaining incidents would require  human intervention and provide greater engagement. If not then once such a milestone is achieved the method can be refined by adding sensors and refining existing sensors to track the top 20% of remaining issues. This is likely since deeper underlying problems (such as cultural and gender gap issues) may be obscured by more visible issues.

Sensors could also be added to address additional (non editorial) concerns so long as the phenomena is discernible using techniques of discourse analysis. This simplest should involve retention of experienced editors. 

In cases where a rules and  pattern are too weak to detect issues (Such as neutrality) it will be necessary to produce a collection of good/bad edits and train models to discriminate their issues. Building some "Gold standards" will be one of the project's mile stone since it will involve community expertise and feedback.

Also if such issues become a priority the community will be requested to assist by tagging problematic revisions. Though this can be accelerated by earmarking a budget to outsource corpus development by outsourcing the work online using Amazon mechanical Turk at a token cost per tagged revision. This approach has been reported to be successful in developing an anti spam corpus.

Some initial sensors that would be developed:
*'''biting''' sensor - incidents of new user biting. 
**bite scale from 1-10.
**biter's leader board to mitigate biting and identify delinquent biters.
*new user sensor
*new editor sensor
*new article sensor
*afd sensor
*speedy sensor
*article information sensor
**determines quantitive the amount of information in this article.

==Community Scaling – Advice==
Sensing problems is not enough – '''Simone''' needs to offer useful advice. Since this will be the most visible aspect of the project it needs to be developed with great sensitivity to users and community acceptance. Advice defaults will provided but this will only be a starting point. Again community input will be requested on this subject.
Suppose the community solicits our projects with 3 advice notices for a certain issue - say lack of Citations. '''Simone''' would systematical test these messages and score them based on their success (i.e. check how each advice correlates with the sensor score for Citations in following edits.)
Tracking advice - and correlating using R in a DB will be another Project milestone.

{| width="100%" border="0" style="{{round corners}}; border: 2px solid #3C78B5; padding: 5px;"
|+Sample Advice for Context Issue
|-
!Probability
!Advice
|-
|0.5     || Write another 120 words
|-
|0.25    || Add an info box template
|-
|0.0625  || Add a category.
|-
|0.0625  || Add the article skeleton using section titles.
|-
|0.0625  || Add categories.
|-
|0.03125 || Add project description in talk page.
|-
|0.03125 || Add project stub.
|-
|0.03125 || Add an interview link to a version in another language.
|}

*Advice is offered in the scope of missions which are grouped under asingle goal.
*Goals are collected in strategies.

{| width="100%" border="0" style="{{round corners}}; border: 2px solid #3C78B5; padding: 5px;"
|+Sample Strategy for New Users
|-
!Sensor Definition
!Goal
!Mission
|-
|S1     || Threshold || Provide Context
|-
|S2     || Threshold || Provide Content
|-
|S3     || Stub|| Provide Content
|-
|S4     || Stub || Notability
|-
|S5     || Stub|| Add project description in talk page.
|-
|}

==Deployment and Scaling on Language==
To speed '''Simone's'''' community acceptance, default advice will be provided to the initial project by the developer. This is perhaps the most labour intensive task -- primarily because '''Simone''' would like to support new user in their first thousand edits to bring them into relatedness with other wikipedians. To provide meaningful and engaging advice different editing issues would have to be explained in text and video.

Once '''Simone''' has stabilised through three or four initial development cycles it will be ready for its initial deployment.
However Simone won't need all this media in order to go online. IT will co-operate with talented coaches to develop more advice for new situation. It will be done by issuing a "Request for Advice" - a crowd sourcing request to the community. This is a strategy to increase the number of stake holders involved in the project and who wish to see it succeed. The best scoring advice will periodically supplant the default advice and undergo translation.
For example: 
*a sandbox based tutorial
*a in-depth study on making references

Since the initial default advice will be in English, these will be Internationalized by translating using the resources of 

TranslateWiki. 
This would involve:
*Making its sensors cross lingual.
*Translating its templates
*Translating its advice.

I have consulted with the WMF developer of TranslateWiki which handles the Internationalization and localization of MediaWiki and he recommended for the translation of the templates:
*https://www.mediawiki.org/wiki/Help:Extension:Translate
*https://www.mediawiki.org/wiki/Help:Extension:Translate/Page_translation_example

==Scaling on Projects==
For example Wiktionary has quite different editorial guide lines from Wikipedia as do wiki source and Wikibooks. Scaling '''Simone''' for work on new projects requires:

# determining a socio-editorial policy using the deletion/reversion history and deriving concrete list of violation.
# sending out a ''Request For Advice'' to the project's community.
# beta testing the behaviour on a control group.

Thus Each project could adapt '''Simone''' to its needs.

=Metrics= 
As already explained metrics are  at the very heart of this projects - we call the sensors.

*Simone will generate detailed logs of its activity.
*Reporting will be generated from the logs.
*Simone will be configred initialy to have a control group which will not get the benefits of coaching.

Simone will provide periodic reports with summary statistics on
# incident (selected sensors correlating to high level events).
## new user biting
## speedy nominations
## article improvment from C to B and other levels
## user projected activity based on recent edit history<ref group="note">inactivity would be statistical estimate of the users expected future contribution base on a poisson distribution with the user's historical </ref>
# incident reponse
## new user biting interventions.
## speedy interventions.
## user coaching enrolement.
## editor's leaving coaching.
# long term change.
## new editor acclimatization to the community.
## editor under coaching mass action. The overall edit count * positive change due to these as seen by sensors.
## morbidity rates estimates for editors under coaching and the control group.<ref group="note">since Simone is expected to effect the community as a whole change in the conrol group may be significant</ref>

=Long term - Sustainability=

No claims about the long-term of the project only suggestions. I hope it is taken over by the WMF development team and more tightly integrated into MediaWiki whose shortcomings it addresses. However, since it is not the traditional model for robot based tools, the more realistic community process will be explained in the different roles of different stake holder involved.

==Role of Developers==
Today Bots developers are the most influential members of the community. Within the limited scope of thier projects they provide more agile solutions than the WMF. They are also better aligned to the community's needs because not only must they get approval to run their bots from committee, they must then respond to issues raised by users.

'''Simone''' will be created in short development cycles as a MediaWiki based project - using git for source control, labs for test deployment, Bugzilla for bugs, code review for reviewing external patches once these are provided, translate wiki for internationalization.

Bots are mostly constructed from a few exiting frameworks and thus there exist a possibility of a high mobility within the community if a better technology is introduced. It some areas, such as ranking articles for completion and importance, Simone would provide a significant technological edge. This suggests that there is potential for poolling resources with exiting robot operators.

She will be deployed and start collecting data early on and generating reports. Her nature is planed to be extension. This means that the community will be given freedom to contribute to this project. 

To promote Simone amongst developers:
#Her code will be available as open source.
#Data analysis scripts will be provided.
#Documentation will be provided on installation.
#Evangelism at at Wikimedia, and hackathons and other conferences would boost this community.
#After the end of the fellowship the volanteers will be found to take over and maintain the project.

==Role of Administrators==
The stake of administrators in running Simone - is that she will oversee editing in a the grey area between spam and good edits. This will provide a massive reduction in patrolling. 

After undergoing the robot approval process Simone will be operated on Wikipedia. As usage will increase responsibility to operate it will be transferred to qualified Administrators.

Simone would be primarily controled by a configuration files. If required a web-based interface could be added too based on edit filter. Administrator will:

*fine tuning its sensors and their thresholds.
*adjusting corpuses for normative standards
*add remove or change strategies
*add remove or change defining goals, mission, and advice hierarchies.

==Role of Policy Makers==

There is a number of suggestions proposed to policy in this proposal. These changes will be performed informaly by Simone. As well tempered users invade the community these policies will gain weight and gradualy replace the existing practices.

A key assumption is that as Simone influence will be felt by policy makers. In this view when they decide to repel anti-social policies Simone will be reconfigured to fill in the resulting vacuum. This will allow to reform current policy to a shorter and clearer set of guidelines — more suitable for a larger more diverse society of contributors.

==Role of Human Coaches & Domain Experts==

Human Coaches will be asked to provide expert advice on coaching and help for debugging based on the open source doctrine of ''many eyes find bugs faster''. Thiers stake in '''Simone''' is that it will make coaching more fun and productive. For humans the tasks will be less technological and more of a social activity. 

Using a project page they will be asked to provide original and alternative texts for ''advice''. They will be invited to suggest ''new missions'', ''goals'', ''strategies'' and ''sensors for current'' development cycles. Once these suggestions are provided they would be added into Simone as part of the next development cycle

These will be use by developers and administrators.

Domain experts can be paired with more seasoned editors to work as pairs. One contributing domain knowledge, the other providing wiki know how. This is another situation of changing the '''game''' from zero-sum to win-win rule.

==Role of Projects within a MediaWiki instance==
Specific project in Wikipedia such as biography of living persons have their own needs. Simone will be able to address these projects as a group, recommend new users to join the projects teach people the rules required in these and arrange collaboration on top priority articles. This coordination will allow a participating project to get boost in personal and to focus them on work in teams on the articles of greatest importance. This is something of a lost capability now that matching articles to individuals. Simone's user Soliciting module could be fine tuned from the start to training users in their areas of expertise in a socially acceptable way.

==Role of The Researchers==
Simone will provide reports on its activities. Its high level strategies would have better results of based around research carried by experts in this field. By providing better answers to the key theoretical questions it will be possible to define better goals and sensors and preemption strategies for Simone to pursue.

For example if there is more knowledge on conflict situation Simone could be instructed to preempt conflict <ref group="note">I omitted the details in deference to [http://meta.wikimedia.org/wiki/Wikimedia_Fellowships/Proposals/Dispute_resolution Dispute resolution]</ref>; existing user retention.

However, if researcher were to discover a better model of the editing and review game or a more suitable incentive paradigm these would be used by Simone to coach better editors for tommorow.<ref group="note">a list of same game theoretic motivated questions on wiki editing behaviour together with raw Simon data would be provided if the project is funded.</ref>
*life span in days/edits of editor generations.
*percentage of new articles deleted under each SD clauses.
*percentage of new articles deleted under other clauses.
*degree of communication before reversion event.
*time from first edit to becoming an administrator.
*persistance of unregistered editors

==Community Role==
The most important role for Simone is that of community interaction. As users work with her she will become wiser and able to give better advice. Maintenance of advice as Wikis' policy changes is one major role of the community.

==Footnotes==
{{reflist|group=note}}

==References==
{{reflist}}

==Submitted by==
Oren ([[User talk:OrenBochman|talk]]) 13:43, 3 March 2012 (UTC)

=Endorsements=
<!--Do not remove this text. It is for the benefit of those who endorse the project idea.-->
<blockquote>This section is for endorsements by Wikimedia community volunteers. Please note that this is ''not a debate, vote, or poll'', but is rather a space for volunteers to describe in detail why they think a project idea is of value. If you have concerns or questions rather than an endorsement to make, please use the idea Talk page. Endorsements by volunteers willing to work in collaboration with a fellowship recipient on a project are highly encouraged.</blockquote>

:{{support}}, I believe implementing this idea is very helpful for giving a smooth start to new wiki editors, and for saving expert editors' time spent on coaching new editors. --[[User:Ciphers|Ciphers]] ([[User talk:Ciphers|talk]]) 05:12, 21 March 2012 (UTC)

[[Category:Fellowship project ideas|A Digital Wiki Coach To Provide Focused Advice]]