Research talk:Automated classification of article importance

Importance of Wikidata Items

It seems like the proposed notion of importance of articles to a language version of Wikipedia is restricted to articles that already exist in that language. But for smaller Wikipedias, there may be many important articles that have not yet been created. It would be really valuable to have a notion of importance for any entity in Wikidata that has an article in at least one language. In particular, it would be great to have a global, language independent list of Wikidata items ranked by importance as well as a separate importance ranking for each language. These rankings could be used to not only prioritize work on existing articles, but also prioritize work on creating new articles and filling knowledge gaps. Ewulczyn (WMF) (talk) 00:53, 2 February 2017 (UTC)

Hi Ewulczyn (WMF), thank you for the insightful comments! You are right that the current proposal aims to determine importance of already existing articles. I agree that being able to determine importance for articles that do not exist, or for any Wikidata entity for that matter, would be valuable. Then some of the other notions of importance could potentially be a filter on top of that calculation (e.g. "any Wikidata entity with >= 1 article in >= 1 language"). However, I do not think we know much about how importance works across different scopes. Some of it is found in the discussions around List of articles every Wikipedia should have, where contributors argue about whether the list reflects a global perspective. There are also some research papers that look into cultural differences in importance (e.g. Research:Newsletter/2014/June#"Interactions of cultures and top people of Wikipedia from ranking of 24 language editions" reviews a preprint of a PLOS ONE paper that does this).

In summary, I am wondering if this suggests three potential research topics:

1. Importance by scope
   • Are there meaningful differences in importance depending on the scope? In other words, if we ask a set of WikiProject members, will they agree with importance ratings that were gathered using an algorithm based on Wikidata? What are their reasons for agreeing/disagreeing?
2. Importance in the context of Wikidata
   • How can we determine the importance of entities in Wikidata?
   • How does Wikidata importance relate to Wikipedia article importance?
3. Recommendations for article creation
   • This should specifically target smaller Wikipedias. In other words, we are perhaps interested in determining some sort of base set of articles. This might turn out to be List of articles every Wikipedia should have, or it might turn out to be something different.
   • A key element would be to study what happens when local and global scope collide. In our WikiSym 2012 paper we did a rudimentary investigation of articles in a single language and found that they had a limited scope. Based on the PLOS ONE paper mentioned above, different language editions have slightly different focuses. It would therefore most likely be useful if we had a way of adding a local influence to a global scope, or something along those lines, in order to improve the quality of the recommendations.

I'm not certain to what extent these should be incorporated into the proposal. Pinging User:Halfak (WMF) so he knows about this thread. Thanks again for the comments! Cheers, Nettrom (talk) 19:44, 3 February 2017 (UTC)

Rethinking importance

Thanks for the proposal. In Research:Automated_classification_of_article_importance#Proposed_Project you say: "Training a machine learner based on Wikipedian article importance assessments that can predict article importance both globally (e.g. for a Wikipedia edition as a whole) and locally (e.g. within a specific WikiProject).". What do you mean by "Wikipedian article importance assessments"? Are you referring to predicting standard class types used in some Wikipedia languages, for example, English Wikipedia (see here for a table of classes)?

On a separate note and to document what we have already discussed offline: You have two main paths for tackling the problem of defining what importance should mean. One is the macro-economic path (and I won't be able to comment on that), the other is the more applied path that fields such as CS use. For the latter, you will need to focus on a goal to be able to define importance. For example, if your goal is to find the top n articles a reader should read to learn about topic x, you may need to define importance differently than if your goal is to focus on the task of sorting existing articles that need to be expanded.

And last but not least, feel free to use the features used in section 2.2 of https://arxiv.org/abs/1604.03235 when/if they become relevant to your work. Good luck! :) --LZia (WMF) (talk) 18:36, 6 February 2017 (UTC)

Hi LZia (WMF), thanks for the comments! The "article importance assessments" that I'm referring to are the Low/Mid/High/Top-importance ratings found on article talk pages, yes. I see from the link you provided there are also "Bottom", "NA", and "??/Unknown" ratings. "Bottom" appears to be used sparingly compared to the common ones, and I don't see "NA" and "??" ratings being very useful as they are basically a "non-label".

I agree with your example that importance might be defined differently depending on what scope of articles we're looking at. It reminded me of what I wrote in my response above, we currently do not know much about how importance differs with different scopes. To some extent I think it also could blur the line between importance and relevance. The latter is a topic I have so far mostly scratched the surface of, but I've read a couple of interesting papers in JASIST (Hjørland, B. "The foundation of the concept of relevance", 2010; and Cole, C. "A theory of information need for information retrieval that connects information to knowledge", 2011). At the moment I am not sure I am able to articulate my thoughts about it well, so I'll keep working on it.

And thanks for mentioning the paper, I'll be sure to revisit it and definitely grab features from it where it would be useful. Cheers, Nettrom (talk) 01:26, 8 February 2017 (UTC)

Looking within WikiProjects

I generated some inlink counts for a few articles in WikiProject biology. See https://quarry.wmflabs.org/query/16204

article	Articles in WikiProject	Links from within WikiProject
Biology	2520	418
Abdominal_cavity	2520	4
Abyssal_plain	2520	7

--Halfak (WMF) (talk) 22:28, 6 February 2017 (UTC)

Oooh! Here's a query that gets inlink counts for all pages linked from entire WikiProject. https://quarry.wmflabs.org/query/16210 --Halfak (WMF) (talk) 23:36, 6 February 2017 (UTC)