User talk:Community Tech bot

Archives

---

1

Empty report

Tracked in Phabricator:
Task T159774

Hi, really nice to see that the report has been taken up again. On en:Wikipedia:WikiProject Anime and manga/Popular pages, the bot seems to have created an empty report though. Opencooper (talk) 00:06, 5 March 2017 (UTC)

@NKohli (WMF): pinging. Opencooper (talk) 20:11, 9 March 2017 (UTC)

Hi! The bot is still a work-in-progress. There are a bunch of reports that have turned up empty. This is because the PROJECT param that project uses in its assessment template is different from the project name. If you could help with fixing that, you can change the "Name" field for the project on User:Community_Tech_bot/Popular_pages_config.json. If you don't feel comfortable editing JSON, I can do it as well. It'll take me a day or two to get there though and some more days to update the bot again. Thank you for your patience. :) -- NKohli (WMF) (talk) 01:58, 10 March 2017 (UTC)

Patience - yes - it is removing more than adding of the projects I have on watch - please fix it! Thanks :JarrahTree (talk) 10:46, 10 March 2017 (UTC)

repeatedly removing - and not replacing with anythng :JarrahTree (talk) 14:52, 14 March 2017 (UTC)

@Opencooper: The reason WikiProject Anime and manga isn't working is because that project doesn't use the WPBannerMeta template for its talk page banner (which this bot relies on). That's also why it doesn't show up in other interfaces like CopyPatrol. It's possible to work around this limitation by adding a bunch of extra code to the WikiProject Anime and manga banner template, but like I tried to explain on the template talk page, there's no reason why WikiProject Anime and manga couldn't use the WPBannerMeta template. I'll go follow up on that discussion and see if we can reach a consensus. Kaldari (talk) 06:51, 15 March 2017 (UTC)

NKohli, I would edit it but I'm not sure what to change it to so it would be appreciated if you could make the changes instead. That is if the issue Kaldari mentioned about the WPBannerMeta template isn't a blocker. Kaldari, thanks for looking into it, I certainly think the templates should all use the same base for consistency and so other tools can utilize them properly. Opencooper (talk) 07:10, 15 March 2017 (UTC)

@JarrahTree: If you could let us know which projects you are noticing problems with, we'll try to get them fixed quickly. FWIW, the bot will no longer write empty reports as of earlier today. If it can't locate the articles for a project, it will just skip writing the report entirely. Kaldari (talk) 05:36, 16 March 2017 (UTC)

sorry I have a very complicated watch list - might not be able to track in the short term :JarrahTree (talk) 13:03, 16 March 2017 (UTC)

Popular pages

I realize it might still be in development, but wanted to send a notice that these reports don't appear to be working:

• WikiProject Video games/Popular pages
• WikiProject Philosophy/Anarchism/Popular pages

czar 06:37, 14 March 2017 (UTC)

Empty WikiProject_Poker/Popular_pages 2005 (talk) 01:57, 15 March 2017 (UTC)

@Czar and 2005: I looked into the 3 WikiProjects mentioned above:

• WikiProject Video games: Bot hasn't gotten to it yet, but should work.
• WikiProject Philosophy/Anarchism: Known bug (https://phabricator.wikimedia.org/T159774)
• WikiProject Poker: This is set up as a task force in {{WikiProject Gambling}}, although it's actually not a task force, it's a full WikiProject. I've fixed this in the config for the bot, but it should really be fixed at the template level. All WikiProjects should have their own banner templates.

Kaldari (talk) 06:35, 15 March 2017 (UTC)

The bot also emptied en:Wikipedia:WikiProject Western Australia/Popular pages (diff). Probably the same issue as above - the project uses parameters within the Australia project's banner, since by definition any page within the WP:WA project's scope is also within the parent project's scope. - Evad37 (talk) 02:00, 16 March 2017 (UTC)

@Evad37: WikiProject Western Australia is fixed for the next bot run. The issue, as you noted, is that WikiProject Western Australia is set up as a task force of WikiProject Australia in the banner template, so it gets reported to the PageAssessments extension as "WikiProject Australia/WikiProject Western Australia" rather than just "WikiProject Western Australia". Let us know if you see any others that need to be fixed. Kaldari (talk) 05:42, 16 March 2017 (UTC)

That is very weird - because all the states of australia have projects and we were under the inpression that they were setup as 'sub projects' - not 'task forces' as far as I can remember when they were created - :JarrahTree (talk) 13:01, 16 March 2017 (UTC)

I wonder if the bot should be aggregating page views for redirects and the redirect target in compiling the report? Imzadi 1979 → 22:47, 15 March 2017 (UTC)

@Imzadi1979: We'll be adding that soon: phabricator:T160201. Kaldari (talk) 05:24, 16 March 2017 (UTC)

Coming from WP:NZ. Can we rank pages not by views but by views/wikiprojects? Most of our high ranked pages are in dozens of different projects and we have little to do with them. Stuartyeates (talk) 07:50, 19 May 2017 (UTC)

Updates

Wondering if we can have the updates occur in the section "list"? And not remove the top stuff here. Best Doc James (talk · contribs · email) 19:49, 14 March 2017 (UTC)

Also does this bot produce total pageviews for the pages within a Wikiproject in that month? Doc James (talk · contribs · email) 00:55, 15 March 2017 (UTC)

About having updates occur only in section "list", it'll need some investigation and modifications in the existing code. Can you file a ticket for this?

And no, the bot does not produce total pageviews for a project. Is this something that'd be useful to add? Feel free to file a ticket if so. -- NKohli (WMF) (talk) 17:38, 15 March 2017 (UTC)

Note that the previous version of the bot didn't overwrite page headers (e.g., project nav banners). Stevie is the man! ^{Talk • Work} 14:50, 16 March 2017 (UTC)

Okay created the ticket here[1] for total pageviews Doc James (talk · contribs · email) 01:24, 17 March 2017 (UTC)

Page views for WikiProject Sanitation?

There used to be a bot giving us the popular pages views statistics for WikiProject Sanitation. Not sure if this bot here is that same as that one? If yes, when will this page update itself? Do I need to do something to get it updated? See here: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Sanitation/Popular_pages Currently it is stuck at March 2016. - Thanks. EMsmile (talk) 12:18, 16 March 2017 (UTC)

Updated. The bot hadn't reached it when you posted this. :) -- NKohli (WMF) (talk) 19:12, 16 March 2017 (UTC)

Current issues in popular pages

Here are current issues that I know of:

1. Empty reports (see above). Also add en:WP:LOU/PP to the pile. en:WP:KY/PP hasn't been run at all yet. (These are projects I work on a lot.)
2. Redirects aren't included in the stats like before (phab:T160201)
3. Page headers are being overwritten, like in en:WP:LOU/PP, the project banner went away. The previous bot left page headers in place.

Stevie is the man! ^{Talk • Work} 14:43, 16 March 2017 (UTC)

Redirects though once they are followed end up being added to the viewer numbers for the primary page no? Doc James (talk · contribs · email) 15:51, 16 March 2017 (UTC)

I'm not expert in this aspect, but the previous bot had to add redirect calls because the views were accumulated for the redirects and not the pages they redirected do. There is a ticket linked above to investigate this. Stevie is the man! ^{Talk • Work} 17:51, 16 March 2017 (UTC)

Ah well if that is the case it needs to be fixed. Doc James (talk · contribs · email) 01:25, 17 March 2017 (UTC)

I guess the issue we have at this point is how gradual the bot is running. It's April 11, and still working on WikiProjects that start with 'B', the last one being 'Boxing'. It's not purely running in alphabetical order, but that's generally how the bot is proceeding. At this rate, it clearly will go into succeeding months just to complete March stats. I didn't want to be the one to complain, but this is a problem. Stevie is the man! ^{Talk • Work} 17:11, 11 April 2017 (UTC)

I haven't investigated the code, but it seems like it's quering article that's in multiple projects multiple times, but it gets the same data (or it should get :D ). So I think it would be logically (and probably much more faster) to simply get the list of pages, that are in at least one project scope and then query views for them (store data in database). And then simply update reports. --Edgars2007 (talk) 16:17, 22 April 2017 (UTC)

That's correct. The bot is indeed slower than we'd anticipated because we are fetching data for redirects and that pushes the number of pages queried to be very high. We are looking into ways to make it run faster without compromising on the data. To respond to @Edgars2007: every project does its own evaluation for a page. So it is very much possible for a page to have different class/Importance factors depending on the wikiproject. -- NKohli (WMF) (talk) 02:41, 8 May 2017 (UTC)

NKohli, OK, yes, each project has it's own evaluation, but that doesn't mean, that pageviews will be different for those articles in other project :) How I see bot algorithm (in big steps):

a) get's list of all articles, that are in at least one project scope (so it gets article "United States" only once, not many)

b) get's pageviews for each article (with all those redirect views) and store them in datrabase or some array

c) get's let's say Wikiproject Maths article list with their quality levels and lookups for pageviews in table or array.

d) voila!

--Edgars2007 (talk) 07:08, 8 May 2017 (UTC)

Yeah, we are looking into caching pageviews for articles but it will not provide us as much of a speedup as it appears because the Pageviews API does caching on their end, so the time saved would be very small. Our best possible alternative, apparently, is to run two or more instances of the bot so it can cover multiple projects in parallel. We risk running into memory or API limits though. -- NKohli (WMF) (talk) 18:47, 8 May 2017 (UTC)

I would like to include another, (not sure if 2. covers this or not):

4. Sub-project groups (eg. WikiProject Caribbean), that are tagged to parent pages (eg. Haiti=yes) are also not incurred no longer in these statistics. Please, please find a solution for this one (as well as the mentions above of course). Popular pages is probably my favorite statistic on the platform and it was working so well, don't want it to become nostalgic. Cheers! Savvyjack23 (talk) 20:48, 3 May 2017 (UTC)

Hi @Savvyjack23:, thanks for reporting this. We can take a look into it. -- NKohli (WMF) (talk) 02:41, 8 May 2017 (UTC)

Hello. I think the same is happening for Wikipedia:WikiProject Scottish Castles. Where the talk page has {{WikiProject Scotland|castles=yes}} (instead of {{WikiProject Scottish Castles}}), then it doesn't show up in the popular pages list. This is the case for en:Balmoral Castle, which gets most page views of all yet does not appear in the list! Thank you for looking into this. What Are The Civilian Applications? (talk) 12:29, 8 May 2017 (UTC)

@Savvyjack23: The problem with Haiti is that Haiti is set up both as a full WikiProject and as a task force of WikiProject Caribbean. Currently the bot is only generating a report for the full WikiProject. I can also have it generate a separate report for the task force, but they can't be combined into the same report. Someone needs to decide if Haiti is a full WikiProject or a task force of WikiProject Caribbean. It can't be both. It looks like Cuba has the same problem, but Jamaica, Dominican Republic, Bermuda, Saint Vincent and the Grenadines, Barbados, etc are set up correctly. The easiest solution is to do something like Template:WikiProject Jamaica. Kaldari (talk) 23:01, 8 May 2017 (UTC)

Report not updated

The data for Wikipedia:WikiProject Jainism has not been updated for march and April. Capankajsmilyo (talk) 01:17, 8 May 2017 (UTC)

Done. -- NKohli (WMF) (talk) 02:42, 8 May 2017 (UTC)

Add project to popular pages reporting

Can you add GLAM/Pritzker [2] so that this WikiProject receives the popular pages report too? We'd like to see the top 100 pages for GLAM/Pritzker. Thanks for your assistance! TeriEmbrey (talk) 13:49, 18 May 2017 (UTC)

Hi @TeriEmbrey:. I want to clarify if the project you are referring to is "Pritzker Military Library". Could you tell me how you do article assessments for the project? I'm interested in the Project name field you use for doing the evaluations. Thanks. -- NKohli (WMF) (talk) 18:19, 18 May 2017 (UTC)

@NKohli (WMF): "Pritzker Military Library" is correct. Admins set this up for the WikiProject in 2013. The institution's official name has since changed to the Pritzker Military Museum & Library. Thanks for helping with this! TeriEmbrey (talk) 18:25, 18 May 2017 (UTC)

This has been done now. I generated the initial report to test it. It's available at en:Wikipedia:GLAM/Pritzker/Popular pages. -- NKohli (WMF) (talk) 20:13, 18 May 2017 (UTC)