Jump to content

Talk:Mix'n'match/Catalogues: Difference between revisions

From Meta, a Wikimedia project coordination wiki
Latest comment: 4 years ago by Eru in topic Scraping
Content deleted Content added
Eru (talk | contribs)
Line 11: Line 11:
{{ping|Adam Harangozó}} Can you scrape Sarvavijnanakosam (Malayalam Encyclopaedia) under [[Mix%27n%27match/Catalogues#Encyclopedias_(general)|Encyclopedias (general)|Encyclopedias (general)]], it uses Mediawiki. I am not too familiar with the tool. Thanks. [[User:Gotitbro|Gotitbro]] ([[User talk:Gotitbro|talk]]) 09:04, 15 January 2020 (UTC)
{{ping|Adam Harangozó}} Can you scrape Sarvavijnanakosam (Malayalam Encyclopaedia) under [[Mix%27n%27match/Catalogues#Encyclopedias_(general)|Encyclopedias (general)|Encyclopedias (general)]], it uses Mediawiki. I am not too familiar with the tool. Thanks. [[User:Gotitbro|Gotitbro]] ([[User talk:Gotitbro|talk]]) 09:04, 15 January 2020 (UTC)
:{{ping|Gotitbro}} Unfortunately I don't know how to create scrapers either, but maybe {{ping|Magnus Manske}} or {{ping|Gerwoman}} can help. --[[User:Adam Harangozó|Adam Harangozó]] ([[User talk:Adam Harangozó|talk]]) 12:47, 28 January 2020 (UTC)
:{{ping|Gotitbro}} Unfortunately I don't know how to create scrapers either, but maybe {{ping|Magnus Manske}} or {{ping|Gerwoman}} can help. --[[User:Adam Harangozó|Adam Harangozó]] ([[User talk:Adam Harangozó|talk]]) 12:47, 28 January 2020 (UTC)
::Hi, the scraper didn't seem to respond for this site, perhaps because this url doesn't work http://web-edition.sarvavijnanakosam.gov.in/Special:AllPages but only this one http://web-edition.sarvavijnanakosam.gov.in/index.php?title=Special:AllPages
::Anyway, I tried another wiki (https://simpsonswiki.com/wiki/) and I get warnings from the api :
::<source lang="javascript">
<br />
<b>Notice</b>: Trying to get property 'query' of non-object in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>322</b><br />
<br />
<b>Notice</b>: Trying to get property 'allpages' of non-object in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>322</b><br />
<br />
<b>Warning</b>: Invalid argument supplied for foreach() in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>322</b><br />
<br />
<b>Notice</b>: Undefined offset: 0 in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>284</b><br />
{"status":"OK","data":{"html":"","log":["0 (AutoScrapeLevelMediaWiki): Reset [{\"mode\":\"mediawiki\",\"url\":\"https:\\\/\\\/simpsonswiki.com\\\/wiki\\\/Special:AllPages\",\"pos\":0,\"apfrom\":\"\"}]","0 (AutoScrapeLevelMediaWiki): Reset [{\"mode\":\"mediawiki\",\"url\":\"https:\\\/\\\/simpsonswiki.com\\\/wiki\\\/Special:AllPages\",\"pos\":0,\"apfrom\":\"\"}]","Found 0 entries."],"results":[],"last_url":""}}
</source>
::So I think only devs can answer that. [[User:Eru|Eru]] ([[User talk:Eru|talk]]) 18:16, 29 January 2020 (UTC)

Revision as of 18:16, 29 January 2020

I am changing the page to use my new {{MnM}} template. This lets us do things like set a status, specify a catalog, and generate pre-filled scraper links. I started with the films section, please chime in! --Magnus Manske (talk) 11:28, 6 November 2019 (UTC)Reply

Why did you removed AarhusWiki, Ribewiki and KoldingWiki from the list? --Trade (talk) 15:36, 6 November 2019 (UTC)Reply
It wasn't removed, there was a formatting error and it didn't show, it's corrected now. --Adam Harangozó (talk) 20:08, 28 November 2019 (UTC)Reply

new catalog

I am trying to add new catalog to the list. Where is this described. I have a CSV file with URLs and IDs. From an official source (CMS in US).

Scraping

@Adam Harangozó: Can you scrape Sarvavijnanakosam (Malayalam Encyclopaedia) under Encyclopedias (general)|Encyclopedias (general), it uses Mediawiki. I am not too familiar with the tool. Thanks. Gotitbro (talk) 09:04, 15 January 2020 (UTC)Reply

@Gotitbro: Unfortunately I don't know how to create scrapers either, but maybe @Magnus Manske: or @Gerwoman: can help. --Adam Harangozó (talk) 12:47, 28 January 2020 (UTC)Reply
Hi, the scraper didn't seem to respond for this site, perhaps because this url doesn't work http://web-edition.sarvavijnanakosam.gov.in/Special:AllPages but only this one http://web-edition.sarvavijnanakosam.gov.in/index.php?title=Special:AllPages
Anyway, I tried another wiki (https://simpsonswiki.com/wiki/) and I get warnings from the api :
<br />
<b>Notice</b>:  Trying to get property 'query' of non-object in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>322</b><br />
<br />
<b>Notice</b>:  Trying to get property 'allpages' of non-object in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>322</b><br />
<br />
<b>Warning</b>:  Invalid argument supplied for foreach() in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>322</b><br />
<br />
<b>Notice</b>:  Undefined offset: 0 in <b>/data/project/mix-n-match/autoscrape.inc</b> on line <b>284</b><br />
{"status":"OK","data":{"html":"","log":["0 (AutoScrapeLevelMediaWiki): Reset [{\"mode\":\"mediawiki\",\"url\":\"https:\\\/\\\/simpsonswiki.com\\\/wiki\\\/Special:AllPages\",\"pos\":0,\"apfrom\":\"\"}]","0 (AutoScrapeLevelMediaWiki): Reset [{\"mode\":\"mediawiki\",\"url\":\"https:\\\/\\\/simpsonswiki.com\\\/wiki\\\/Special:AllPages\",\"pos\":0,\"apfrom\":\"\"}]","Found 0 entries."],"results":[],"last_url":""}}
So I think only devs can answer that. Eru (talk) 18:16, 29 January 2020 (UTC)Reply