User:Piotrus/Wikipedia interwiki and specialized knowledge test

From Wikipedia, the free encyclopedia

Lies, damned lies, and statistics. -- Benjamin Disraeli

There are about 2 millions articles in need of translation from non-English Wikipedias, and more then 400 million of specialized topics in need of creation. Wikipedia is just in its infancy...

Contents 1 Introduction 2 Polish Wikipedia interwiki test 2.1 To do 2.2 Update 3 Specialized knowledge test 3.1 To do 3.2 Updates 4 See also 5 External links

[edit] Introduction

One of the interesting questions about Wikipedia is 'how much more information there is for Wikipedia to assimilate'?

Part of that answer, if we look at English Wikipedia, is the number of articles from non-English wikipedias that need to be translated. I was somewhat suprized that we have no such statistics - at least I was unable to find information on how many articles on a given Wiki (for example, Polish wiki) have interlinks to a specific (English) Wiki?

I checked pages of User:YurikBot and on Wikipedia:Interwikimedia link, Wikipedia:Interlanguage links (shouldn't those two be merged?), and Wikipedia:Multilingual coordination, but they don't seem to have the ansewer (or I can't find it :>)

Note: while the initial comparison (Polish Wikipedia, PSB) was done by me ( _{Piotr Konieczny aka Prokonsul Piotrus  talk} ), please don't hesitate to edit this page, and add more information (from 'to do' lists or whatever you seem is appopriate). But let's discuss it at the discussion page, not here.

[edit] Polish Wikipedia interwiki test

So I decided to run a little test: take a random sample of 100 pages from Polish Wikipedia (4th largest Wikipedia with over 250,000 articles) and see how many have interwiki links to en wiki. The sample was taken by clicking the 'random page' button and noting down if article has an interwiki or not.

Results: out of 100 pages randomly selected on Polish Wikipedia, 72 had no interwiki links to en Wikipedia. (test as of 22 July 2006)

Notes:

1. The results may be slightly undercounted, as it is possible some pages have an unlinked equivalent on en-wiki. If I were to check, I expect that around 10 pages should have equivalents on en wiki.
2. I did count disambig pages
3. Interestingly quite a few of pages that have no interwiki to English wiki have interwiki to wikis in another language. I did not keep track of that, just noted the general trend.
4. Many of the missing pages were towns, villages, administrative districts. Interestingly, many of those were foreign (French, Italian, etc.) and belonged in the category 'has interwikis to non-English wiki but not to English wiki').

Conclusion: While generalizing from Polish Wiki to other wikis is not recommended and we should do similar tests for other wikis, it appears that between 60-70% of articles on Polish wiki have not yet been translated to en wiki. According to Wikipedia article, Wikipedia had more than 4,600,000 articles in many languages, including more than 1,200,000 in the English-language version. Therefore it is possible that just by translating articles from non-English wikipedias into English Wikipedias we would increase the size of English Wikipedia by ~2,000,000 articles.

[edit] To do

• see if the same number holds true for other Wikipedias. Analysis of German - second largest, and French - third largest, would be quite useful. The other 7 to get 'top ten' would further increase statistical significance of these analysis accross the Wikipedia's projects.
• get a bot to run those tests every month or so
• record data on interwikis to non-english languages if english is not present
• record data on how many articles should have interwikis but don't

[edit] Update

1. 7 August 2007: out of a hundred random pages, 59 pages had no English interwiki. 21 of them however had interwikis to other wikis. This would indicate improvement of coverage of Polish Wikipedia subjects by 14% (last year, 72% had no English equivalent; now, 59%).
2. In September-October 2007, a bot (User:Kotbot) has started to fill the gaps in Polish gminas, villages, and similar content on en-wiki. This may impact the interwiki test.
3. 9 May 2008: 52 had no interwiki, 48 had interwiki.

[edit] Specialized knowledge test

Next, I decided to run a comparison of 'how many articles from a random encyclopedic publication' are missing on Wikipedia. The publication I selected, Polski Słownik Biograficzny (encyclopedia of famous Poles), was not completely random, but as far as I know there is no project dedicated to creating relevant stubs on en-wiki, and as one of my past projects there is a nice index at User:Piotrus/List of Poles. Note also that PSB is not a general knowledge encyclopedia but a specialized knowledge encyclopedia.

Results: out of selected 1000 entries of User:Piotrus/List of Poles/Kisielinski-Korzelinski, about 30 entries have blue links (I ignored entried in need of disambigation, like 10 entries for Konrad). (test as of 22 July 2006)

Notes:

1. As the bot we used to generate this index was not perfect, and it doesn't help if there is an entry with diactrics but no redirect without them, this means the blue links may be somewhat undercounted, but I would be very suprised if by 50%.
2. Due to the nature of PSB (print publication, not easily updated) and my own observation (how many Poles who have articles on Wikipedia are not listed in PSB), it should be noted that PSB does not represent perfect coverage in its area. But for now I assume that the number of articles about Poles Wiki has that are not on PSB, and the number of missing articles from Wikipedia that are covered on PSB evens out.

Conclusion: assuming PSB represents the average coverage of specialized knowledge on (English) Wikipedia, Wikipedia has covers currently about 0,3% of such knowledge. Therefore Wikipedia will be approaching somewhat comprehensive coverage of specialized knowledge when we have about 400,000,000 articles. This is a very rough estimate, but it is my reply to some people who said there is not enough encyclopedic knowledge to merit 2,000,000 articles, as well as to the very optimistic estimates of Wikipedia:WikiProject Missing encyclopedic articles (Biographies - 92.6% done ?? who are they kidding? :D )

[edit] To do

• run similar comparisons on some other specialized knowledge databaes
• run similar comparisns on general knowledge databases
• given statistical data (see Wikipedia:Modelling_Wikipedia's_growth), do a trend estimation to see when we reach ~400m articles

[edit] Updates

1. 8 August 2007. I counted 34 blue links in 'Kisielinski-Korzelinsk'. I counted two more for better stats: 'Olbrycht-Pawleta' - 37; 'Ebenberger-Gembicki' - 28 - so the ~3% still holds
2. 16 May 2008. 'Jesionowski-Kisielewski': 47. 'Skowron-Spiczakow': 23. 'Biergel-Bzowski': 36. Some interesting outliers, but it is safe to say ~3% still holds.

[edit] See also

• Wikipedia:Size comparisons
• Wikipedia:Size of Wikipedia
• Wikipedia:Modelling Wikipedia's growth
• Wikipedia:WikiProject Missing encyclopedic articles
• meta:List of Wikipedias

[edit] External links

• Wikipedia Statistics (contains list of Wikipedia's in different languages)