Wikimedia Diff

> Out of interest, how would you design a page that caters for both tastes? I wouldn't! The great thing about design is that Designers can often do research to make the best decision for everyone. This Wikipedia article is a good starting point if you want to read into how line lengths are chosen. [1] You might wonder why Wikipedia itself is then using a unlimited line width. And the answer is: they're changing it [2] [1]: https://en.wikipedia.org/wiki/Line_length [2]: https://diff.wikimedia.org/2020/09/23/wikipedia-is-getting-a-new-look-…

Worth emphasizing that this is described as basically just adding new ways to consume Wikipedia at scale, not taking anything away (not that that's impossible in the future, but unlikely IMO). In theory, systems that use a lot of Wikipedia data (search engines, "AI" assistant/question answering systems, language models) can keep doing what they're doing with Wikipedia dumps, but new APIs could make these services better (e.g., faster reaction time to current events). A big upside of testing this out is the opportunity to keep better track of what services are Wikipedia-dependent. While search results often send people to Wikipedia (see e.g. https://diff.wikimedia.org/2021/09/23/searching-for-wikipedia-duckduck… voice assistants or Q&A systems might be (i.e. likely are) surfacing Wikipedia content in more subtle fashion. Same with large language models. Under status quo, it will be increasingly hard to track these relationships, so exploring alternatives is well worth it.

Interesting article! Shopify's approach is cool, it's interesting they're using Kafka to generate datasets. I wonder if the explicit human rankings will get stale (and also be hugely outweighted by implicit judgements in the training data). The real-time feedback aspect sounds cool, I wonder if it's just for metrics or also for re-training in real-time. I worked on a Learning To Rank implementation a year or so ago. What struck me then (and now reading about Shopify's implementation) is that the approach is often very similar across sites, but the implementation is usually rather tailored. You see the same patterns: online/offline metrics; nDCG; click models and implicit/explicit relevance judgements; re-ranking top-k of results, and so on. Unfortunately there doesn't seem to be a technology tying all of the components of an LtR system together. A managed service like Algolia could be an answer. I wonder if industry will eventually converge on a framework, such as an extension to Open Source Connection's Elasticsearch Learning to Rank plugin (https://diff.wikimedia.org/2017/10/17/elasticsearch-learning-to-rank-p… It's a really interesting area of theory and practice - I hope Shopify write more about their implementation! I'd also recommend reading Airbnb's really excellent paper - https://arxiv.org/pdf/1810.09591.pdf.

Last year Wikipedia reported an 8% drop in humans traffic[1]. That's huge. [1] https://diff.wikimedia.org/2025/10/17/new-user-trends-on-wikipedia/

> and a from a quick google, not really invasive EFF article about this (2021): https://www.eff.org/deeplinks/2021/02/indonesias-proposed-online-inter… > this site has a ridiculous amount of ads. Literally Other sources: - Indonesia Threatens to Block Wikipedia Over PSE Registration Deadline: https://rri.co.id/en/national/2336218/about.html - Under blackout threat, Wikimedia to hold talks with Indonesia: https://www.france24.com/en/live-news/20260417-under-blackout-threat-w… Related: - https://diff.wikimedia.org/2026/04/15/indonesia-blocks-editor-access/ (talks about the auth domain blockage only)

Setting aside the absurd notion that a site presenting live-editability as its entire core premise is "pretending to be static", do the actual folks at Wikimedia, who have been running a top 10 website successfully for many years, and who have a caching system that worked well in the environment it was designed for, and who found that that system did not, in fact, trivialize the load of AI scraping, have any standing to complain? Or must they all just be bad at their jobs? https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-the-operatio…

According to their own stats (visible in the graph some folds down), it seems to have a fairly steady rate of edits. As for visits, it looks quite constant https://diff.wikimedia.org/2025/10/17/new-user-trends-on-wikipedia/

Yes https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-the-operatio… https://www.usebox.net/jjm/blog/the-problem-of-the-llm-crawlers/

HN removed the hash after the link, pointing to the proper section. Here it is: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(WMF)#Office_acti… Section name is "Office action: Removals on the article Caesar DePaço". Previous article about this: "High stakes for the Wikimedia projects in Portugal: Fighting a strategic lawsuit against public participation (SLAPP)" https://diff.wikimedia.org/2023/07/27/high-stakes-for-the-wikimedia-pr… --- SLAAP: https://en.wikipedia.org/wiki/Strategic_lawsuit_against_public_partici…

https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-the-operatio… This is interesting on the subject. Maybe gives you some understanding. Posted on HN a few minutes ago.

The Wikipedia Library project [0] grants active editors access to a wide range of otherwise paywalled sources. I wonder if it could not be extended to this sort of bot. [0] https://diff.wikimedia.org/2022/01/19/the-wikipedia-library-accessing-…

It's very much designed for and around Wikidata. > Wikifunctions will allow easy access to large knowledge bases such as Wikidata, but also to binary input and output files https://diff.wikimedia.org/2023/08/07/wikifunctions-is-starting-up/ > Using the simple facts housed in Wikidata, you will be able to write functions that make calculations, provide a person’s age, estimate population densities, and more, and integrate the results into Wikipedia. https://www.wikifunctions.org/wiki/Wikifunctions:FAQ > In the future: > - It will be possible to call Wikifunctions functions from other Wikimedia projects, and integrate their results in the output of the page. > - It will be possible to use data from Wikidata in functions.

Related : https://diff.wikimedia.org/2023/07/13/exploring-paths-for-the-future-o…

a statistic I'd be interested in: what percentage of searches can be answered computationally cheap. As in: Wikipedia title index, simple word lookup dictionaries. Indices that could complement a caching search-engine proxy to not hit its origin crawl repository. A study[1] by wikipedia done with DDG notes it showed up in the top5 results and information module for ~13% of searches with a click-through rate for each at ~8% - so a total of ~16% click-through rate. Granted, that is not a number gained from title searches but the whole articles. [1]: https://diff.wikimedia.org/2021/09/23/searching-for-wikipedia-duckduck…

Did you really not know that the editors and "moderators" (a term not used on Wikipedia) are unpaid? The Wikimedia Foundation intervenes very rarely in content. Essentially, it only takes action on content if there is a court order to that effect. ALL OTHER CONTENT CURATION IS DONE BY UNPAID VOLUNTEERS. The Foundation "does not write or curate any of the content found on the projects", as they are happy to tell you themselves here: https://diff.wikimedia.org/2021/10/05/a-victory-for-free-knowledge-flo… Even most of the emails sent to the Wikimedia Foundation are answered by unpaid volunteers, the Volunteer Response Team: https://en.wikipedia.org/wiki/Wikipedia:Volunteer_Response_Team Sorry to sound harsh, but you are laboring under a misconception and spreading it.

Yes. And the endowment is also moving off Tides now that it has its own approved 501(c)(3) status. https://diff.wikimedia.org/2022/10/26/governance-updates-for-the-wikim…

Wikimedia article: https://diff.wikimedia.org/2023/02/21/u-s-supreme-court-declines-to-he…

HN's disdain for redesigns in very short-sighted. Nobody will be using your cool backend algos if they are not exposed to the user in some meaningful, user-centric, consistent way. We should be thankful so much effort is put in UX development, otherwise all the cool backend algos would be pointless. Especially since Firefox did not just say "hey let's make this button bigger and this button green“. They did empirical research, heatmaps etc., to analyse how the UX could be made objectively, measurably better. I invite everyone to read this article about how Wikipedia took several years to implement the "show preview when hovering a link" feature. This is how much thought needs to be given to UX to not make it half-assed https://diff.wikimedia.org/2018/04/20/why-it-took-a-long-time-to-build…

FWIW, Jimmy is on the board but hasn't been involved with the day-to-day for a very long time (Although I'm sure as a board member he would be involved with major decisions like budget). I don't think its fair to call this willful hiding. I mean, they've written blog posts about this subject - https://diff.wikimedia.org/2016/01/14/wikipedia-15-foundation-endowmen… you don't write blog posts about things you're trying to hide. Disclaimer: used to work for them, don't anymore.

https://diff.wikimedia.org/2021/03/16/introducing-the-wikimedia-enterp… Discussed not long ago: https://news.ycombinator.com/item?id=26484080

They do A/B testing. https://meta.wikimedia.org/wiki/Fundraising_2010/Banner_testing https://diff.wikimedia.org/2013/04/08/intro-to-the-statistics-of-ab-te…

Acid-free paper lasts a long time. If you want it to last longer, you could try vellum. Centuries would be achievable, and after a few hundred years you could probably just reprint it on whatever technology they have. http://carlos.bueno.org/2010/09/paper-internet.html https://diff.wikimedia.org/2015/06/19/meet-print-wikipedia/ https://www.heise.de/newsticker/meldung/Wikipedia-Taschenbuchreihe-gep… https://www.heise.de/newsticker/meldung/Deutschsprachige-Wikipedia-wir…

Thanks for the detailed answers! Again, I share at least some of your underlying concern, and don't want that to be overshadowed. That said, some responses: This is the perfect opportunity to do so, and they aren't doing it. Instead, they are picking the smallest possible battle they can. It looks like they've written three articles "strongly" opposing the "tremendous threat" posed by this bill: two when it was being considered[1,2] and another after it passed[3]. Yes, these articles are focused on the impact of the bill on Wikimedia's projects, but I think that's clearly a rhetorical strategy to garner some credibility from the notoriously-stuffy UK legislature. "Foreign nonprofit thinks your bill is bad in general" isn't exactly a position of authority to speak from (if you're thinking like a politician). More recently, they've proposed the "Wikipedia test" to the public and to lawmakers (such as at the 2024 UN General Assembly[6]) that pretty clearly implicates this bill. The test reads as such: Before passing regulations, legislators should ask themselves whether their proposed laws would make it easier or harder for people to read, contribute to, and/or trust a project like Wikipedia. That's ridiculous. ID systems endanger everyone, particularly the adults who participate. I was more making a point about why social media companies aren't involved than justifying that choice for them on a moral level. I suspect you have stronger beliefs than I about the…

With 3.0 there were multiple versions of the licence, which varied depending on the country in which they were used. With 4.0, a single licence is applicable worldwide, with official translations in over 30 languages. This eliminates the need for different ported versions of the licence with changed language tailored for specific jurisdictions. Also, it clarifies that linking to a webpage with attribution information is allowable. It also enables people to fix attribution mistakes within a reasonable time, which is important to help address simple mistakes without the need for overly aggressive copyright enforcement demands. This comment is based on the "Stepping into the future: Wikimedia projects’ transition to Creative Commons’ 4.0 license" article from 29 June 2023 by Stephen LaPorte, Jacob Rogers and Shaun Spalding [0], and may be shared and modified, with attribution, under the terms of the CC BY-SA 4.0 licence. [0] https://diff.wikimedia.org/2023/06/29/stepping-into-the-future-wikimed…

I find amusement in how the three big "knowledge for all" organizations are perfectly distributed along the D&D alignment 'law' axis (taken literally): - Wikimedia (“a world in which every single human being can freely share in the sum of all knowledge”): lawful – religious copyright law adherence, thorough fair use rationales; proactively removes Het Achterhuis from Wikisource over DMCA "actual knowledge" of it being copyrighted in the US through an obscure series of laws[1] - Internet Archive (“Universal Access to All Knowledge”): neutral – gets archival exemption for bypassing copy protection added to the DMCA;[2][3] making archived items publicly available lies in a grey area - Sci-Hub (“free and unrestricted access to all scientific knowledge”): chaotic – just pirate all the paywalled articles [1] https://diff.wikimedia.org/2016/02/10/anne-frank-diary-removal [2] https://archive.org/about/dmca.php [3] https://www.federalregister.gov/documents/2010/07/27/2010-18339/exempt…

>REAL-WORLD APPLICATION >Translating Wikipedia for everyone Hmmm. While there is very definitely utility in doing things like this, I do kinda fear "poisoning the well"-like effects of feeding (even partially-) AI-generated-data into extremely common AI-data-sources. There's some info on it in a blog post[1] and the MediaWiki "Content translation" page[2], but does anyone know of any studies on the quality of the translations produced? I can absolutely see it being a huge time-saver for people who are essentially fluent in both (there's a lot of semi-mechanical drudgery in translating stuff like this that could be mostly eliminated)... but people are pretty darn good at choosing the easy option of trusting whatever they're given rather than being as careful as they should be. It kinda feels like it runs the risk of passively encouraging people to trust the machine's choice over their own, as long as it isn't obviously nonsense, and the cumulative effect could be rather large after a while. [1]: https://diff.wikimedia.org/2021/11/16/content-translation-tool-helps-c… [2]: https://www.mediawiki.org/wiki/Content_translation

https://diff.wikimedia.org/2022/04/20/how-smart-is-the-smart-copyright… > you can let Congress know that you oppose mandatory censorship filters. Fight for the Future is leading a petition opposing the harmful impacts of the legislation that will be delivered to Congress on 25th April, 2022. You can sign the petition at https://nocensorshipfilter.com and make sure Congress knows just how many people are concerned about the impacts this bill will have on free speech. The "SMART Copyright Act" would: - Create universal automated censorship chokepoints across the internet by requiring expensive content filters that we already know don’t work and regularly silence lawful speech - Encourage surveillance backdoors in everything from Canva to Google Drive to internet service provider traffic, since it is written to apply at all layers of the internet - Mandate filtering of all content online at all levels, from Big Tech platforms like Twitch and Instagram to ISPs like Verizon and Comcast—and for any other platform that lets people speak or share online - Force these companies to compromise the safety and security of the internet with mandated use of software even if it has massive flaws, or risk being sued https://onezero.medium.com/nonstandard-measures-cf47c67e8f05 > the labels, publishers and other rightsholder groups backing “stay down” filters will all tell you that Content ID routinely allows infringing materials through. That’s because it is such a blunt…