The Scholarly Kitchen

I'm reminded of something I read recently about disclosure of AI use in scientific papers [1]: > Authors should be asked to indicate categories of AI use (e.g., literature discovery, data analysis, code generation, language editing), not narrate workflows or share prompts. This standardization reduces ambiguity, minimizes burden, and creates consistent signals for editors without inviting overinterpretation. Crucially, such declarations should be routine and neutral, not framed as exceptional or suspicious. I think that sharing at least some of the prompts is a reasonable thing to do/require. I log every prompt to a LLM that I make. Still, I think this is a discussion worth having. [1] https://scholarlykitchen.sspnet.org/2026/02/03/why-authors-arent-discl…

Having watched the discussion here for the past couple of days, I agree with frozenwind's comment. I wrote my own reflection here (the last comment): https://aisnakeoil.substack.com/p/a-misleading-open-letter-about-sci/c… Regarding impacts upon academia, I wrote two comments here: https://scholarlykitchen.sspnet.org/2023/04/12/guest-post-gpt-3-wrote-… I'd also like to emphasize that it is not only about open source LLMs. I haven't followed the situation myself, but some sources say that LLMs are being developed also in the dark web, and who knows what kind of content the criminals there possess? (Borderline cases like NoiseGPT are already done openly.) Given the current geopolitical situation, the same goes for nation states. Fun times ahead indeed.

You are probably talking about the comments for this article: http://scholarlykitchen.sspnet.org/2016/03/02/sci-hub-and-the-four-hor… I agree, some of the opinions tossed around by incumbents in the industry are mind-blowing. My favorites: "A PDF is a weapons-grade tool for piracy." ...And on parents of sick kids looking up articles: "Do you really think a search and downloading of technical medical literature means anything?"

Not specific to Wikipedia: https://aarontay.medium.com/3-new-tools-to-try-for-literature-mapping-… https://archive.is/Ul13s Specific to Wikipedia: Wikipedia Citations: Reproducible Citation Extraction from Multilingual Wikipedia [2024] https://arxiv.org/abs/2406.19291v1 https://doi.org/10.48550/arXiv.2406.19291 > Wikipedia is an essential component of the open science ecosystem, yet it is poorly integrated with academic open science initiatives. Wikipedia Citations is a project that focuses on extracting and releasing comprehensive datasets of citations from Wikipedia. A total of 29.3 million citations were extracted from English Wikipedia in May 2020. Following this one-off research project, we designed a reproducible pipeline that can process any given Wikipedia dump in the cloud-based settings. To demonstrate its usability, we extracted 40.6 million citations in February 2023 and 44.7 million citations in February 2024. Furthermore, we equipped the pipeline with an adapted Wikipedia citation template translation module to process multilingual Wikipedia articles in 15 European languages so that they are parsed and mapped into a generic structured citation template. This paper presents our open-source software pipeline to retrieve, classify, and disambiguate citations on demand from a given Wikipedia dump. Prior work referenced in above abstract with some team overlap: Wikipedia citations: A comprehensive data set of…

A reporter's tips for understanding science journalism: http://www.cjr.org/analysis/understanding_science_journalism.php A dialog between two scholars about how make scientific research more trustworthy: http://www.washingtonpost.com/blogs/monkey-cage/wp/2015/03/13/how-to-m… A blog post, "How Can We Improve the Article Review and Submission Process?" with many suggestions for improving scientific publishing: http://scholarlykitchen.sspnet.org/2015/03/26/ask-the-chefs-how-can-we… In the end, we have to count on incremental improvements in publishing, and incremental improvements in reading by all of us, to prompt medical studies and all kinds of scientific studies to become more accurate and informative about important issues. We especially have to improve our own tendency to seize a hopeful answer whether or not it's a true answer as we grapple with life's problems. (When I write "we" here I am including all of us, including myself, whether we are producers or consumers of scientific studies.) Medical studies specifically and scientific studies in general have helped humankind gain a lot of useful knowledge, but it has been a slow, unsteady process with lots of back and forth. P.S. All the links I've posted here, including the one opening the thread, are from the "Weekend Reads" weekly post on the Retraction Watch group blog, which I think many Hacker News participants will find…

>Of course we do, we make amazing science at such a huge rate most people can't even keep up with it anymore. If you believe computing has only little and only "quantitative" impact on science and communication than there is little I can do for you, computing has completely revolutionized every single science field. But sure, keep thinking it's just cat videos and porn. Maybe we should just go farming instead... The condescending tone is because you believe you are talking to some child or something? People have differing opinions, and what you believe as the "objective truth" is not some kind of gospel. As a matter of fact, the very issue of current science having much less impact nowadays has been stated before on HN, with very valid explanations and analysis behind it. One reason, for example, is that the more important stuff in most scientific fields are the lower hanging fruits, and early scientists got most of them. It's a case of diminishing returns. Say whatever you want, but any current discovery is not of the caliber of Maxwell's equations, the discovery of DNA or evolution, Einstein's theory of relatively, et al. It's marginal, incremental work, not evolutionary. And people "can't even keep up with it anymore" just because it's so vast and concerns minutiae, not because it's so groundbreaking. Here are some further elaborations on…

That could be a useful feature. It's got vibes from the 90s, when there were a lot of different browsers and some of them allowed users to annotate pages and links [1]. I'm sure that there are a number of extensions to do that and still it's OK to have it in the browser by default. [1] https://scholarlykitchen.sspnet.org/2013/04/30/iannotate-whatever-happ…

reminds me of of the El Naschie controversy: https://scholarlykitchen.sspnet.org/2008/11/25/elsevier-math-editor-co…

There's an incentive for scholarly publishers and authors to have their works cited on Wikipedia, since there is a belief (supported by some research [0]) that citation on Wikipedia increases subsequent scholarly citations to those articles. Such a benefit could be used to increase a Journal Impact Factor & similar metrics [1,2] or the citations to one's own works, which is beneficial to academics competing for promotion, tenure, and research grants which are partially based on citation metrics.[3] I wouldn't be surprised if there was some indiscriminate citation on Wikipedia to bolster the metrics. [0] https://scholarlykitchen.sspnet.org/2022/11/01/guest-post-wikipedias-c… [1] https://en.wikipedia.org/wiki/Impact_factor [2] https://en.wikipedia.org/wiki/Citation_impact [3] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6668985/

Also in the case the license was changed to be more restrictive, which gives a headache on how to prove that the use before the license was changed is permitted if the license information was not captured (manually or by archive sites) at first use. [1] gave a good discussion about whether it is possible to revoke CC license. It doesn't seem likely, but if it does, then it will lead to the license paradox for creators and users alike. [1] https://scholarlykitchen.sspnet.org/2022/05/11/q-can-you-revoke-a-crea…

> I mean sci-hub doesn't do that. Not if you're just downloading papers, but where do you think they get those papers from? People giving them Shibboleth logins. Obviously they are very secretive about how they get those logins. There are claims they use phishing. https://scholarlykitchen.sspnet.org/2016/02/25/sci-hub-how-does-it-wor… > [logins] are apparently ‘donated’ by academics. It’s not clear whether the academics who have donated such information are aware of what they have done. According to Edward Sanchez, head of library information technology at Marquette University, academics there have been targeted by a phishing campaign specifically aimed at obtaining university access credentials. These have subsequently been used by Sci-Hub. Sci-Hub contests this.

It's a rumour people working with the traditional academic publishers have been spreading, but I haven't been able to find any proof. All I got when asking for it was "the evidence exists, but there is no gain in publicizing it": https://scholarlykitchen.sspnet.org/2017/12/21/wakeup-call-looking-bac…

I'm not sure where you're getting these figures from. > PubMed Central (PMC) costs US taxpayers about $4.45 million per year to run, according to documents recently obtained by an ongoing Freedom of Information Act (FOIA) request. https://scholarlykitchen.sspnet.org/2013/07/16/the-price-of-posting-pu… The National Library of Medicine's entire budget is $341 million, and they have brick and mortar buildings to maintain and many other responsibilities - there's no way that almost their entire budget would go to paying for PubMed.

Depends on who is suing you: OpenAI for using their model results, or the owner(s) of the data their model was trained on? Either way, it's a grey zone that copyright law hasn't come to grips with yet. See https://scholarlykitchen.sspnet.org/2020/02/12/if-my-ai-wrote-this-pos…

I think pretty much any university could provide proof that Scihub uses phished credentials to proxy articles for their users. It's no secret in HE. Here's the first Google link. It's far too alarmist but should at least give the gist. https://scholarlykitchen.sspnet.org/2018/09/18/guest-post-think-sci-hu… One other point I missed that we have to often deal with : when phished accounts are used to mass-download PDFs, many publisher sites auto-block the IP of the requester, which in this case is the University's Ezproxy server. This then means no user at the university can access the resource till the block is lifted (or they could just use Scihub in the meantime :~D ).

As you might expect from similar stories, this is 100% bullshit. The article doesn't give much detail, but it's easy to Google the kid. More info: https://scholarlykitchen.sspnet.org/2014/01/03/the-jack-andraka-story-…

This reminds me of the time when all papers in some journals had this boilerplate at the end: The publication costs of this article were defrayed in part by page charge payments. This article must hereby be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact. [1] [1] https://scholarlykitchen.sspnet.org/2014/01/09/history-as-caution-when…

When every talk, study, etc. discussing it (e.g. https://greenelab.github.io/scihub-manuscript/) includes a statement like "Users of Sci-Hub do so at their own risk. This study is not an endorsement of using Sci-Hub.", I don't think it can be called pristine. And from articles like https://scholarlykitchen.sspnet.org/2018/09/18/guest-post-think-sci-hu… there seems to be some connection between Sci-Hub and the cyber underworld, although perhaps it's just the cyber thieves sending Sci-Hub papers and Sci-Hub isn't an active participant.

This is a misunderstanding: “non-profit” does not mean that an organisation can’t earn money. It means the company cannot be sold, and all profits have to be reinvested (rather than e.g. paid out to shareholders). And PLOS, like for-profit publishers, has in the past had very large profit margins (and has been criticised for this, e.g. [1]). [1]: https://scholarlykitchen.sspnet.org/2015/09/29/plosone-hikes-pub-fees/

Perhaps this is my bias showing, but if you read through this whole interaction [1,2,3], Kent Anderson comes across as a real dick, while both Yann LeCun and George Monbiot come across as pretty reasonable albeit ticked off about academic publishing (imo quite rightfully). [1] https://www.theguardian.com/commentisfree/2011/aug/29/academic-publish… [2] https://scholarlykitchen.sspnet.org/2011/09/01/uninformed-unhinged-and… [3] https://blogs.harvard.edu/pamphlet/2012/03/06/an-efficient-journal/

Good. It's one of those bloody articles that requires a layman to read all of the attached articles, two or three links deep, to have any chance of understanding just the parts without mathematical formulae. And before anyone accuses me of wanting to dumb down Wikipedia: yes, that's exactly what I want. Wikipedia should be written to a generalist audience, not by specialists for specialists. edit: I can't say it any better than this article: https://scholarlykitchen.sspnet.org/2012/09/24/wikipedias-writing-test…

yep, "Impact Factor" (IF) is the term, and (just to clarify) it encourages but does not force. there are open access journals like PLoS w/ significant IF, albeit not comparable to Nature. since the link below was written, PLoS has become not only open access for articles but also open access for the data those articles are based on. imo, open access to data (w/ privacy measures, of course, for human subjects) is of the same order of importance as access to articles. http://scholarlykitchen.sspnet.org/2013/06/20/the-rise-and-fall-of-plo…

Yes, if anything we need better documentation of methods. This is something often overlooked with the open-scientific-data movement also: data on its own is not very useful (in the sense of being insufficient to draw valid conclusions) if you don't have good documentation of how precisely it was collected, which is critical to interpreting it correctly... otherwise you just have some numbers in a CSV file. Traditionally PhD theses are one of the few places that really exhaustively documents what exactly they did and how, while papers tend to gloss over a lot of things. There's been a little noise about that [1], but not a lot of movement so far. I do think a movement of theses away from monograph-style to collection-of-articles style is okay, though. [1] http://scholarlykitchen.sspnet.org/2014/11/18/nevermind-the-data-where…

I think it would make much more sense to instead have the journals publish articles in more formats, rather than just HTML and PDF. For example, they should offer them in EPUB[0]. Since EPUB is much more accessible to blind/visually impaired people than PDF, perhaps the federal government could step in and mandate that all articles with content produced using federal grants must be available in a format that the blind/visually impaired can consume as well. 0: http://scholarlykitchen.sspnet.org/2013/03/19/is-it-time-for-scholarly…

You are answering the wrong question. The question was NOT whether it spreads information from the articles to people who wouldn't have paid for it. The question was whether it suppresses sales of the articles to people who otherwise might have paid for it. That's a more complicated question of fact. Some people now read Wikipedia and won't buy the article. Some people encounter the reference on Wikipedia and decide to buy the article. Which happens more? I don't have data. But publishers do. And https://scholarlykitchen.sspnet.org/2022/11/01/guest-post-wikipedias-c… shows what publishers concluded. Publishers concluded that Wikipedia references are good for sales. And so jumped on the chance to cooperate with https://wikipedialibrary.wmflabs.org/. Which is therefore able to give free access to 90% of subscription only databases to you if you can prove that you're the kind of person who is likely to add citations to Wikipedia. Legal questions are funny like that. You have to answer the question actually asked. If you merely answer another one that sounds similar to you, your answer is generally wrong.

Scholarly Kitchen is the 'house' blog of the publishing industry, they've run very similar articles >https://scholarlykitchen.sspnet.org/2018/09/18/guest-post-think-sci-hu… >Let me be clear: Sci-Hub is not just stealing PDFs. They’re phishing, they’re spamming, they’re hacking, they’re password-cracking, and basically doing anything to find personal credentials to get into academic institutions. While illegal access to published content is the most obvious target, this is just the tip of an iceberg concealing underlying efforts to steal multiple streams of personal and research data from the world’s academic institutions. Comment 1: >The notion that Sci-Hub is not involved in hacking is laughable. The founder of Sci-Hub, when asked about this, disingenuously replies that Sci-Hub itself does not engage in hacking and phishing, without disputing that it relies on these to operate. More evidence that there is something big and powerful behind Sci-Hub: the Russian mafia. Or this howler, a blog-post on Russian information warfare that then goes into Sci-Hub: >In the scholarly information community, some individuals apparently sympathetic with the open information calls of Sci-Hub and LibGen actively shared authentication information, inadvertently providing institutional credentials to cybercriminals and cyberwarriors, who are probably still sitting on the usernames and passwords or, more likely, the information they grabbed before the…

I'm on the fence about Sci-hub. Every time I read about it, I remember this article about how they operate. >Let me be clear: Sci-Hub is not just stealing PDFs. They’re phishing, they’re spamming, they’re hacking, they’re password-cracking, and basically doing anything to find personal credentials to get into academic institutions. While illegal access to published content is the most obvious target, this is just the tip of an iceberg concealing underlying efforts to steal multiple streams of personal and research data from the world’s academic institutions. This might just be a hit piece by the same companies who are losing money, but it has some merit with proof of attacks changing passwords, etc. Real, tangible damage. I'm not sure this is what Aaron Swartz envisioned. I'm all for vigilante justice or whatever pirates use to justify it (seriously, I petitioned my local college to stop subscribing to them) but this is hardly the same thing. https://scholarlykitchen.sspnet.org/2018/09/18/guest-post-think-sci-hu…

So, there was an earlier attempt at a preservation storage system with a "pay up front to fund perpetual storage", called the "Digital Preservation Network". It came out of the library and archive sector, and focused on institutional customers -- libraries and archives with long-term archival/preservation needs. It was also non-profit. It of course seems attractive, if you really want to ensure long-term preservation, to set up a financial model that can somehow do this independent of the risk of the business continuity and budgets of the original depositors. And for cultural heritage institutions -- in the business of preserving incrementally more stuff every year -- it's a big advantage to budgeting if you can budget per GB you add to preservation storage up front, and never have to pay for it again -- so your yearly budget for preservation storage is just based on number of NEW TB's you are adding, instead of constantly going up with total number of TB's preserved, and thus this portion of your budget stays pretty constant instead of ever-increasing. However, the DPN folded in Dec 2018, after fewer than 5 years of operation. It did not attain financial sustainability. (So all those up-front fees did not end up paying for perpetual storage after all). You can read more about it's shutdown here: https://duraspace.org/the-digital-preservation-network-dpn-to-cease-op… https://scholarlykitchen.sspnet.org/2018/12/13/digital-preservation-ne… This new…

2) why the market failure? Because the scientific reputation (and career prospects) are tied to publications in reputable journals, to certain titles. And the titles belong to the publishers. There has been considerable consolidation over time, the big 4 or 5 have just bought up smaller publishers over time, and it's now run not by idealistic academics, but (by all appearances) by cold-blooded money guys trying to squeeze out as much as they can. Competitors can't just enter - there's a signalling problem: how do you distinguish scientists that don't publish in the top journal because they're idealistically opposed to their closed-access practices, from those that cannot get published there because their work is not up to standards? 1) Open Access Journals: the editors do some editing and vetting, and, crucially, peer review. If you put an article on your website, it doesn't have the same standing, "it doesn't count" as a real article or for your publication list. So, in terms of scientific reputation, for now: Traditional journal > OA journal > self-published. The challenge is to get OA >= traditional, and that takes a lot of time, or, rather, coordination: Don Knuth sent a letter in 2003 to the editors of the Journal of Algorithms to persuade them to collectively resign and move to a different journal. People of that stature are needed to break the power of despicable academic publishers (Elsevier is particularly revolting,…

Full disclosure: I'm a founder of a company called Scholastica that provides software that helps journals peer-review and publish open-access content online. One of our journal clients, Discrete Analysis, is linked to in the NYT article. It is incredibly obvious that journal content shouldn't cost as much as it does. - Scholars write the content for free - Scholars do the peer-review for free - All the legacy publishers do is take the content and paywall PDF files Can you believe it? Paywalling. PDFs. For billions. Of course the publishers say they create immense value by typesetting said PDFs, but as technologists, we can clearly see that this is bunk. There's a comment in this thread that mentions the manual work involved in taking Word files and getting them into PDFs, XML, etc. While that is an issue, which you could consider a technology problem, it definitely doesn't justify the incredible cost of journal content that has been created and peer-reviewed at no cost. Keep in mind that journal prices have risen much faster than the consumer price index since the 80s (1). The future is very clear, academics do the work as they've always done and share the content with the public at a very low cost via the internet. PS. If you want a peek into how the publishers see the whole Sci-Hub kerfuffle, check out this post from one of their industry blogs - the comment section is a doozy:…