Pangram Blog

Sol is, indeed, insightful: https://www.pangram.com/history/d08a0026-b19d-4e44-9ffb-f4cdb2d51ca0?u…

You're saying because it has some false positives that Pangram is 100% a scam? Is their research also a scam too? https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangram_4_techni… https://www.pangram.com/blog/pangram-4-technical If so, what is the best one out there other than Pangram then?

It obviously is! Human pattern-matching abilities are rather amazing, and unfortunately I've seen so much AI text that it instantly flagged in my brain within 3 seconds of opening the page. Pangram agrees: https://www.pangram.com/history/3de33376-94e3-404d-bbb0-751a0891a62b

Hey, just FYI to everyone: Even though this article is conveying good (and useful) information, it seems to be completely AI-generated. Pangram clocks it as 100% AI: https://www.pangram.com/history/c410d4b4-abfd-4ca0-b52d-db0d495e6abd Someone else in the thread lists it as 99%: https://www.salahadawi.com/hacker-news-ai-detector Also, it sort of jumps out at you if you're used to reading AI writing. Claude loves to start paragraphs with "Here is the uncomfortable part". I'm not necessarily saying this automatically makes it bad, but everyone should be aware of the source of their information.

Looks largely AI-written, with some human edits: https://www.pangram.com/history/9afe7542-1085-4264-9691-2172c2f4c1d9 FWIW -- I'm not as repulsed by it as the parent comment. But I do want to substantiate that it _is_ heavily LLM-written. (If you're unfamiliar, Pangram has garnered a reputation as the leading LLM-detector, with a minimal rate of false positives; IME this has come with the tradeoff of being easy to manipulate/tweak your way into turning an LLM-generated piece of text into reporting a false negative, but for most folks that's worthwhile.)

I have collected some links about AI's writing style and how to recognize it: https://dbohdan.com/ai-writing-style. You can start with https://buildingalone.substack.com/p/your-words. The most reliable automatic detector right now must be https://www.pangram.com/. If Pangram says something is AI, it is very likely AI. Sometimes it decides more unusual AI text is human-written; Talkie tricks it! (https://x.com/aliceisplaying/status/2059778915275567129.)

Rather obviously they're choosing the one that makes them look best. Another they link to¹ shows 98% for example. Much more importantly, 9/10 dentists agree it's the best. 1: https://arxiv.org/pdf/2501.15654, linked from² https://www.pangram.com/blog/third-party-pangram-evals (the second section) 2: the third study they link there is based entirely around the assumption that Pangram is correct, and seems to have been a collaboration or something as they're included in the credits area.

Pangram has a very low false positive rate, but not the best false negative rate: https://www.pangram.com/blog/third-party-pangram-evals

Title: Good Taste the Only Real Moat Left Followed by an entire AI generated fluff piece https://www.pangram.com/history/347cd632-809c-4775-b457-d9bc3cd83d1e Flagged

See also: “I'm Kenyan. I Don't Write Like ChatGPT. ChatGPT Writes Like Me” by Marcus Olang', https://marcusolang.substack.com/p/im-kenyan-i-dont-write-like-chatgpt… For what it’s worth, Pangram reports that Marcus’ article is 100% LLM-written: https://www.pangram.com/history/640288b9-e16b-4f76-a730-800080cb35f4

Pangram labs thinks this post is fully AI generated, for what it's worth. https://www.pangram.com/history/47460dd9-f9cc-45a8-81dd-dc59bfc18047/?…

Pangram[0] thinks the closing part is AI generated but the opening paragraphs are human. Certainly the closing paragraphs have a bit of an LLM flavor (a header titled "The Pattern", eg) [0] https://www.pangram.com

The article itself reads as an AI generated output, complete with classic Not Just X … Y hallmarks from forever ago, 100% on pangram's low false positive detector. I'm not sure if it's some experiment on their readerbase or what. pangram result: https://www.pangram.com/history/02bead1c-c36e-461b-8fa7-8699a18dad93 So many AI generated AI bashing articles lately. I wrote a post complaining about running into these, and asking people who've sent me these AI articles multiple of them came from HN. https://lunnova.dev/articles/ai-bashing-ai-slop/

It's very clear to me on its face that it's AI, but not "obvious as the sky is blue" others seem to be implying. I would dislike the writing style even if it weren't AI. For the record, an AI detector that appears to have put work into reliability and that I trust very much from my own testing, Pangram (https://www.pangram.com), says this is 100% AI generated. I've used it plenty before when experimenting with AI-collab writing, both fiction and non-fiction, and it's frustratingly accurate in identifying what is and isn't my contribution. I have since largely given up trying to do AI-collab writing, because no matter how nice the writing looks in the moment, it always reeks when read closely, or on later days.

>Pangram can detect AI-generated content in both short-form and long-form written content, with up to 99% accuracy. [0] Yeah, 0% accuracy is still "up to 99%"! [0] https://www.pangram.com/

I put some article content to Pangram. Pangram says it's AI https://www.pangram.com/history/282d7e59-ab4b-417c-9862-10d6331a9dfd/?… The author's writing style is really similar to AI. AI already somehow passed Turing test. The AI detectors are not that trustworthy (but still useful).

That seems somewhat similar to perplexity based detection, although you can just get the probabilities of each token instead of picking n-best, and you don't have to generate. It kinda works, but is not very reliable and is quite sensitive to which model the text was generated with. This page has nice explanations: https://www.pangram.com/blog/why-perplexity-and-burstiness-fail-to-det…

I ran the introduction chapter through Pangram [1], which is one of the most reliable AI-generated text classifiers out there [2] (with a benchmarked accuracy of 99.85% over long-form text), and it gives high confidence for it having been AI-generated. It's also very intuitively obvious if you play a lot with LLMs. I have no problem at all reading AI-generated content if it's good, but I don't appreciate dishonesty. [1]: https://www.pangram.com/ [2]: https://arxiv.org/pdf/2402.14873

https://www.pangram.com/history/f119a8ee-7e60-442a-8d0c-3fa0fe96d4b7/ I show this more of as a discussion point than as a definitive answer. There is more research on this tool than others: https://www.nature.com/articles/d41586-025-02936-6 And I think it's interesting that it flags it as confidently AI generated. I also got a whiff of AI and I'm never sure how to take confirmation bias from AI detectors - though that said, I've gotten a whiff of AI before and had this detector say it's confidently human. Reading this I kept waiting for the... point? I feel like the whole thing was more like saying "UDP is like a convertible because you can strap a tarp on top when it rains". Like... sure? But that tarp is going to be crappy compared to a real roof. And the idea that that tiny layer is "the best of both worlds" is frankly ridiculous to me.

Look, I appreciate the article, I believe these are important news worth discussing, but was there really no way to write it without LLMs? I'm sure there are enough proponents of "what's important is the content, not who and how exactly typed it," but my brain has developed something like ad banner blindness but for LLM writing - it just shuts off as soon as it sees "this distinction matters" or "does not automatically guarantee." This is not to mention that LLM generated texts are just really confusing and difficult to read, take for example this sentence: > Linux is not explicitly banned. Desktop Linux users could access a website and scan a QR code using a supported mobile wallet. At least two people here agreed it's a really weird way to put this, but at this point I'm honestly inclined to believe that this is just the product of AI putting together some words that kind of sound on-topic even though they are actually meaningless in the context (a.k.a. AI slop), and not someone's actual thought. --- 100% AI pangram: https://www.pangram.com/history/8c60b6ec-cd89-41fb-b8c8-abc096973f47

This product does not even have a plausible theory of how it could work. LLM-generated text does not carry a watermark or other identifying marks. The "theory" is that an LLM trained on human writing, to mimic human writing, can be distinguished from actual human writing in under 100 words. Notably the first diagram on the research overview page (https://www.pangram.com/research/how-it-works) shows feedback for "misclassified human examples." This is a category error; Pangram will not find out when it has misclassified text in the wild, except in rare cases. Only the "licensed human-written text" in its training data can be used as feedback. Scams like Pangram also cause real harms, mostly because laypeople do not understand that what is being offered is not possible. Pangram advertises 99.98% accuracy, and they pitch it as a tool for teachers and universities. Translated: if a college like University of Alabama rolled this out, you could expect ~40 students to have their lives upended by this snake oil, every year. And this is the best case, using the number on Pangram's homepage.

I was surprised not to see any discussion on whether the author used AI to help write this post. As many people say, writing is thinking. I started getting that "I'm reading another AI-written blog post" feeling around 1/3 of the way through, but I don't consider myself super calibrated on this. Pangram seems pretty confident it's AI (https://www.pangram.com/history/e9f6eb77-86f9-46d0-a6c1-e57ce418c6d7). But I know these tools aren't perfect. I'd love to hear from the author what their process was in writing this piece! Related question (I'm trying to work this out for myself): If you believe using AI to write an email or blog post for you isn't okay, but using AI to write code for you is... what's the difference? Right now my instinct is something like: - Code can be verifiably correct (especially w/ good tests) so it's less of a purely-creative act than writing. - But always, always double-check the tests! - I still wouldn't submit a PR where I can't vouch for every line of code. - AI-written documentation and specs are mostly still bad and should be looked down upon. But mostly because the quality, at least today, is poor. (Lots of duplication, lack of a clear understanding of the reader's intent and needs, no thoughtful curation, etc.) - Be psychologically ready to update these priors as models change. I'd love to hear from anyone who's thought more about this.

I'm not affiliated with it in any way, but you might want to check out the leading model https://arxiv.org/abs/2402.14873 and the mixed authorship detection https://www.pangram.com/blog/pangram-3-0-technical. LLM detection has come a long way since the first GPTZero. Human heuristics - I've prompted millions of tokens across every frontier model iteration for all manner of writing styles and purposes - also helps greatly. Concerning to me are long-time posters who (perhaps unknowingly) advance the decline of this human community by encouraging the people breaking HN guidelines. Perhaps spending a few hours on Moltbook might help develop such a heuristic, since "someone who knows how to write" is just a Claude model with a link to the blogpost. https://hn.algolia.com/?dateRange=all&page=0&prefix=false&… Thanks for your comment!

I don't know why you think you can lie about this. https://www.pangram.com/history/3e369754-75da-4922-8386-c0f3b360247f?u… (Cue pivot to "oh, I only meant that it wasn't slop, I never pretended that I wrote a single word of it.")

Reading this, a lot of the sentence construction felt like LLM slop and it seems like Pangram agrees. https://www.pangram.com/history/35257829-ad18-4128-9d61-fa030ed65271?u… Adding slop text makes even the core idea feel like LLM slop, even if the idea was original thought by the authors. I would also prefer if they add a heading to this article mentioning that they used LLM’s writing it rather than just not mentioning it anywhere.

Pangram has their FP rate and FN rate rates here: https://www.pangram.com/research/model-card/pangram-4 > Pangram 4 achieves a 0.0041% false positive rate (roughly 1 in 24,000) on 1,000,000 human-written English FineWeb evaluation examples > Overall False Negative Rate is 0.3396% on English AI generations (26 generator models)

https://www.pangram.com/history/c837bbcb-4aed-43e1-987c-afe25ad4dde2?u… I know you're being downvoted, because people are tired of hearing "this is AI". But I think your comment "this is AI" is warranted. This content, explaining something of cultural, personal significance (instead of the typical technical explainer) should really be written by a human.

Does anyone have the original source of this information? This article seems to be AI generated: https://www.pangram.com/history/fdf6dfd6-1a9c-4267-a90c-a3cf2711c0f5

pangram says this is AI generated and has a very, very, VERY low rate of saying something is AI generated wrong. https://www.pangram.com/history/0b397596-fa85-4869-bae3-2b1da3d2e423?u…