One Useful Thing

For the last few months Ethan Mollick has been writing some great articles about how to best use Generative AI in his blog: https://www.oneusefulthing.org/

This was pretty cool, knocking out a problem that the best minds in maths couldn't for 80 years: https://openai.com/index/model-disproves-discrete-geometry-conjecture/ Also this is a remarkable (and realistic) evaluation of where these systems are for general work which speaks to both the room to grow as well as the pace: https://www.remotelabor.ai/ For some practical examples of what the leading consumer grade AI can do, Ethan Mollick consistently has great writeups with demos: https://www.oneusefulthing.org/p/what-it-feels-like-to-work-with-mytho…

The DeepSeek v3 paper claims to have trained from scratch for ~$5.5m: https://arxiv.org/pdf/2412.19437 Kimi K2 Thinking was reportedly trained for $4.6m: https://www.cnbc.com/2025/11/06/alibaba-backed-moonshot-releases-new-a… Both of those were frontier models at the time of their release. Another interesting number here is Claude 3.7 Sonnet, which may people (myself included) considered the best model for several months after its release and was apparently trained for "a few tens of millions of dollars": https://www.oneusefulthing.org/p/a-new-generation-of-ais-claude-37

This is truly tremendous to watch. Eleven years from TPP, and we're watching the current best-in-class AI try its best at the same. Who'll get there first, the historical gestalt of Twitch users or the just-shy-of-10^26 FLOPS [0] AI model? Now here's a concept for anyone with more money than sense: ClaudePlaysTwitchPlaysPokemon, where it's TPP but every participant is Claude. Would hivemind AI consensus perform better than a single AI? Anthropic's certainly looking into it! [1] [0]: https://www.oneusefulthing.org/p/a-new-generation-of-ais-claude-37 [1]: https://www.anthropic.com/news/visible-extended-thinking

Not mentioned yet, Ethan Mollick's One Useful Thing: https://www.oneusefulthing.org/ It helped me get up to speed with gen AI as a graduate school professor and now his posts are the most useful ones I sent to others to help them get oriented.

You should not think that LLMs are a good fit for anything. You should check if they are, and then try to guide it to be better at the task. You should do this because AI is a jagged frontier where humans can't predict if AI is good at something or not. >AI is weird. No one actually knows the full range of capabilities of the most advanced Large Language Models, like GPT-4. No one really knows the best ways to use them, or the conditions under which they fail. There is no instruction manual. On some tasks AI is immensely powerful, and on others it fails completely or subtly. And, unless you use AI a lot, you won’t know which is which. https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged

Yes, a great way to think of it is as a widely read intern: https://www.oneusefulthing.org/p/on-boarding-your-ai-intern You’ve still got to avoid prompting for questionable code in the first place, eg, splitting SQL statements on semicolons with an ad-hoc regex is going to fail in edge cases, but may be sufficient for a specific task.

Huh. They're not "Pelicanmaxxing"... they're Ottermaxxing. Take a look at the GLM 5.2 and Deepseek V4 "animal on a plane" examples. In every case, the animal is standing on top of the plane, a clear misunderstanding of the concept of "animal on a plane"... with the exception of the Otter. The otters are sitting in a seat on the plane, looking out the window. That's Ethan Mollick's "Otter On A Plane Using WiFi" image benchmark. https://www.oneusefulthing.org/p/the-recent-history-of-ai-in-32-otters (Sometimes the Racoon is sitting inside the plane as well, but the racoon is a common backup benchmark. I'm surprised it wasn't also holding a sign saying that it loves trash.) Also, Grok seemed to really really enjoy "whale on a plane" in that second round, and kudos to GPT Terra for deciding after 3 rounds that the user was terrible at spelling and generated "Antelope On A Plain". EDIT: I promise I'm a human, but I did just notice my "that's not x... that's y" construction at the start. I am rather Claudepilled — my apologies.

Sorry, when I say I think I have a balanced take on AI what I mean is that I do my best to weigh both the pros and cons of this technology as opposed to a more extreme behavior like spending all day chatting with LLMs or posting all day on X about how AI is already better than me at everything and that jobs are over. If I had to assign a confidence score for whether agents will change the way we all work and many aspects of how we live, I would put it at a 7/10, maybe 8/10. I felt about the same about the smartphone. While many things we do look the same way they did in 2005 (we still drive on roads, kids still go to school), at the same time it's undeniable that much of our lives are intermediated through a small screen and many societal dynamics have shifted due to that technology's existence. I will concede that you should read my post with that context and draw your own conclusions about the veracity of my perspective — but I think it is more well-reasoned than what people generally attribute to "LLM hype". (Of course it's a bit tautological that I believe that, but I try to surround myself with people of all kinds technical and non-technical and like to think I stay reasonably grounded.) All that said, I think the code from a leading company being bad and yet delivering good results is more a sign of the technology's jagged frontier[^1]. Calculators can't write sonnets the same way that LLMs are bad at math, but that doesn't make them useless — it just makes them a…

If you don't want to read the full thing, Mollick's substack has more accessible summary of this paper: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged There are two sides to this thing. The first author of this paper has also written another paper titles "Falling Asleep at the Wheel: Human/AI Collaboration in a Field Experiment". Abstract: "As AI quality increases, humans have fewer incentives to exert effort and remain attentive, allowing the AI to substitute, rather than augment their performance... I found that subjects with higher quality AI were less accurate in their assessments of job applications than subjects with lower quality AI. On average, recruiters receiving lower quality AI exerted more effort and spent more time evaluating the resumes, and were less likely to automatically select the AI-recommended candidate. The recruiters collaborating with low-quality AI learned to interact better with their assigned AI and improved their performance. Crucially, these effects were driven by more experienced recruiters. Overall, the results show that maximizing human/AI performance may require lower quality AI, depending on the effort, learning, and skillset of the humans involved." https://static1.squarespace.com/static/604b23e38c22a96e9c78879e/t/62d5…

The new LLM centered workflow is really just a management job now. Managers and project managers are valuable roles and have important skill sets. But there's really very little connection with the role of software development that used to exist. It's a bit odd to me to include both of these roles under a single label of "builders", as they have so little in common. EDIT: this goes into more detail about how coding (and soon other kinds of knowledge work) is just a management task now: https://www.oneusefulthing.org/p/management-as-ai-superpower/

This is the "Wait Calculation" and it's fiendish because there exists only some small, finite window in which it is indeed better to start before the tech is "better" in order to "win" (i.e. get "there" first, wherever "there" is in your scenario). Here's a nice article about it: https://www.oneusefulthing.org/p/the-lazy-tyranny-of-the-wait-calculat…

> It's very good at some kinds of reasoning and very bad at others. There's not much it's mediocre at. Yes, one of the terms for this phenomenon is "jagged frontier" from here: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged This is also the main reason people have such diverging experiences and the discussion around AI is so bimodal in nature.

In the interests of gathering these pre-release impressions, here's Ethan Mollick's writeup: https://www.oneusefulthing.org/p/gpt-5-it-just-does-stuff Thank you to Simon; your notes are exactly what I was hoping for.

A while back, someone here linked to this story[0]. It's a bit simplified and idealized, but is actually fairly spot-on. I have been using AI every day. Just today, I used ChatGPT to translate an app string into 5 languages. [0] https://www.oneusefulthing.org/p/superhuman-what-can-ai-do-in-30-minut…

See also: The recent history of AI in 32 otters https://www.oneusefulthing.org/p/the-recent-history-of-ai-in-32-otters

I was at Penn's first AI conference last year and heard Dr Lilach Mollick's keynote where she said this is shown to be true over and over. She doesnt seem to publish often, but her husband Ethan always has a lot to say about AI. https://www.oneusefulthing.org/p/everyone-is-above-average

I signed up for Gemini Advanced to get access to Deep Research on Feb 1st and felt on top of the world. So cool, so advanced. Then OpenAI announced theirs on the 2nd: https://openai.com/index/introducing-deep-research/ Ethan Mollick called Google's undergraduate level and OpenAI's graduate level on the 3rd: https://www.oneusefulthing.org/p/the-end-of-search-the-beginning-of And now this. I can't stop thinking about The Onion Movie's "Bates 4000" clip: https://www.youtube.com/watch?v=9JCOBPMIgAA

Yup. But I'm not convinced it'll be a Gartner type hype cycle https://emt.gartnerweb.com/ngw/globalassets/en/articles/images/hype-cy… rather than a Wait But Why wtf https://149909199.v2.pressablecdn.com/wp-content/uploads/2015/01/Intel… Indeed the hype/whatever already is carrying on - eg. "Researchers began speaking urgently about the arrival of supersmart AI systems, a flood of intelligence. Not in some distant future, but imminently." 10-jan-25 https://www.oneusefulthing.org/p/prophecies-of-the-flood

related: Post-apocalyptic education What comes after the Homework Apocalypse by Ethan Mollick https://www.oneusefulthing.org/p/post-apocalyptic-education

• Bard when launched was using one of the PaLM models, which is inferior to GPT-3.5 (the free version of ChatGPT). • Bard since a couple of months ago was using Gemini Pro, which is roughly comparable (a bit worse or better depending on whom you ask) to GPT-3.5. • Bard is now (today) called Gemini, available as "Gemini" and "Gemini Advanced". The former is still comparable to the free version of ChatGPT, and the latter version costs $20/month and uses the "Gemini Ultra" model, and is meant to be roughly comparable to GPT 4 / the paid version of ChatGPT. (Their paper claimed it to be better than GPT 4 on some benchmarks, but real-world usage will show which way it goes — but it should be significantly better than Bard from recently. Edit: See https://www.oneusefulthing.org/p/google-gemini-advanced-tasting-notes from someone who's been using it for six weeks.)

Important note: Bing balanced mode (default) uses GPT 3.5 Only Precise and Creative modes use GPT-4 https://twitter.com/emollick/status/1732495030143549541 Also see: An Opinionated Guide to Which AI to Use: ChatGPT Anniversary Edition https://www.oneusefulthing.org/p/an-opinionated-guide-to-which-ai

More details in this blog post by a Wharton professor: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged My questions to naysayers: * Do you or anyone you know use GPT-4 (not the free GPT-3.5) to do productive tasks like coding and found it to help in many cases? * If you insist it’s useless, why do millions of people pay $20 a month to access GPT-4 and plugins?

A blog post by one of the authors describing this paper is available here, with some interesting data: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged

A very good article on the (early) capabilities by Ethan Mollick: https://www.oneusefulthing.org/p/it-is-starting-to-get-strange

Doesn't necessarily apply to data being processed with plugins. The "Code Interpreter" plugin lets you upload files up to 100 MB for use by AI-generated Python scripts; see https://www.oneusefulthing.org/p/it-is-starting-to-get-strange for examples (including useful output from vague prompts like "please characterize this dataset", at least if you get lucky).

LLMs can definitely produce original other stuff: ask it to create an original poem and on an extremely specific niche subject and it will do so. You can specify the niche subject to the point where it is incredibly unlikely that there is a poem on that subject in its training data, and it will still produce an original poem on that subject [0]. The well-known "otter using wifi on a plane" series of images [1] is another example: this is not in the training data (well, it is now, because well-known, but you get the idea). Is there something unique about code, that is different from language (or images), that would make it impossible for an LLM to produce original code? I don't believe so, but I'm willing to be convinced. I think this switches the burden of proof: we know LLMs can produce original content in other contexts. Why would they not be able to create original code? [0] Ever curious, I tested this assumption. I got Claude to write an original limerick about goats oiling their beards with olive oil, which was the first reasonable thing I could think of as a suitably niche subject. I googled the result and could not find anything close to it. I then asked it to produce another limerick on the same subject, and it produced a different limerick, so obviously not just repeating training data. [1] https://www.oneusefulthing.org/p/the-recent-history-of-ai-in-32-otters

This is just me, and I find that I go through periods of listening to podcasts and then it falls off my routine. When I would think about it, listening to podcasts was low intensity edutainment, it feels like you are learning something but you really aren't, or the learning density is so low that most 45 minute podcasts could be summarized in 10 bullet points. It was not unlike watching a BBC planet earth nature show. Afterwards you know that a bird somewhere can spin its head around twice without breaking its neck but not much more. Oxford Researchers Discovered How to Use AI To Learn Like A Genius https://www.youtube.com/watch?v=TPLPpz6dD3A Use of generative AI tools to support learning https://www.ox.ac.uk/students/academic/guidance/skills/ai-study https://www.oneusefulthing.org/p/assigning-ai-seven-ways-of-using https://www.sciencedirect.com/science/article/pii/S0099133323000599?vi… Bloom's Taxonomy https://www.phind.com/search/cm6o98lki00002e6g5anos0bv