Sean Heelan

The reasoning does not need to be 100% accurate to be useful. Humans are rarely 100% accurate at anything, and yet over time we can build up large models of problems using verification and review. We can do the exact same thing with LLMs. The best example of this is Sean Heelan, who used o3 to find a real security vulnerability in the Linux kernel: https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899… Sean Heelan ran o3 100 times, and it found a known vulnerability in 8% of runs. For a security audit, that is immensely useful, since an expert can spend the time to look at the results from a dozen runs and quickly decide if there is anything real. Even more remarkably though, this same testing exposed a zero-day that they were not even looking for. That is pretty incredible for a system that makes mistakes. This is why LLM reasoning absolutely does not need to be perfect to be useful. Human reasoning is inherently flawed as well, and yet through systems like peer review and reproducing results, we can still make tremendous progress over time. It is just about figuring out systems of verification and review so that we don't need to trust any LLM output blindly. That said, greater reliability would be massively beneficial to how easy it is to get good results from LLMs. But it's not required.

Very well put. The trick is to do either of the following: 1. Find simpler tasks for which the trust in LLMs is high. 2. Give tasks to the LLMs that have a very low cost to verify (even when the task is not simple) - particularly one off scripts. I once had a colleague who was in the "not trust" bucket for the work we were doing. So we found something he was good at that was a pain for me to do, and re-assigned him to do those things and take that burden off of us. In the last few months I've had the LLM solve (simple) problems via code that had been in my head for years. At any point I could have done them, but they were a chore. If the LLM failed for one of these tasks - it's not a big deal - not much time was lost. But they tend to succeed fairly often, because they are simple tasks. I almost never let the LLM write production code, because of the extra burden that you and others allude to. But I do let it write code I rely on in my personal life, because frankly I tend to write pretty poor code for my personal use - I can't justify the time it would take to write things well - life is too busy. I welcome the code quality I get from Sonnet or Gemini 2.5 Pro. That's my point in this thread. Writing code is a pretty diverse discipline, and many are dismissing it simply because it doesn't do one particular use case (high quality production code) well. I didn't take LLM coding seriously until I found well respected, well known SW engineers speak positively about…

People have found 0days assisted by LLMs for a while, and none of them wrote hype pieces to find an excuse not to release their 10x bigger model in the middle of a GPU shortage. https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899…

> LLMs agents are not exploiting kernels to get out of the sandbox. You can't assume that. Attackers with LLMs have enough capabilities to engineer them to build exploits for kernel vulnerabilities [0] or to bypass sandboxes to exfiltrate data [0] in covert ways. It is completely possible to craft a chained attack for an agent to bypass sandboxes even with or without a kernel exploit. From [0] and [1] [0] https://sean.heelan.io/2026/01/18/on-the-coming-industrialisation-of-e… [1] https://www.promptarmor.com/resources/claude-cowork-exfiltrates-files

I learned from a recent post (https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899… that finding security issues can take 100+ calls to an LLM to get good signal. So I wonder about agent implementers who are trying to get good signal out of single calls, even if they are specialized ones.

I find this hard to believe- how would you even know if someone used AI in producing a PR or indie product? Are you omniscient? Further, there are articles here on HN all the time about people using AI for actual serious work. Heres a pretty significant example : https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899…

My takeaway - from this article, from Google’s AlphaEvolve [1], and the recent announcement about o3 finding a zero day in the Linux kernel [2] - is that Gemini Pro 2.5 and o3 in particular have reached a new level of capability where these ideas that were tried unsuccessfully with other models, suddenly just work. [1] https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-cod… [2] https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899…

I wrote to him as a comment under his post > sebstefan: There’s this one from yesterday >https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899… > David Gerard: > My experiment harness executes this N times (N=100 for this particular experiement) and saves the results > That’s just fuzzing but vastly less efficient? > Also, that’s not the question being asked, is it. It wasn’t “did someone use an AI for anything in open source.” I'll try to explain it to him but the guy seems pretty full of shit already

I find it extremely useful as a research tool. It can talk to probably over 100 models at this point, providing a single interface to all of them and logging full details of prompts and responses to its SQLite database. This makes it fantastic for recording experiments with different models over time. The ability to pipe files and other program outputs into an LLM is wildly useful. A few examples: llm -f code.py -s 'Add type hints' > code_typed.py git diff | llm -s 'write a commit message' llm -f https://raw.githubusercontent.com/BenjaminAster/CSS-Minecraft/refs/hea… \ -s 'explain all the tricks used by this CSS' It can process images too! https://simonwillison.net/2024/Oct/29/llm-multi-modal/ llm 'describe this photo' -a path/to/photo.jpg LLM plugins can be a lot of fun. One of my favorites is llm-cmd which adds the ability to do things like this: llm install llm-cmd llm cmd ffmpeg convert video.mov to mp4 It proposes a command to run, you hit enter to run it. I use it for ffmpeg and similar tools all the time now. https://simonwillison.net/2024/Mar/26/llm-cmd/ I'm getting a whole lot of coding done with LLM now too. Here's how I wrote one of my recent plugins: llm -m openai/o3 \ -f https://raw.githubusercontent.com/simonw/llm-hacker-news/refs/heads/ma… \ -f https://raw.githubusercontent.com/simonw/tools/refs/heads/main/github-… \ -s 'Write a new fragments plugin in Python that…