Mike Judge

Not a scientific study, but someone did replicate the experiment on themselves [0] and found that in their case, any effect from LLM use wasn't detectable in their sample. Notably they almost certainly had more experience with LLMs than most of the METR participants did. [0] https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co…

I hadn't heard the app store submission stats. Does this answer Mike Judge's question of where the shovelware is [0,1]? Did we just need to wait a few months? [0] https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… [1] https://news.ycombinator.com/item?id=46262545

Do you have any example repos of these OSS projects? I'm being reminded of this post every time people keep extolling how "productive" LLMs are: https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… Where is the resulting software?

He's just parroting stuff done by this guy: https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… Shouldn't it be the "Judge Paradox" ???

Source? See contradicting data here: https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co…

The link in the last paragraph provides some data to back up the claim. https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… - If the goal is to increase the rate of software production, there isn't much evidence that AI has moved the needle. Sure, code gen is faster now. And the industry might finally be waking up to the fact that writing code is a small part of producing software. Getting infinitely faster at one step doesn't speed up the overall process. In fact, there's good evidence it that rapid code gen actually slows down other steps in the process like code review and QA.

The best argument I've yet heard against the effectiveness of AI tools for SW dev is the absence of an explosion of shovelware over the past 1-2 years. https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… Basically, if the tools are even half as good as some proponents claim, wouldn't you expect at least a significant increase in simple games on Steam or apps in app stores over that time frame? But we're not seeing that.

I think you mean this one: Where's the shovelware? Why AI coding claims don't add up https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… https://news.ycombinator.com/item?id=45120517

Data so far says that it doesn't https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co…

my dear guy, where is the shovelware https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… Where's the Shovelware? Why AI Coding Claims Don't Add Up

Good example of shovelware that some say is absent.¹ ¹ https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co…

> The ROI is clear if it delivers even 10% productivity gain What if what feels like a productivity gain is actually a productivity loss? https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… (see link in the article to a study showing developers thought AI gave them a 20% gain in productivity, but measuring this showed they instead had a 20% loss)

> The game has change. The ‘how’ we build it is easy. The ‘what’ we build is and always has been the hardest part of any SaaS or business. This is what the promptfondlers don't want to admit: the how has been easy for a long time. This last, I dunno, 35 years or so, Visual Basic, Delphi, whatnot, producing code has been very easy. You don't need a fundamentally fascist probabilistic nightmare to do it. The hard problems are indeed is "what" to build and how we maintain it. There's only hype. There's no results. https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… As for fascism, check https://blog.bgcarlisle.com/2025/05/16/a-plausible-scalable-and-slight… for example > By “fascist” in this context, I mean that it is well suited to centralizing authority, eliminating checks on that authority and advancing an anti-science agenda. Or check Woodrow Hartzog & Jessica Silbey, How AI Destroys Institutions , 77 UC Law Journal (2026). Available at: https://scholarship.law.bu.edu/faculty_scholarship/4179

Good question. Why hasn't there been a profusion of new game-changing software, fixes to long-standing issues in open-source software, any nontrivial shipped product at all? Heck, why isn't there a cornucopia of new apps, even trivial ones? Where is all the shovelware [0]? Previous HN discussion here [1]. Don't get me wrong, AI is at least as game-changing for programming as StackOverflow and Google were back in the day. I use it every day, and it's saved me hours of work for certain specific tasks [2]. But it's simply not a massive 10x force multiplier that some might lead you to believe. I'll start believing when maintainers of complex, actively developed, and widely used open-source projects (e.g. ffmpeg, curl, openssh, sqlite) start raving about a massive uptick in positive contributions, pointing to a concrete influx of high-quality AI-assisted commits. [0] https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co… [1] https://news.ycombinator.com/item?id=45120517 [2] https://news.ycombinator.com/item?id=45511128

I'm getting real "put everything on the blockchain" vibes from answers like this. I remember when folks were telling me that hospitals were going to put patient records on the blockchain. As for radiology, it doesn't seem this use of AI is as much of a "slam dunk" as it first appeared[1][2]. We'll see, I guess. Right now I kind of land on the side of "Where is all the shovelware?". If AI is such a huge productivity boost for developers, where is all the software those developers are supposedly writing[3]? But this is just a microcosm of a bigger question. Almost all the economic growth since the AI boom started has been in AI companies. If AI is revolutionizing multiple fields, why aren't relevant companies those fields also growing at above-expected rates? Where's all this productivity that AI is supposedly unlocking? [1] https://hms.harvard.edu/news/does-ai-help-or-hurt-human-radiologists-p… [2] https://www.ajronline.org/doi/10.2214/AJR.24.31493 [3] https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-co…

I think we agree no the limitations of the study--I literally began my comment with "for seasoned maintainers of open source repos". I'm not sure if in your first statement ("there are no studies to back up this claim.. I welcome good analysis") you are referring to claims that support an AI-speedup. If so, we agree that good analysis is needed. But if you think there already is good data: Can you link any? All I've seen is stuff like Anthropic claiming 90% of internal code is written by Claude--I think we'd agree that we need an unbiased source and better metrics than "code written". My concern is that whenever AI usage in professional developers is studied empirically, as far as I have seen, the results never corroborate your claim: "Any developer (who was a developer before March 2023) that is actively using these tools and understands the nuances of how to search the vector space (prompt) is being sped up substantially." I'm open to it being possible, but as someone who was a developer before March 2023 and is surrounded by many professionals who were also so, our results are more lukewarm than what I see boosters claim. It speeds up certain types of work, but not everything in a manner that adds up to all work "sped up substantially". I need to see data, and all the data I've seen goes the other way. Did you see the recent Substack looking at public Github data showing no increase in the trend of PRs all the way up to August 2025? All the hard data I've seen is much,…