Dwarkesh Patel

Its amazing to see this beast and think that all the firey stuff isn't the primary energy source (although totally necessary to pushing the mass out the back fast enough). The reason it’s very small is that almost all of the energy in hydrogen and oxygen is not stored in the chemical binding energy of these things going together. The vast majority of it is stored in just the rest mass energy of the protons and the neutrons, which chemical burning doesn’t affect at all. https://www.dwarkesh.com/p/adam-brown-gr [Edit note: Starship and Booster are methalox, yes. The physicist is talking generically, not about this launch specifically)

Nice presentation of the list! I'd recommend watching a few of his talks/podcasts before during reading these to get the overview and how all the bits in these works tie together. https://www.dwarkesh.com/p/ilya-sutskever https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023-08-14 https://www.dwarkesh.com/p/ilya-sutskever-2

From the recent-ish Dwarkesh podcast, Anthropic seems to be wary about buying/building too much compute [0]. That probably means that they have to attempt to minimize compute usage when there is a surge in demand. Following the argument in the podcast, throwing more money after them, as some in this thread are suggesting, won’t solve the issue, at least not in the short term. [0] https://www.dwarkesh.com/i/187852154/004620-if-agi-is-imminent-why-not…

I would strongly recommend this podcast episode with Andrej Karpathy. I will poorly summarize it by saying his main point is that AI will spread like any other technology. It’s not going to be a sudden flash and everything is done by AI. It will be a slow rollout where each year it automates more and more manual work, until one day we realize it’s everywhere and has become indispensable. It sounds like what you are seeing lines up with his predictions. Each model generation is able to take on a little more of the responsibilities of a software engineer, but it’s not as if we suddenly don’t need the engineer anymore. https://www.dwarkesh.com/p/andrej-karpathy

> It’s not “free tier” it’s all the tiers being significantly discounted (underpriced). It’s ALL pricing. That's possible, but how would we know what the cost to provide the service is from any of the companies, which will anyways vary from company to company since they are using different models and hardware (e.g. OpenAI/Microsoft NVIDIA, Google TPUs, Anthropic/Amazon Trainium). We also don't know how much the average chat subscriber is using the service. A low usage subscriber may be profitable at $20/mo, but a high usage one unprofitable. Chat subscription prices seem to have been steady at $20/mo for a while, but cost of providing the service is falling very fast. In recent Dwarkesh interview, Satya Nadella says costs are falling 5/10/40x year over year, or evern quarter over quarter. Here, at 1:10:56 https://www.dwarkesh.com/p/satya-nadella-2

This is actually discussed in the interview: https://www.dwarkesh.com/i/176425744/llm-cognitive-deficits It seems to be more nuanced than what people have assumed. The best I can summarize it as is that he was doing rather non-standard things that confused the LLMs which have been trained on vast amounts of very standard code and hence kept defaulting to those assumptions. Maybe a rough analogy is that he was trying to "code golf" this repo while LLMs kept trying to write "enterprise" code because that is overwhelmingly what they have been trained on.

Zuckerberg explains a few of the reasons here: https://www.dwarkesh.com/p/mark-zuckerberg#:~:text=As%20long%20as%20it… The short version is that is you give a product to open source, they can and will donate time and money to improving your product, and the ecosystem around it, for free, and you get to reap those benefits. Llama has already basically won that space (the standard way of running open models is llama.cpp), so OpenAI have finally realized they're playing catch-up (and last quarter's SOTA isn't worth much revenue to them when there's a new SOTA, so they may as well give it away while it can still crack into the market)

Do you think it's akin to Ilya's [1] claim that next token prediction is reality? E.g. any deeper claims about the structure of that intelligence or comparing to humans? To be clear, I'm 100% with you that "next token predictor" is stupid to call what these machines are now. We are engineers and can shape the capability landscape to give rise to a ton of emergent behavior. It's kind of amazing. In that sense, being precise about what's going on, rather than being essentialist (technically, yes, the 'actual' algorithm, whatever that even means, is text prediction), is just good epistemology. I still think it's still a very interesting question though to ask about deeper emergent structures. To me, this is evidence of a more embedded cognition kind of theory of intelligence (admittedly this is not very precise). But IDK how into philosophy you are. [1] https://www.dwarkesh.com/p/ilya-sutskever

This is a super interesting dynamic! The CCP is really good at subsidizing and flooding global markets, but in the end, it takes power to generate tokens. In my Uber comparison, it was physical hardware on location... taxis, but this is not the case with token delivery. This is such a complex situation in that regard, however, once the market settles and monopolies are created, eventually the price will be what market can bear. Will that actually create an increase in gross planet product, or will the SOTA token providers just eat up the existing gross planet product, with no increase? I suppose whoever has the cheapest electricity will win this race to the bottom? But... will that ever increase global product? ___ Upon reflection, the comment above was very likely influenced by this truly amazing quote from Satya Nadella's interview on the Dwarkesh podcast. This might be one of the most enlightened things that I have ever heard: > Us self-claiming some AGI milestone, that's just nonsensical benchmark hacking to me. The real benchmark is: the world growing at 10%. https://www.dwarkesh.com/p/satya-nadella#:~:text=Us%20self%2Dclaiming%…

Efficient markets route around bottlenecks. Technological revolutions accelerate the speed at which that re-routing happens. In software, we, the developers, have increasingly been a bottleneck. The world needs WAY more software than we can economically provide, and at long last a technology has arrived that will help route around us for the benefit of humanity. Here's an excellent Casey Handmer quote from a recent Dwarkesh episode: > One way to think about the industrial revolutions is [...] what you're doing is you're finding some way of bypassing a constraint or bypassing a bottleneck. The bottleneck prior to what we call the Industrial Revolution was metabolism. How much oats can a human or a horse physically digest and then convert into useful mechanical output for their peasant overlord or whatever? Nowadays we would giggle to think that the amount of food we produce is meaningful in the context of the economic power of a particular country. Because 99% of the energy that we consume routes around our guts, through the gas tanks of our cars and through our aircraft and in our grids and stuff like that. > Right now, the AI revolution is about routing around cognitive constraints, that in some ways writing, the printing press, computers, the Internet have already allowed us to do to some extent. A credit card is a good example of something that routes around a cognitive constraint of building a network of trust. It's a centralized trust. It's a great episode, I…

I thought Dwarkesh's position was pretty thoughtful: https://www.dwarkesh.com/p/dow-anthropic > Nobody is qualified to steward the development of superintelligence. It is a terrifying, unprecedented thing that our species is doing right now, and the fact that private companies aren’t the ideal institutions to take up this task does not mean the Pentagon or the White House is. > The only way we can preserve our free society is if we make laws and norms through our political system that it is unacceptable for the government to use AI to enforce mass surveillance and censorship and control. Just as after WW2, the world set the norm that it is unacceptable to use nuclear weapons to wage war.

I think a lot of LLM progress in the last 2 years or so is really from densifying the training data to niches like this and not so much from general problem solving capability. I guess the hope is that the deluge of super specific training data will pay off through better overall competence. cf https://www.dwarkesh.com/p/the-sample-efficiency-black-hole

There’s export controls on nvidia. According to Jensen, they expect that Chinese models will start being optimized for Huawei https://www.dwarkesh.com/p/jensen-huang

Insider tips from Google and AWS telling me that they run inference at a profit (though that was over a year ago now). Dario telling Dwarkesh three months ago that they have a margin on inference: https://www.dwarkesh.com/p/dario-amodei-2?timestamp=3528.0

Increasingly AI's are trained using reinforcement learning [1] so these are not really human tasks but things like trying to prove theorems, play games, solve code problems and getting feedback from compilers and similar. A lot of the early pop science coverage of AI was around the ideas of "data walls" and constrains of human data, most of which just wasn't really true or long term true anyway. [1]. https://www.dwarkesh.com/p/dario-amodei-2

More context on what’s going on with LLMs solving Erdos problems: https://www.dwarkesh.com/p/terence-tao TLDR, most of what is getting solved so far is “easy” problems that were not seriously looked at by experts, and where there isn’t a new insight, just trying all the existing techniques from the toolbox. Essentially the low hanging fruit for automation. Raw count solved is a problematic eval due to its difficulty lumpiness. Seems this problem might be different, having some new insight as part of the solution.

Also see the Daario interview with Dwarkesh: > If every year we predict exactly what the demand is going to be, we’ll be profitable every year. Because spending 50% of your compute on research, roughly, plus a gross margin that’s higher than 50% and correct demand prediction leads to profit. That’s the profitable business model that I think is kind of there, but obscured by these building ahead and prediction errors. (a lot more at the link) https://www.dwarkesh.com/p/dario-amodei-2?open=false#%C2%A7005849-how-…

Holy rabbit hole. SBF was in Patel's previous podcast in July 2022 and FTX unraveled in November 2022. Hmm. https://www.dwarkesh.com/p/sbf > I flew to the Bahamas to interview Sam Bankman-Fried, the CEO of FTX! He talks about FTX’s plan to infiltrate traditional finance, giving $100m this year to AI + pandemic risk, scaling slowly + hiring A-players, and much more. And that was right in the middle of FTX being accused by many prominent people . April 29, 2022 https://x.com/AlderLaneEggs/status/1520023221294145536 June 20, 2022 https://x.com/MartyBent/status/1538645746655936519

Do you really pay so little attention to the space that you think this is all they do? Almost every public discussion or interview involving these figures turns at some point to society's unpreparedness for what's coming, for instance Amodei's interview last week. https://www.dwarkesh.com/p/dario-amodei-2

It was Jeff Dean and Noam Shazeer: https://www.dwarkesh.com/p/jeff-dean-and-noam-shazeer

Different cutoff but similar question thrown out in https://www.dwarkesh.com/p/thoughts-on-sutton#:~:text=If%20you%20train… inspiring https://manifold.markets/MikeLinksvayer/llm-trained-on-data-from-1900-…

There's still not that much information on this, but the closest you'll get is the Dwarkesh-Ilya interview: https://www.dwarkesh.com/p/ilya-sutskever-2

Are you saying that LLMs _do_ have a model of the human mind in their weights? Imagine you use an ARIMA model to forecast demand for your business or the economy or whatever. It's easy to say it doesn't have a 'world model' in the sense that it doesn't predict things that are obvious only if you understand what the variables _mean_ implicitly. But in what way is it different from an LLM? I think Stallman is in the same camp as Sutton https://www.dwarkesh.com/p/richard-sutton

Dylan Patel joined Dwarkesh recently to interview Satya Nadella: https://www.dwarkesh.com/p/satya-nadella-2

,> Pretty sure the article is AI slop Yeah, the actual video with transcripts (YouTube link in bottom of TFA): https://www.dwarkesh.com/p/ilya-sutskever-2 Ed: TFA is basically a dupe of https://news.ycombinator.com/item?id=46048125

There's also an extended article. https://www.dwarkesh.com/p/timelines-june-2025

> I keep hearing that LLMs are trained on "Internet crap" but is it true? Karpathy repeated this in a recent interview [0], that if you'd look at random samples in the pretraining set you'd mostly see a lot of garbage text. And that it's very surprising it works at all. The labs have focused a lot more on finetuning (posttraining) and RL lately, and from my understanding that's where all the desirable properties of an LLM are trained into it. Pretraining just teaches the LLM the semantic relations it needs as the foundation for finetuning to work. [0]: https://www.dwarkesh.com/p/andrej-karpathy

> I think when the environment gets more and more complex, interactions with the environment, and learning from these interactions will matters more and more Thanks for sharing your experience ! It correlates with this recent interview with Sutton [1]. That real intelligence is learning from feedback with a complex and ever changing environment. What an LLM does is to train on a snapshot of what has been said about that environment and operate on only on that snapshot. [1] https://www.dwarkesh.com/p/richard-sutton

This provides a lot of interesting info on geothermal: https://worksinprogress.co/issue/watt-lies-beneath/ And more: https://www.complexsystemspodcast.com/episodes/fracking-austin-vernon/ One interesting point made here is that the cost of turbines puts a floor price on any form.of generation which uses them, whether renewable or not, meaning in the long run solar has a big advantage: https://www.dwarkesh.com/p/casey-handmer. I don't know how accurate that is

I think at that point there is strong financial pressure to figure out how to continuously evolve models instead of changing new ones, for example by building models out of smaller modules that can be trained individually and swapped out. Jeff Dean and Noam Shazeer talked about that a bit in their interview with Dwarkesh: https://www.dwarkesh.com/p/jeff-dean-and-noam-shazeer