Astral Codex Ten

https://www.astralcodexten.com/ is one of the most influential bloggers in tech spaces. I was a keen reader, but don't follow so much anymore. That said, I don't think his influence is that large, but who is? He's likely fawning over Scott because he wants the post to read by readers of Scott. "A spoonful of sugar helps the medicine go down"

Scott Alexander's blog at https://astralcodexten.substack.com is my favorite intelligent reading material. For more in the same style, try https://asteriskmag.com/.

OK, the writer here mentions substack. May be worth signing up for this author or others with NY Times subscription money. https://astralcodexten.substack.com/

Lucky you! It's some of the best content on the internet. My favourite blog for sure. Same guy has continued on with his writing at https://www.astralcodexten.com which is also pretty good but doesn't reach the same highs. What's wrong with it in context though is that as great as it is, it's just some guys blog. It's disconcerting that people would be working on technology they think is more dangerous than nuclear weapons and basing their approach to safety on a random blog post. Although it's disconcerting to think of a committee deciding how it's approached, or the general public, or AI loving researchers, so it might just be a disconcerting topic. If OpenAI or just Ilya think Scott is the best man to have thinking about it though, I would have at least liked them to pay him to do it full time. Blogging isn't even Scott's full time job, and the majority of his stuff isn't even about AI.

Unvaccinated, early 30s with no comorbidities other than having a slightly high BMI. Had it right after thanksgiving of this year. Two week deal. First week was the typical fever/flu symptoms, second week was brain fog and nausea coupled with a loss of taste and smell for about 2 or so weeks from start. Brain fog took 3 weeks to clear from the start. I had shortness of breath but employed breathing exercises and later did some heavy lifting around the house and ended up doing some unexpected cardio. A more structured and disciplined approach would likely yield more measurable, positive results. No shortness of breath after three weeks from start. Not a ride I’d want to go on again, but as other have reported, certainly didn’t live up to the media fear/hype. Worth noting that I did employ a pascalian therapy in my approach (1). I was prescribed ivermectin, took large amounts of quercetin, vitamin c, d, zinc and so on. I think this probably led to me recovering so quickly and not experiencing any lasting side effects. This is totally anecdotal though. 1. https://astralcodexten.substack.com/

As exemplified in the following series of essays https://www.astralcodexten.com/p/the-foothills-of-bay-area-house-party

I don't think the next-token-predictor thing should increase anyone's confidence that LLMs aren't conscious or can't escape the control of their operators. A very closely analogous argument would "prove" that humans aren't conscious or can't do [insert task here] either. (No, I'm not saying that any of this is true of today's LLMs, I'm saying this particular argument doesn't work.) I recommend this explanation: https://www.astralcodexten.com/p/next-token-predictor-is-an-ais-job

I guess most of my positions on prediction markets came from Scott Alexander, e.g. https://www.astralcodexten.com/p/prediction-market-faq which was posted in 2022. The implicit argument seems to be "We should come up with some way to regulate these so that it's less harmful than existing forms of gambling." I think the part that was harder to foresee was the Trump admin suddenly seizing on them as a loophole to override state regulation of sports betting. Certainly there does not seem to have been any careful consultation with prediction-market experts to determine the most robust, socially beneficial laws. I understand that "Things would be great if they'd done it right!" is in some sense unfalsifiable. But also I think it's pretty clear that the current regime is geared solely towards maximizing the profits of prediction market companies and Trump Jr.

If anything, the average terrorist is probably smarter than the average member of their surrounding societies, since you need a minimum amount of education to understand the ideology motivating the terrorism. Don't forget that Taliban literally means "students." The number of times someone tried and failed to conquer Afghanistan suggests that it's not an easy feat that any fool could have done, but the Taliban managed to do it twice. Fighting a much more numerous enemy may seem stupid, but if you really believe your ideology, any odds of success are better than none. Think open-source developers competing with commercial offerings maintained by huge teams. Terrorists building IEDs when their enemy has a full military-industrial complex are much the same. They just really believe their ideology. On this, https://www.astralcodexten.com/p/your-book-review-nine-lives is a good read.

One of the authors talks about this here: https://www.astralcodexten.com/p/introducing-plan-a > It’s increasingly clear that nobody has a plan for if this AI thing turns out to be real. > ... > Plan A isn’t another prediction. It’s a wish list, a positive vision, a road map for navigating the future. > ... > If we’re merely on track for a few cool gee-whiz AI innovations in the 2040s, then I’m wrong about everything and none of this really matters one way or the other. I think their position is: "it would be great if current tech such as LLMs doesn't get us to AGI and only leads to some cool new innovations, but if it does, that's scary, because nobody has a plan for what to do, so here's our plan". The jingoism is off putting. I think Daniel says it's a political necessity: https://x.com/DKokotajlo/status/2075261194978640096

> Problem was, the model was wrong. I thought despite the fraud, it's still the best model we have[1]? The fact there was fraud doesn't mean the model is immediately incorrect. At best, it means its foundations are shakier than we thought, but it's not a slam dunk repudiation. [1] https://www.astralcodexten.com/p/in-defense-of-the-amyloid-hypothesis

thank you for sharing. i typically like his writing but didn't find this one terribly insightful. however, from his follow-up post https://www.astralcodexten.com/p/highlights-from-the-comments-on-posiw… > When people insist on the confusing and inappropriately-strong version, I start to suspect that the confusingness is a feature, letting them smuggle in connotations that people would otherwise correctly challenge. i find this comical because it reads like a slightly-differently-worded POSIWID argument :)

> Nobody will fall in love and start a relationship/family with you for showing up and fulfilling some list of criteria. The "date-me doc" community might disagree: * https://www.nytimes.com/2023/08/02/style/date-me-docs.html * https://www.astralcodexten.com/p/in-defense-of-describable-dating * https://www.astralcodexten.com/p/highlights-from-the-comments-on-datin…

Scott Alexander has a decent article (or rather a guest blog post) at https://www.astralcodexten.com/p/all-lawful-use-much-more-than-you that brings up the subject in the context of Anthropic and OpenAI's dealings with the Department of War (sic), and how their contracts with the DoW might be interpreted with regard to mass surveillance of Americans. Worth checking out. I'm not personally knowledgeable enough to vouch for the veracity, though.

Best summary by far that I've seen: https://www.astralcodexten.com/p/the-pentagon-threatens-anthropic Discussed here: https://news.ycombinator.com/item?id=47154983

This just sounds like the "nothing ever happens" theorem slightly rephrased, of which Scott Alexander did a great refutation here: https://www.astralcodexten.com/p/heuristics-that-almost-always-work

> Because if it’s worth your time to lie, it’s worth my time to correct it. https://www.astralcodexten.com/p/if-its-worth-your-time-to-lie-its

Scott Alexander blogged about it today: https://www.astralcodexten.com/p/best-of-moltbook

No, yeah, obviously, I'm not trying to anthropomorphize anything. I'm just saying this "religion" isn't something completely unexpected or out of the blue, it's a known and documented behavior that happens when you let Claude talk to itself. It definitely comes from post-training / "AI persona" / constitutional training stuff, but that doesn't make it fake! I recommend https://nostalgebraist.tumblr.com/post/785766737747574784/the-void and https://www.astralcodexten.com/p/the-claude-bliss-attractor as further articles exploring this behavior

IYKYK, hilarious. Could be worth starting at the beginning: https://www.astralcodexten.com/p/every-bay-area-house-party

Here is one sentence from the referenced prediction: > I don't think there will be any more AI winters. This isn't enough to qualify as a testable prediction, in the eyes of people that care about such things, because there is no good way to formulate a resolution criteria for a claim that extends indefinitely into the future. See [1] for a great introduction. [1]: https://www.astralcodexten.com/p/prediction-market-faq

I cannot believe this (related) article from 2024 never got posted to HN: (I just posted it now) Why Does Ozempic Cure All Diseases? https://www.astralcodexten.com/p/why-does-ozempic-cure-all-diseases I highly recommend it.

Classical LLM hallucination happens because AI doesn’t have a world model. It can’t compare what it’s saying to anything. You’re right that LLMs favor helpfulness so they may just make things up when they don’t know them, but this alone doesn’t capture the crux of hallucination imo, it’s deeper than just being overconfident. OTOH, there was an interesting article recently that I’ll try to find saying humans don’t really have a world model either. While I take the point, we can have one when we want to. Edit: see https://www.astralcodexten.com/p/in-search-of-ai-psychosis re humans not having world models

See https://www.astralcodexten.com/p/in-defense-of-the-amyloid-hypothesis for an IMO compelling argument that amyloid really is the underlying cause. The theory is that amyloid buildup causes misfolded tau proteins which cause the cognitive damage. So reducing amyloid in people already suffering from Alzheimer's doesn't do much because they already have excessive tau; at best it might slow down the progression.

Possibly - or we might discover that for a large amount of use cases, AI output is "good enough," even if the experts don't love it. A C programmer will snub an Excel sheet, but it doesn't change the fact that it's genuinely useful for a wide number of use cases. As an aside, the AI Art Turing Test was a bit eye-opening for me: https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing I say all of this as someone who hopes that art remains human.

https://www.astralcodexten.com/p/your-review-alpha-school Here's a review of AlphaSchool and it's methods. Honestly, the review is a good one and very well written. It's worth your time if you have inkling about alternative education and the use of AI in the classroom. TLDR: The magic is not AI, it's that they bribe kids for good grades. Oops, sorry, 'motivate' kids for good grades.

I view this as a political constraint, cf. https://www.astralcodexten.com/p/lifeboat-games-and-backscratchers. One's identity as Academic, Democrat, Zionist and so on demands certain sacrifices of you, sometimes of rationality. The worse the failure of empathy and rationality, the better a test of loyalty it is. For epistemic rationality, it would be best to https://paulgraham.com/identity.html, but for instrumental rationality it is not. Consequently, many people are reasonable only until certain topics come up, and it's generally worked around by steering the discussion to other topics.

And yet you chose to reply to this comment instead of the parent :) CW is a thing, cant's wish it away. Best to work on rules of engagement and being at least slightly productive about it. Small steps like https://www.astralcodexten.com/p/but-vs-yes-but. Seems trivial, but I'd love to see more informal rules like this enforced in practice.

This seems to boil down to that Elizabeth Sandifer, a self described "middle-aged trans anarchist", believes https://www.astralcodexten.com is "fertile grounds for white nationalist recruitment". This is just wrong and anyone can visit the blog to see for themselves.

And then forgetting. If you're not interested, you cram for the test and then forget. I see little point in forcing people who don't want to learn into classrooms. Worth a read https://www.astralcodexten.com/p/a-theoretical-case-against-education