David Crawshaw

The best writing on this is the "agent principal-agent" problem, which correctly frames the problem of agents and code review in terms of trust. This is why the solutions for high-trust environments (small teams) and low-trust environments (big companies, open source projects) will be different. https://crawshaw.io/blog/agent-principal-agent

I think your only problem is too-high expectations. >examples where quality far exceeds our existing best in class work I'm no expert, but what percentage of the economy needs/is best in class work? I agree running up against the limits of these models after reading the marketing material can give the impression they're "a kludge for the mediocre" and they certainly are in part that, but they're also so much more.[0] [0] https://crawshaw.io/blog/programming-with-llms

Would love to see a blog post on building it some time, I think the last vaguely similar (but serious) rundown in this vein was “One process programming notes”. https://crawshaw.io/blog/one-process-programming-notes

Tailscale is one of my favorite companies. They're clearly on to something. Here's a great post by their CTO explaining a lot of the motivation and vision behind it: https://crawshaw.io/blog/remembering-the-lan IMO the main outstanding questions/concerns are: * Is the VPN model really the way to go? If someone gets their hands on one of your Tailscale nodes, they can access every service on your tailnet, which are likely running with reduced security since that's a huge part of the appeal. This is exactly the situation BeyondCorps/Zero Trust was created to avoid. Tunneling services[0] are more of a Zero Trust approach, but they can't match the seamlessness of Tailscale once a node is connected to the tailnet. * Can it expand into the layman market? I wonder if the average person will ever be willing to install a VPN app on all their devices. On the flipside, I could see TS partnering with someone like Google to integrate TS tightly with Android and set up a private network between all your Google-signed-in devices. * The relay system - DERP is nice, but it's primarily intended for signaling/fallback. It feels like CGNAT adoption is growing faster than IPv6 is, and I wouldn't be surprised if fewer and fewer p2p connections succeed over time[1]. DERP forces everything over a single TCP connection (HOL blocking), and I'm not sure it even has any flow control. * Use in web browsers - They got a demo of this working, but it's pretty involved. You have to compile the entire…

database/sql adds some overhead; in return, it gives you a connection pool, and tries not to have you worry about connections. A lot of the interesting features of SQLite are connection oriented, and compared to other databases, connections are cheap (because it's all local). So the things database/sql offers are (relatively) less valuable, and it makes other things harder. Yet (IMO) connection pools still offer some performance benefits, not having to worry about (re)preparing statements on new connections is still helpful, and many of the connection oriented things you might want to do can be exposed through database/sql with some imagination. I have a lot of respect for crawshaw and zombiezen; I started from their PoV. But I've since come around to database/sql. A lot of what is mentioned in [1] has been fixed in database/sql or can be fixed in drivers: parameter names now work in database/sql; their approach to statement caching isn't necessarily better, but can be added to a driver, or live in a compatible framework; panics vs. errors is contentious; savepoints can be made to work just fine [2]; and for contexts, I started with crawshaw and built an improved version of it into my driver, that is both low overhead, interacts well with the Go scheduler, and doesn't start additional goroutines. Living in database/sql land gives you access to other amazing tooling like SQLC, just to name a single one. And it doesn't need to be either/or: drivers should support both…

I don’t see any argument in this article for the claim of the cost being absurd. On the contrary, the final line > Maybe there is a way to do better, I hope there is, but I suspect not. to me, says the writer suspects that it isn’t absurd, but as good as it can get. I also think there’s broad agreement on the fact that finalizers almost always are a bad idea. For golang, there’s https://crawshaw.io/blog/tragedy-of-finalizers, which says: “So a common instinct we all have is to say "gee, it sure would be nice if the file descriptor were closed when the File is garbage collected, let's use a finalizer". This does not work.” and “It is a bug to depend on the GC to run before your resources are exhausted” and “What are finalizers good for? Nothing generally.” It also says “I believe I have one very small use for finalizers as they exist today, which I will talk about in a followup blog post”. I couldn’t find that post, though. Java even plans to remove them from the language (https://openjdk.org/jeps/421)

Can you elaborate on what the app does and also prioritize your criteria? If simplicity is your main requirement you might find inspiration in this: > My focus has been on how to adapt the lessons I have learned working in teams at Google to a single programmer building small business work. There are many great engineering practices in Silicon Valley's big companies and well-capitalized VC firms, but one person does not have enough bandwidth to use them all and write software. The exercise for me is: what to keep and what must go. > If I have been doing it right, the technology and techniques described here will sound easy. I have to fit it all in my head while having enough capacity left over to write software people want. Every extra thing has great cost, especially rarely touched software that comes back to bite in the middle of the night six months later. https://crawshaw.io/blog/one-process-programming-notes https://news.ycombinator.com/item?id=17701882

I haven't been offended by it, and I'm the founder of a company that's been around longer and is probably the closest competitor: https://www.zerotier.com/ I don't hate on people for doing similar things. If anything I am absolutely shocked it took someone this long, since to me the idea of a full mesh virtual network is how things should work, everything else is stupid and clunky, the fact that we have to bounce off servers to transfer things is dumb, and NAT is pure concentrated evil and must be destroyed. I absolutely loved David Crawshaw's Remembering the LAN post: https://crawshaw.io/blog/remembering-the-lan It echoes exactly the sentiments that motivated me to start working on ZeroTier, so I wish them well. As far as the upvotes go I just figured they have a lot of friends from their time at Google and their posts get upvoted a lot. I notice the same thing when any company that has a lot of HN users among its employee base does anything. When AWS, Apple, and Google do a bunch of product releases the front page gets bombed for days. The site practically turns into a news feed of new AWS Elastic Beanie Cap products when the ironically titled AWS ReInvent happens. If anything the FAANG companies get more free advertising here than anyone.

The latest post from the very interesting exe.dev project (previously of 'I am building a cloud' [0]). They have a lot of interesting work that goes in to making SSH work with their system (eg previously [1]), and this is a continuation of that. Using Unix sockets to transfer file handles is not a new idea, but using for SSH connections is interesting to me. I wish it were a little heavier on the details here, but otherwise this is an interesting approach to handling network traffic while maintaining zero-downtime upgrades [0] https://crawshaw.io/blog/building-a-cloud [1] https://blog.exe.dev/ssh-host-header

exe.dev | full time member of technical staff | sf bay area only competitive salary, meaningful early equity, 2 days a week in the downtown sf office come build a cloud. we are doing everything from custom hardware to custom ssh servers to our own global load balancer. very senior small high-trust team looking for someone we can rely on to get done whatever is most important in the stack for the business. more about what we are building here: https://crawshaw.io/blog/building-a-cloud email me at [email protected].

Related: Crawshaw's "one process programming": https://crawshaw.io/blog/one-process-programming-notes

Low signal, see the post he refers to: https://crawshaw.io/blog/programming-with-llms

Reposted from original: https://crawshaw.io/blog/programming-with-llms HN discussion of original a few days ago: https://news.ycombinator.com/item?id=42617645

Is there a system where a single thread writes (ala David Crawshaw's “One Process Programming -- https://crawshaw.io/blog/one-process-programming-notes), and then changes are replicated out quickly and efficiently? That plus a globally incrementing version number is what I want…

This is the post: https://crawshaw.io/blog/sharp-edged-finalizers

> However, contemporary applications rarely run on a single machine. They increasingly use remote procedure calls (RPC), HTTP and REST APIs, distributed key-value stores, and databases, I'm seeing an increasing trend of pushback against this norm. An early example was David Crawshaw's one-process programming notes [1]. Running the database in the same process as the application server, using SQLite, is getting more popular with the rise of Litestream [2]. Earlier this year, I found the post "One machine can go pretty far if you build things properly" [3] quite refreshing. Most of us can ignore FAANG-scale problems and keep right on using POSIX on a handful of machines. [1]: https://crawshaw.io/blog/one-process-programming-notes [2]: https://litestream.io/ [3]; https://rachelbythebay.com/w/2022/01/27/scale/

If you want an even-earlier-published article to point to as well, this has been mine: https://crawshaw.io/blog/one-process-programming-notes There's also the venerable "Taco Bell Programming": http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html… It all boils down to unix principles, I guess: use the most simple (but not simplistic) tools for the job.

Counterpoint: > When you have a small team that write tests and a compiler that is on the boundary of interactive speed, compiler speed dominates performance. https://crawshaw.io/blog/fast-compilers

This could be too high level, but I stumbled onto the founder's (crawshaw) website and found one of his blog posts very helpful for understanding basic fundamentals of the product (i.e. Zero Trust networks and microsegmentation) https://crawshaw.io/blog/zero-trust

How to create a website with Go and SQLite and why that might be a good idea to start with: https://crawshaw.io/blog/one-process-programming-notes

[ Woops, I went into GPT-3 mode. I'm really bad at rambling concisely, and fitting the result into a couple of comfortably short paragraphs. :/ ] Thing was, Alley Cat was the only (interesting) game on the machine in question... and when I (sadly) explained what I'd done to the person who gave me the system, they responded by promptly deleting the EXE file. While a bit of a tangent, my honest thanks for helping me realize (as I type this) that this memory was what kicked off the deep-seated negative associations I've had around gaming my whole life, and why I often lock up whenever I think about the subject. (!) I completely agree that filesystem recovery is somewhat of a more advanced topic, and entirely unfair to expect mastery of at ~10 :D. I kind of understand about 42% of https://marcan.st/2015/10/rescuing-a-broken-ntfs-filesystem/ myself, for example. I'm very curious how you managed to end up staring at a system with the DriveSpace driver un-loaded though; that just sounds broken. I wonder if you tripped over an uncaught bug/edge case in the setup process. (And yeah, the system I was using had DOS 6.20 on it, thus had DoubleSpace instead.) [TIL about the Dysan brand of floppy disks] How'd you figure out that BAS = QBASIC? In my case, the person who gave me the PC randomly decided one day that it needed a menu, "so we didn't need to type commands in". The PC sat in my bedroom, and Mum rarely used it... but okay, whatever works. Well, said menu ended up being a…