Richard Crowley

You remind me of when I started to actually get this stuff and it occurred to me that monads are just metadata+data with some special properties, and I posted about it on reddit and got downvoted to hell and told the many ways how that's wrong. The FP crowd...must be great at parties. I get this frustration. I think a big part of it is difference in definitions. There's the mathematician's monad, which is pure and abstract and obeys all these laws. And then there's the engineer's monad: an interface you can map, ap, and flatmap over. People get so hung up over "algebraic purity" they forget why they were invented/co-opted for software in the first place: to solve the problem of applying pure functions to side-effecty operations. > A monad is a wrapper. Anyone who tells you otherwise is being obtuse. https://rcrowley.org/2011/09/21/monads.html

For POSIX, yes. https://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html Windows has a deep well of POSIX in the kernel (plus hard file locks), and it appears to hold there. https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem

> Are we talking about the POSIX interface to files Those give you these atomic ops and optimistic or cooperative locking: https://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html Multi-file transactions can be built by moving whole directories over symlinks to the previous version. The linux-specific RENAME_EXCHANGE flag can simplify this. > or is there a new transactional API CoW filesystems give you snapshots and reflink copies via ioctls. Under heavy concurrency this can provide cheaper isolation than locking. > and makes the writes always durable (sync; not fsync)? For durability fsync is sufficient if you do it on the file and the directory. To combine atomic and durable you can do the write, fsync, rename, fsync dir dance. https://lwn.net/Articles/457667/

> that offers transactions, or "compare and swap"-type operations? "Things UNIX can do atomically" https://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html

You can read/watch an overview of the previous generation of OpenDNS stats here: http://rcrowley.org/2009/06/23/building-opendns-stats-at-velocity

I can see the value of RVM for developers, running the same code against various gemsets and versions of Ruby, I used it this way, but not for deployment, why not simply compile everything necessary by hand, install the necessary gems, say ruby 1.9.2 with rails 3.0.3 and some other gems, and them use the result to chroot personalized deploys (or containerize them) instead of installing various versions of ruby with one shell script? EDIT: Well, since you cannot answer in this thread feel free to answer in my first comment in this thread above if you wish. EDIT2: I found this article about the matter, I found it to be good, I agree with him in the large picture: http://rcrowley.org/articles/dependencies.html

I think you misunderstood the first half of my comment. Many of those things have an asterisk after “synchronous”, but in their default mode they often are in fact sync. Saying “technically commands are posted and later observed, and that’s how we get security vulns” is an extraordinary claim. The vast, vast majority of program statements do not work thus. Memory operations immediately move instructions through the CPU to access data. Many IO operations request that the kernel immediately send an interrupt to a hardware device and wait for a response from that device to be written to memory—that’s synchronicity in the domain of electrical engineering, not just software. And sure, there’s periodicity and batching there (waiting for scheduler ticks and interrupt poll frequency and such), but none of that makes something less than synchronous; it just might slow it down. Unless you were referring only to timing attacks in your claim that security vulnerabilities result from not-really-synchronous actions, then I think that’s wrong twice over. To expand on the examples: mkdir(2)’s durability (which is what we’re talking about when we refer to “synchronous-ness” of filesystem ops) depends—it depends on the filesystem and caching configuration of the system. But on many (I’d hazard most) file systems and configurations, new directories are persisted either immediately through the dentcache to the disk, or are during one of the next two calls to fsync(2), the next example I…

1. I don't know if there's a single name for this. I will point out that AWS EBS and Google Persistent Disk as industrial examples of distributed, replicated block devices are also providing durability via replication. They're just providing it at a lower level that ends up sacrificing performance. I'm struggling to come up with a citation but I think it's either Liskov or Lynch that offered a proof to the effect of achieving durability in a distributed system via replication. 2. The thinking laid out in the blog post you linked to is how we went about it. You can do the math with your own parameters by computing the probability of a second node failure within the time it takes to recover from a first node failure. These are independent failures, being on physically separate hardware in physically separate availability zones. It's only when they happen together that problems arise. The core is this: P(second node failure within MTTR for first node failure) = 1 - e^( -(MTTR node failure) / (MTBF for a node) ) 3. This one's harder to test yourself. You can do all sorts of tests yourself (<https://rcrowley.org/2019/disasterpiece-theater.html>) and via AWS FIS but you kind of have to trust the cloud provider (or read their SOC 2 report) to learn how availability zones really work and really fail.