Irene Zhang

See https://irenezhang.net/papers/demikernel-sosp21.pdf for a more thorough paper on the Demikernel from 2021. There are some great ideas for improving the kernel interface while still allowing efficient DPDK-style pipelines.

I'm not thinking of strict serializability - I do actually mean linearizability. My previous comments were referring to the basic algorithm presented as the main contribution of this paper in section 3. In this algorithm, "Every command accesses only one object o." With each operation applying to a single object and each object satisfying linearizability, the system as a whole will be linearizable. The distinction between linearizability and strict serializability is somewhat subtle. I highly recommend this blog post [1] by Irene Zhang and this blog post [2] by Peter Bailis for some really great discussion on the subtleties involved. [1] https://irenezhang.net/blog/2015/02/01/consistency.html [2] http://www.bailis.org/blog/linearizability-versus-serializability/

great point! This was about as simple as it got in order to ship something quickly that was minimally intrusive. So the prototype as it is now buys you sequential consistency, but I want to detangle some great points you made. 1. Performance: The performance is probably going to be limited. In the sense that most databases have better performance than most blockchain implementations. So you're arbitrarily bottlenecking your writes. There are a ton of really great papers that show you can do WAY better if you are more careful with your distributed design. They usually involve getting into the weeds of redesigning the internals database. Just for fun, I really liked these papers: http://nms.csail.mit.edu/~stavros/pubs/hstore.pdf https://irenezhang.net/papers/tapir-tocs18.pdf https://www.usenix.org/system/files/conference/nsdi17/nsdi17-wei-micha… So there is a ton of space to improve the performance here, I just wanted to build something quickly. 2. Authentication: Definitely an interesting technical challenge here. Most web apps are written assuming they are one-of-many servers accessing the database, but honest and has root authority. So how do we make sure that a validator has the authority to write to a particular table/row? Right now, this is still an open challenge and would love to engage the community on ideas here. There might be some way to tag/sign all tables/rows with creator, and do some write access control based on that? I'm not sure how many web apps…

http://irenezhang.net/papers/demikernel-hotos19.pdf Excerpt: "Researchers have long predicted the demise of the operating system [21, 26, 41]. As datacenter servers increasingly incorporate I/O devices that let applications bypass the OS kernel (e.g., RDMA [12] and DPDK [15] network devices or SPDK storage devices), this prediction may finally come true. While kernel-bypass devices do eliminate the OS kernel from the I/O path, they do not handle the kernel’s most important job: offering higher-level abstractions. This paper argues for a new high-level, device-agnostic I/O abstraction for kernel-bypass devices. We propose the Demikernel, a new library OS architecture for kernel-bypass devices." That's the WHY of the Demikernel...

Paper can be found here: http://irenezhang.net/papers/demikernel-hotos19.pdf