Lennart Poettering

Raising the soft limit to the hard limit is also recommended by the author of systemd: https://0pointer.net/blog/file-descriptor-limits.html I doubt the kernel would actually allocate the resource space upfront. Like SO_SNDBUF and SO_RCVBUF, it's probably only allocated when it's actually needed.

systemd has definitely made huge improvements to boot security which not a lot of "systemd haters" see. this is a great post from lennart: https://0pointer.de/blog/brave-new-trusted-boot-world.html

You can also do some neat things with "--ephemeral" and "--volatile" to basically overlay the image (or a subset) with tmpfs; any changes to those overlays will be lost when the container is brought down. The specific mount points can be controlled in greater detail via "--tmpfs" and "--overlay". https://0pointer.net/blog/running-an-container-off-the-host-usr.html I'm not sure how easy that is to customize in Podman.

> Why is ordering startup important? This is pretty fundamental stuff. You can't run a program if the disk that it's sitting on hasn't been made accessible yet. You can't start a network service that listens on a certain IP address if that IP address hasn't been configured yet, or if the destination for service logs isn't ready, or if our service needs to talk to some other service like a database which isn't running yet. Etc etc. Booting up a modern system to a graphical environment requires hundreds of things to happen in the right order. As for everything else, the project announcement blog post from 2010 does a pretty great job summing up the shortcomings of doing "the simplest possible thing": https://0pointer.de/blog/projects/systemd.html . You can certainly still do things that way if you want, but most people value the benefits gained from a more intricate approach.

The systemd library for using it is worth looking at. https://0pointer.net/blog/the-new-sd-bus-api-of-systemd.html

Couldn't agree more. Once familiar with the big picture this page is the best starting point whenever to you want change something. To get the big picture I read the systemd posts on https://0pointer.net/blog/archives.html But that's 10 years already! No idea what would be recommended today.

> well... systemd is a clone of launchd sort of. Systemd took a lot of lessons from launchd but also from sysv and upstart. For anyone who hasn't read Lennart Poettering's "Rethinking PID 1" post[1] I highly recommend it. You'll understand the history, but most importantly you'll understand systemd a ton better. [1]: http://0pointer.de/blog/projects/systemd.html

Well. This http://0pointer.de/blog/projects/socket-activation.html sells socket activation as the best thing since sliced bread, including for boot time. It has its uses for some rarely used services and for some startup order problems, yes. About the DNS service, I remember a bug where the fallback behavior was not what people wanted (it was something that seemed to be a good idea to systemd developers but didn't work well in completely reasonable real-world configurations), and the developers rejected the idea that systemd's behavior was wrong. Can't find it right now. I have disabled it on my system as well due to a problem I don't remember. Maybe it didn't use my router's DNS server and went straight to the internet. Maybe I just wanted it to leave my resolv.conf alone.

> you might as well move that logic to the packaging system Depends on who do you mean by "you". Software vendor can definitely write a script but they have no power over distributions to "move that logic to the packaging system". This would have to be a collaborative work by different distros but from my casual look it's just not happening as everyone is happy with their own package manager that's "obviously the best". I do agree on systemd. I didn't like it before I moved to Linux now I see a lot of value that it brings. This may also be relevant: http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.…

Good question! A great introduction that was written while the author was still building systemd can be found here: http://0pointer.de/blog/projects/systemd.html Also, the "systemd for administrators" series: https://gist.github.com/bcremer/8cdf6900c35dda65f387

(2) already exists in at least glibc: getaddrinfo_a. Using it ~5 years ago though I found plenty of bugs in it (e.g. https://sourceware.org/ml/libc-help/2012-07/msg00024.html ) However, getaddrinfo is not a great inferface: you still can't use it to e.g. look up an MX record. For that you need res_query(3), which does not have an async interface in libc. Pottering himself wrote a library to use res_query in a separate thread http://0pointer.de/lennart/projects/libasyncns/. However, I don't like threads, and will avoid them where possible in libraries (an example reason: I like to be in a defined state after fork()). Which means I need an async dns library that implements a resumable state machine. Lately I've been using http://25thandclement.com/~william/projects/dns.c.html

It's very interesting how the whole article (and Lennart) shies away from even uttering "package manager" and instead talks about containers. This is fundamentally an implementation of Lennart's blog post - http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.… That article used the words "We want an efficient way that allows vendors to package their software " ... which was followed by a storm of angry systemd-is-the-borg-it-needs-to-die tweets and articles. Even on HN - the response to this article seems fairly muted... although it is a bigger political issue than even init systems. I wonder what the response would have been to "systemd introduces new portable packaging format"

It didn't just start growing. There was talk of it being the base of the OS (excluding the kernel) since 2012 (as mentioned here http://0pointer.net/blog/projects/systemd-update-3.html). > We have been working hard to turn systemd into the most viable set of components to build operating systems, appliances and devices from, and make it the best choice for servers, for desktops and for embedded environments alike. I think we have a really convincing set of features now, but we are actively working on making it even better. I'm pretty sure I saw others posts, but my googlefu is a bit weak. So, the expansion was in the plan from nearly the beginning (for good or ill)

That matches my expectations. Really, if you are doing something as slow as writing logs to a disk, and the number of processes/threads is not in the tens or hundreds, I don't imagine locking overhead is your problem, given the speed of disk storage. That said, I think the main problem with that is to do it cross platform, which the article goes to paint to mention quite a bit. I imagine the whole point here is to be portable, and I'm not sure what mechanisms work best with that, and what platforms they are available on. I uncovered some unsettling info about Fcntl locking[1], but generally I just used flock when I had to care about it, but i don't think that exists on windows normally(?). 1: http://0pointer.de/blog/projects/locking.html

Sigh. Obligatory reference to http://0pointer.de/blog/projects/the-biggest-myths.html: "Myth: systemd is incompatible with shell scripts. This is entirely bogus. We just don't use them for the boot process, because we believe they aren't the best tool for that specific purpose, but that doesn't mean systemd was incompatible with them. You can easily run shell scripts as systemd services, heck, you can run scripts written in any language as systemd services, systemd doesn't care the slightest bit what's inside your executable. Moreover, we heavily use shell scripts for our own purposes, for installing, building, testing systemd. And you can stick your scripts in the early boot process, use them for normal services, you can run them at latest shutdown, there are practically no limits."

>The last frontier after that is package format convergence, and Lennart has said repeatedly he intends to use the systemd monoculture to push a common package format, which is a really good thing for me. Great, so now instead of adopting a package system system with a solid theoretical foundation like Nix or guix, we're going to dump all dependencies into fat binaries and more or less end up with the solution the NeXT people came up with in the 90s. Such progress. EDIT: Not to mention that Lennart's proposed package system[1] would depend on btrfs-specific features, adding even more code coupling. [1]http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.…

Socket activation: http://0pointer.de/blog/projects/socket-activation.html Socket activated containers: http://0pointer.de/blog/projects/socket-activated-containers.html Resource management: http://0pointer.de/blog/projects/resources.html Better service information: http://0pointer.de/blog/projects/systemctl-journal.html In fact, the whole series is a recommended reading. Anyway, all these features requirer tighter integration with other components than SysV init. E.g. Linux' cgroups, services that support socket activation, integration with the logging system for status information, etc. Also, a lot of techniques in systemd were already pioneered in launchd on OS X since Tiger.

Also, I highly recommend reading Lennart Poettering's initial description of systemd on his blog [1]. It lays out the paradigms behind systemd design really well. I think that it is important to understand the problems that systemd is trying to solve in order to evaluate whether it is doing it's job well. [1] http://0pointer.de/blog/projects/systemd.html

Before this thread devolves into the same arguments all over again, maybe everyone could read the following: http://0pointer.de/blog/projects/the-biggest-myths.html

If you haven't done so I recommend reading 'systemd - The Biggest Myths' http://0pointer.de/blog/projects/the-biggest-myths.html

1: boot time is Poettering's own argument in favor: http://0pointer.de/blog/projects/systemd.html If the systemd team wants to drag goalposts all across the field, that's fine. I'm just going to note their original location. If you want to build a better xinetd, or better SysV init based dependency system (insserv), or alternative (upstart), then do it. OK, upstart also fucks with init, but with a lot less whack then systemd. As to the "I've seen poorly written init scripts": on my distro of choice (Debian), package maintainers do a very good job of providing sane scripts (which are a lot easier to follow than RH scripts, something I noticed when first cutting over to Debian), in part because the distro provides a solid, 18-years of evolution, SysV init based process, and a policy that tends to iron out occasional bouts of dumpth.

> What about /boot - encrypted and tamper proof? Resists evil maid attack? > Rabbit hole, I know, but so fascinating if you solved it all :) Encrypting `/boot/` is the wrong thing to do. One can just replace it with something that logs the encryption key instead (and sends it off to somewhere). You actually need to verify that what you expect to be there is actually there and a encryption isn't gonna do that for you. A reasonably secure boot chain looks roughly as follows: bootloader is checked by the UEFI (read secure-boot), bootloader chain loads a signed UKI which also uses secure boot verification (or just skip the bootloader if you only have a single kernel with no fallback), this then loads a signed dm-verity `/usr/` which brings you to the login screen. Root is just data (e.g. mounted `noexec`) and is encrypted using e.g. PCR 11 so that it only unlocks if the UKI is the one that is expected (with some other backup unlock method). This is basically #2 of the "Design Goals" of Poetterings "Fitting everything together"[0]. It's a good read if you are looking to designing your own Linux image and are looking for some inspiration. There is ParticleOS[1] which somewhat dogfoods this kind of system, though I don't think it is anything remotely considered for a production system. [0]: https://0pointer.net/blog/fitting-everything-together.html [1]: https://github.com/systemd/particleos

Not really, AIUI pottering just thought launchd's socket activation/inetd like functionality was neat: https://0pointer.de/blog/projects/systemd.html. Upstart is more of a direct ancestor: > Why didn't you just add this to Upstart, why did you invent something new? > Well, the point of the part about Upstart above was to show that the core design of Upstart is flawed, in our opinion. Starting completely from scratch suggests itself if the existing solution appears flawed in its core. However, note that we took a lot of inspiration from Upstart's code-base otherwise. > If you love Apple launchd so much, why not adopt that? > launchd is a great invention, but I am not convinced that it would fit well into Linux, nor that it is suitable for a system like Linux with its immense scalability and flexibility to numerous purposes and uses. launchd is horrible though, the folks complaining about systemd would be up in arms if they had to write poorly typed XML key/value files

One of the incredible & fantastic superpowers of systemd. There so much isolation it can give you, rescind so many powers! Alas this article doesn't include even the most rudimentary of examples. Let's look at recent gnome gdm session picker changes. They now use dynamic users, rather than statically allocated ones, helping make sure the picker never gets access to things it shouldn't have (since there's no static user id that an attacker might grant access to). A nice feature from systemd in 2017. So easy to add! DynamicUser=yes and you're good! https://0pointer.net/blog/dynamic-users-with-systemd.html This also gives you a bunch of other security options by default. PrviateTmp=yes. ProtectHome=readonly. ProtectSystem=strict. This is just the tiniest cut of what systemd can do to make your system more secure, how it can help you make use of the kernel to limit your process. There's so so many ways to limit programs. Limit maximum tasks, set Nice= CPUWeight=, IOWeight=, limit CapabilityBoundingSet=, TasksMax=, limit network interfaces, tie a program to certain nftables, just so many. This huge page is but a start: https://www.freedesktop.org/software/systemd/man/latest/systemd.resour… !! Systemd makes excellent excellent use of the kernels namespacing feature to radically boost isolation of programs, to reduce their ability to impact the world in unexpected ways. Even better, you can manage multiple competing realms of apps very very effectively with a…

I suspect that systemd people are looking at this thread in perplexity, and probably doing their thing (that I've seen over the years) of regarding the world of Debian as being amazingly behind the times in places. The SSH server being a socket unit with systemd doing all of the socket parallelism-limiting and accepting was one of the earliest examples of socket activation ever given in systemd. It was in one of Lennart Poettering's earliest writings on the subject back in 2011. * https://0pointer.de/blog/projects/inetd.html And even that wasn't the earliest discussion of this way of running SSH by a long shot, as this was old news even before systemd was invented. One can go back years earlier than even Barrett's, Silverman's, and Byrnes's SSH: The Secure Shell: The Definitive Guide published in 2005, which is one of many places explaining that lots of options in sshd_config get ignored when SSH is started by something else that does all of the socket stuff. Like inetd. This has been the case ever since it has been possible to put an SSH server together with inetd in "nowait" mode. Some enterprising computer historian might one day find out when the earliest mention of this was. It might even be in the original 1990s INSTALL file for SSH.

I find your comment here sadly indicative of the level of discussion around systemd from its haters. Your metric is utterly misleading not only because you're almost certainly counting non-code files, but more crucially, because the systemd repository contains the code for *sixty-nine* entirely separate binaries, separate tools under the overall systemd project umbrella, so counting their collective code size as if it is the code base of one single gigantic tool, a single gigantic program that compiles into a single gigantic binary, with all the interweaving that implies, is just disingenuous at best. That number does not represent the size of a single attack surface area, and pretending it does is nonsensical. It's like pulling in the source of all of gnu and doing a line count. And for the record, I've actually done this properly, cloning the systemd repository and then reading the documentation to figure out what directories and source files did what, and assembled a list of the directories and source files that represented just systemd-the-init-system, and got about 240,000 lines of code. And remember, it won't actually be systemd the init system that will be replacing sudo, it will be systemd-run, which is a separate binary, with a separate memory space, and a separate permissions model, that merely communicates with systemd the init system to get certain things done. I guarantee you it's probably smaller than the code base of sudo, and this architecture, as LP points…

Good recommendations. OSes today are more than kernels. Filesystems, init systems, shells, and the configuration conventions would all be part of it. And probably also the windowing system and desktop environment. The most innovative concepts historically would be Plan9, BeOS, and seL4. Plan9's approach of "everything is a file" is something that was unique. BeOS's extensive use of journalling and metadata in the filesystem gave it big advantages. seL4 being the most rigorous application of a micro-kernel architecture, and making use of formal verification make this one stand out. Capability-based operating systems also offer immense benefits. Fuchsia and Redox are clearly drawing some inspiration from these. Along with GNU Hurd, which you didn't mention. As for shells, nushell is clearly revolutionary. Something that points to what a better world might have been. I would also recommend looking at Lennart Poettering's vision that extends on innovations like cgroups, journalling filesystems, ostree, etc. https://0pointer.net/blog/projects/stateless.html And of course, Nix and Guix both align with this vision of reproducibility, statelessness, and transactionality... along with Fedora Silverblue. I kinda think of this as making the filesystem work more like git. A distro is a branch. A rollback is a branch from a previous commit. And then the build system would be completely idempotent... every compilation from any system would be byte-for-byte identical, and thus…

> So no, I don't see anything UNIX related where Apple has helped to caught on Perhaps if you read “Rethinking PID 1”, from Lennart Poettering in 2010, who originally wrote systemd along with Kay Sievers http://0pointer.de/blog/projects/systemd.html “But first, let's clear a few things up: is this kind of logic new? No, it certainly is not. The most prominent system that works like this is Apple's launchd system: on MacOS the listening of the sockets is pulled out of all daemons and done by launchd. The services themselves hence can all start up in parallel and dependencies need not to be configured for them. And that is actually a really ingenious design, and the primary reason why MacOS manages to provide the fantastic boot-up times it provides. I can highly recommend this video where the launchd folks explain what they are doing. Unfortunately this idea never really took on outside of the Apple camp.” Other than that, your answer has nothing to do with what I wrote.

The article title is very misleading. This isn't bypassing FDE in any way. It's just getting a root shell on a machine you have physical access to with a particular boot configuration. Clever? Yes. But no encryption is bypassed. Most systems will only be listening to PCR 7 anyway, so a similar attack could be done by loading your own custom bootloader, or possibly reading messages on the SPI bus when booting. This is just a nice trick that's easier/faster. There is a balance of convenience versus security and this could be prevented easily by disabling recovery shell or registering more PCRS (with correct boot setup), but would be much more annoying to remotely administer since you could get failure states where the TPM won't release the keys in a variety of situations. Ultimately TPM-only unlock is a significant increase in security vs unsophisticated attackers and probably fine for 99% of people, but isn't something to rely on if you are concerned about sophisticated attackers. Even with perfect PCR setup and enrolling only custom keys in UEFI, a running machine is still vulnerable. Cold boot or DMA attacks (Thunderbolt or PCI) are just a few that come to mind. These sound extremely sophisticated but are easily done even with hobbyist equipment. Any running machine with currently unlocked disks should be assumed to be possible to compromise with physical access. If interested in Linux boot chain Poettering has a good read:…

This is an amazing release. As far as controversy clickbait topics go, personally I'm much sadder to see umerged-usr bite the dust, even though i'm a huge fan of merged-usr, just because it was super nice being able to boot off whatever / you have and drop another systems' /usr in. But at least there's systemd-sysext to allow some combining overlays. Anyhow, excited to talk through various changes that look exceptionally neat to me: RootEphemeral=, which lets processes run in a filesystem snapshot; super neat for seeing what changes a process might make to your system. https://github.com/binpash/try was a popular submission a couple days ago, & this builds in a similar capability to the os. RestartSteps= and RestartMaxDelaySec= to finally finally offer some level of control over exponential-backoff in the restart behavior. FileDescriptorStorePreserve= is kind of mysterious, lets systemd store open file descriptors for a process even after it terminates. Some interaction with systemd-analyze. Not sure what this is for exactly but definitely cool extra visibility/observability of processes. I tend to think it also potentially could be useful allow for some weird/fun stuff like restarting services without closing ports. Soft-reboot which can be useful for switching roots without having to juggle bootloading. Very useful for anyone doing partitioning or what-not. And as I hinted above, you can pass file-descriptors, meaning you can avoid service-interruption while rebooting;…