Rob Pike

Rob Pike of Plan 9 and Go fame lamented this back in 2000 in his “Systems Software Research is Irrelevant” talk: http://herpolhode.com/rob/utah2000.pdf I’d love to see a new operating system that explores radically different APIs for applications. The trouble is writing an operating system is a large effort. Barring market effects, OS has to be heads-and-shoulders better than existing ones in order to convince application developers to write software for it. Windows, macOS, and Linux are good enough for most people, even techies. Additionally, it is often easier to modify an existing operating system such as Linux than to go through the trouble of writing a brand new operating system.

Some of the original Bell Labs Unix people have a very austere view of the Unix philosophy. In their view, it all started to go wrong -- Unix began to depart from the founding principles -- as early as 1978. In their view, Berkeley Unix does not follow their Unix philosophy -- they don't like network sockets, for example, whose design was mostly imported from some earlier non-Unix system (maybe Tenex?). Some of this attitude appears in Rob Pike's 'Good, Bad and Ugly' talk from 2001 [1]. He says the early Unix tools approach had, by then, already been replaced for most users by big do-everything programs: Emacs, Perl, browsers... 1. http://herpolhode.com/rob/ugly.pdf

It's worth looking at Rob Pike's (one of the Plan 9 inventors) paper, "Systems Software Research is Irrelevant": http://herpolhode.com/rob/utah2000.pdf My own view is that Linux is Good Enough for most people. And that "good enough" beats "better", at least for most values of "better". This is an admission that "path dependence" matters for the operating system market, and that network effects can cause the appearance of market failure.

This reminds of Robert Pike's "Systems Research is Irrelevant" speech [1]. Now, 20 years after his speech, we are still stuck with the same notions (such as everything being a string). It's not that there are not plenty of alternatives around, however, expectations are so high that it's almost impossible to make a new computer system economically viable. On the other hand, the hacker and maker scene is very active, some of them building operating systems and hardware such as tiny Lisp-based machines [2] and OSes [3]. (My only gripe is that most of the new "avant-garde" systems are still text/file-based.) I'd love to see a next wave in personal computing, starting with a clean slate, building on the research, insights and learning from the mistakes that have been made. I have no doubt that it will happen, the question is only when. As for interoperability: Even on the same platform there are countless problems getting software to talk to each other, so I don't think that a new system will make the situation any worse. [1] http://www.herpolhode.com/rob/utah2000.pdf [2] https://www.tindie.com/products/lutherjohnson/makerlisp-ez80-lisp-cpm-… [3] https://mezzanos.herokuapp.com/

What Rob Pike is refering to is of course Plan 9 and its derivatives (say Inferno). Whether you want to consider them Unix-likes or not is your choice (many HNers will say it's more Unix than Unix). If you look for desktop alternatives to Unix, OSes to look at are BeOS (an open source clone is developed under the name HaikuOS) and the Windows NT kernel (don't hurt me: the Windows NT kernel is IMHO far more elegant than, say Linux or Mach. What is crappy is especially the WinAPI; please look deep below the surface: Here Windows gets nice). The reason why you won't find many desktop/server alternatives to Unix can be read here: http://herpolhode.com/rob/utah2000.pdf Thus lots of OS reserach is either focused on embedded stuff or virtualization instead of desktop or server. Accepting this, especially have a look at the L4 kernel family (a family of very small and fast microkernels). If you like highly secure designs, you'll probably like seL4, the first formally verfied kernel. Also QNX is worth a look (QNX is Unix, accepted; nevertheless a very elegant kernel design).

Rob Pike actually wrote something similar in his short, self-proclaimed polemic, "Systems Software Research is Irrelevant": http://herpolhode.com/rob/utah2000.pdf "Only one GUI has ever been seriously tried, and its best ideas date from the 1970s. Surely there are other possibilities. (Linux’s interface isn’t even as good as Windows!) There has been much talk about component architectures but only one true success: Unix pipes. It should be possible to build interactive and distributed applications from piece parts. The future is distributed computation, but the language community has done very little to address that possibility." "The world has decided how it wants computers to be. The systems software research community influenced that decision somewhat, but very little, and now it is shut out of the discussion..." And particularly relevant to the HN community at large: "Be courageous. Try different things; experiment. Try to give a cool demo."

(In my opinion) Here's a bunch of reasons why Plan9 isn't popular, and then a reason or two why something like it likely will be before too long: The first obvious answer (also mentioned in other comments) is because "Systems Software Research is Irrelevant". See Rob Pike's paper by that name: http://herpolhode.com/rob/utah2000.pdf To the reasons Pike offers there, I'll add: 1. The companies that owned the software had poor strategies for encouraging widespread adoption, if that was even ever their goal. 2. The first dot-com boom ca 2000 greatly expanded the IT / programmer workforce and (I claim) significantly dumbed it down. The advantages of a new operating system lack economic impact until a huge number of programmers are trained to use those advantages. We have huge sunk costs invested in maintaining a huge supply of mostly weakly skilled programmers and admins, a sort unlikely to adapt to and adopt a new OS. 3. In a related way, the sectors that could in principle drive something like Plan 9 adoption are heavily invested, by now, in massive amounts of bloatware that, dysfunctional as it is, is both critical and non-portable. 4. Modern hardware is fast enough that fairly high level programming languages and environments tend to dominate. These often include a "least common denominator" view OS capabilities so that programs port easily among Windows, Linux, Unix, MacOS, etc. This hurts demand for OS features other than the "least common…

X-Rays of Rob Pike's Shoulder http://herpolhode.com/rob/xray.html

> This reminds me of Rob Pike's article "Systems Software Research is Irrelevant," written about 15 years ago. 25 years ago: http://herpolhode.com/rob/utah2000.pdf (Time flies.)

Sounds like you like myself have taken Rob Pike’s take from long ago to heart http://www.herpolhode.com/rob/utah2000.pdf While plenty of things have happened since that paper, I have this sinking feeling that he was right and we just stopped trying to really expand and explore what’s possible. But it may be more a matter of the state of academia than about the domain itself. It’s not like people were getting a bunch of conference invitations for GPGPU applications with ML until hype freight trains hit. This sobering reality of academic hegemony and grant chasing kept me from ever getting terribly interested in systems research unfortunately.

I read that more in a context of "teaching oneself". However, I think he's at google atm: http://herpolhode.com/rob/

I am writing Smalltalk code lately. I find it useful to think of Smalltalk as an operating system, like Linux or Windows or Emacs. It has a different way of doing things, and learning this is both frustrating and rewarding. It brings back vague memories of learning Linux for me. It feels like a nice way to break out of the bubble of that Rob Pike famously lamented in http://herpolhode.com/rob/utah2000.pdf.

Similarity isn't a clear-cut yes/no question, but the NT kernel is so similar to UNIX that it was able to adopt a second syscall ABI compatible with Linux. Present operating systems are basically a monoculture in terms of high-level design decisions; Rob Pike complained eloquently about this in 2000, and outside of possibly unikernel development (which still usually has a libc/unix-like layer implementing at least a filesystem) little has changed since: http://herpolhode.com/rob/utah2000.pdf

the time for grand OS ideas has long passed. here's an eulogy by one of the authors of plan9 and inferno (you won't hear anything from the authors of other operating systems except tannenbaum): http://herpolhode.com/rob/utah2000.pdf

There's not much difference between an academic language designer being a sourpuss that his ideas/language is not appreciated outside of his circle, and Rob Pike being a sourpuss that "Systems Software Research is Irrelevant"[1]. When you think about it. [1] http://herpolhode.com/rob/utah2000.pdf

http://herpolhode.com/rob/utah2000.pdf is about lack of computer systems research, and it touches on the problem you mention.

> New operating systems today tend to be just ways of reimplementing Unix. If they have a novel architecture -- and some do -- the first thing to build is the Unix emulation layer. > How can operating systems research be relevant when the resulting operating systems are all indistinguishable? > There was a claim in the late 1970s and early 1980s that Unix had killed operating systems research because no one would try anything else. At the time, I didn’t believe it. Today, I grudgingly accept that the claim may be true (Microsoft notwithstanding). > A victim of its own success: portability led to ubiquity. That meant architecture didn’t matter, so now there’s only one. Linux is the hot new thing... but it’s just another Unix. http://herpolhode.com/rob/utah2000.pdf

"Systems Software Research is Irrelevant" by Rob Pike http://herpolhode.com/rob/utah2000.pdf There's some pretty brutal truths here.

I don't see a link to Rob Pike's other contribution, "Systems Software Research is Irrelevant" http://herpolhode.com/rob/utah2000.pdf It's really a very good piece, which I'd summarize as a catch-22. To be important you must be different. To be successful you must be the same. Linux solves that by being the same. (And indeed Apple, acquiring NeXT did as well.)

... and to be fair: Rob Pike absolutely nailed this 13 years ago: http://herpolhode.com/rob/utah2000.pdf

Be sure to also see http://herpolhode.com/rob/utah2000.pdf "Systems Software Research is Irrelevant" by Rob Pike

That's the one, though I prefer this format: http://herpolhode.com/rob/utah2000.pdf I don't agree strictly with everything he says. It is a polemic rather than a careful argument. But still, he has a point and the data about OS research is strong (even beyond his data).

See also Rob Pike's talk "Systems Software Research is Irrelevant" (http://herpolhode.com/rob/utah2000.pdf). BeOS was pretty interesting, too, but its story is too far off topic.

>The future is already here — it's just not very evenly distributed. Nobody is denying it, indeed we see many related development. However, it's not only about "the lost future that never comes", I think it's also due to the people being increasingly alienated about the current state of computing. Primarily, it's about the lost mentality of the future. Cyberpunk promised a future where computing is the disruptive technology. Since 2006, the ever-increasing clock speed came to a halt. Since 2013, the general performance of Intel processors remained constant. Selling of PCs keeps declining. No major breakthrough in practical operating systems are made beyond Unix (http://herpolhode.com/rob/utah2000.pdf). Cyberpunk promised a future where megacorps and governments being increasingly oppressive to the population with technology, it has occurred as predicted. Cyberpunk also promised the "cyberspace" is going to be the electronic frontier where technologists reclaimed liberty... Today, the damn web browser that runs all the craps written in Electron and JavaScript is hardly the "frontier", neither Facebook, Twitter and the endless timeline of buzz, which constitute 90% of the Internet activities count. Also, from 2000-2013, decentralization was killed, not built. Move importantly, today, we don’t even know how to waste time on the Internet anymore (https://news.ycombinator.com/item?id=17068138). Though, It, still, has arguably occurred to some extents too, but progressed…

> The success of browsers is an indictment of the entire field of operating systems research. Rob Pike wrote in the year 2000 that systems software research was irrelevant (http://herpolhode.com/rob/utah2000.pdf), and it seems to me that not much has changed since then. How many ideas from the last few decades of operating systems research have actually been put into practice? The real problem is more fundamental: people want their software to keep working. As Rob Pike put it in the talk linked above, "to be a viable computer system, one must honor a huge list of large, and often changing, standards: TCP/IP, HTTP, HTML, XML, CORBA, Unicode, POSIX, NFS, SMB, MIME, POP, IMAP, X, ... With so much externally imposed structure, there’s little slop left for novelty." For example, I'm sure some software out there is using the OpenGL API in a fundamentally insecure way. Changing the API to be safe would break this software. And maybe that would be a good tradeoff, if reworking OpenGL were the only thing you needed to do to safely run untrusted code. But almost every part of the system would have to change. You'd be left with a system which breaks or degrades pretty much everything you try to run on it.

I do think there is a big difference in academic computer science now and then. Rob Pike has written about it from an operating systems perspective [1], and Richard Gabriel from a programming language perspective [2]. It seems that until the early 1990s, there were lots of projects focused on building "systems", i.e. big pieces of software which are in themselves practically useful; then there is a sharp shift and academic research focus on "theory" (in PL research, e.g. a type safety proof for small core calculi, or a particular algorithm for program analysis). People have always pursued "small" ideas. But I think the fact that we stopped writing "big" systems is a real change. (I guess the reason is that off-the-shelf operating systems and programming languages gradually got better, until it became impossible to "compete" with them. C.f. the Lua langauge, which got written basically because "western" langauges were not easily available at the time). My impression is that there is a lot less diversity of ideas now than there used to be, because everyone is incrementally improving the same set of OSs/langauges. [1] http://herpolhode.com/rob/utah2000.pdf [2] https://www.dreamsongs.com/Files/Incommensurability.pdf (see the section starting on page 10)