Franklin Chen

Agreed, but now I think of it there is a level beyond the copying/stealing of code: "The best code is the code you didn't write". Of course, it is even better to beat a complex program with a small amount of shell-fu.[1] [1]: http://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys-…

I still believe that people love or hate CoffeeScript mainly because of the differences in syntax: http://franklinchen.com/blog/2011/11/06/the-real-reason-for-the-coffee…

Concerning stephen's item (2). The stricter set of rules was laid out by Richard C. Waters in Optimization of Series Expressions: Part I: User's Manual for the Series Macro Package, page 46 (document page 48). See reference Waters(1989a). The paper's language is a bit different than contemporary (2023) language. `map()` is called `map-fn`. `reduce()` a.k.a. `fold` seems to be `collect-fn`, although `collecting-fn` also seems interesting. sorting, uniqueness and permutation seem to be covered by `producing`. Just think of McIlroy's famous pipeline in response to Donald Knuth's trie implementation[mcilroy-source]: tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed $ {1}q As far as pipeline or stream processing diagrams are concerned, the diagram on page 13 (document page 15) of Waters(1989a) may also be worth a closer look. What the SERIES compiler does is pipeline the loops. Think of a UNIX shell pipeline. Think of streaming results. Waters calls this pre-order processing. This also seems to be where Rich Hickey got the term "transducer" from. In short it means dropping unnecessary intermediate list or array allocations. Shameless self-plug: Eliminated unnecessary allocations in my JavaScript code by adding support for SERIES to the PARENSCRIPT Common Lisp to JavaScript compiler. The trick was (1) to define (series-expand ...) on series expressions so that they can be passed into (parenscript:ps ...) and (2) the parenscript compiler…

An excellent point. To corroborate it, there was a challenge issued by Jon Bentley to Don Knuth to write a program that Doug McIlroy would then critique. The program was to parse a file and count the frequencies of words, outputting a list of the frequencies sorted highest to lowest. Knuth wrote -everything- from first principles, custom file reading, parsing, the whole nine yards. At the end he had a 10 page WEB literate programming document that solved the problem. McIlroy, in his critique, wrote a six line shell script to do the same thing tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q McIlroy pointed out that by using generalized abstractions over small tasks, file reading, parsing, sorting, etc. you can write software that can be re-used and solve the business need in a reasonable timeframe. If we did everything from first principles then nothing would get done. The merits of McIlroys argument in the context of Bentley's challenge are another matter, but I believe that his point is a good one in the general case, we are hired to build software that meets business needs, and most of the time that does -not- mean hand-crafting purpose-built data structures and algorithms. For more on the Knuth, McIlroy story: https://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys…

That reminds me of https://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys… Basically, Knuth was writing a beautiful but excessively low level program. His challenger wrote a dozen lines of shell-script.

For more on Knuth, see https://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys… and also a C version in http://www.cs.upc.edu/~eipec/pdf/p583-van_wyk.pdf I think the main thing that stands out to me here is just how long ago it was; Knuth was writing just past the assembler era. None of the modern infrastructure was available, such as languages with builtin dictionaries like Perl and Python. However, software compositionality is one of those things that's been tried in so many not-quite-successful ways. Pipes? Objects? Interprocess object systems (COM/OLE) or object brokers (CORBA)? RPC? Microservices? Packages? Object pipes (powershell)?

Here's the paper: https://www.cs.tufts.edu/~nr/cs257/archive/don-knuth/pearls-2.pdf (ACM ref, too: https://dl.acm.org/citation.cfm?id=315654 ) Also, I found a blog post which links to a C version of the exercise in the spirit of Knuth's approach, and a Haskell version in the spirit of McIlroy's: https://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys…

I had commented on Chen's post, here: http://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys-… and linked to a blog post of mine, in which I too had written two solutions (one in Python and one in shell) to the original problem posed to Knuth by Bentley. Fun. Here are my solutions: https://jugad2.blogspot.in/2012/07/the-bentley-knuth-problem-and-solut… A roughly similar problem is posed and solved in the Kernighan and Pike book, The Unix Programming Environment, IIRC, where they mention it as a nice example of the power of combining components, which is a key part of the Unix philosophy - quoted by commenter sswam in Chen's post: [ This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. -- Doug McIllroy ]

Yes, I use Alan Kay's pyramid image[1] in my talks about software architecture[2][3][4]. But then I compare it to the Cologne Cathedral, which encloses a vastly larger space with vastly more light using 1/50th the material, yet it's the same kind of material: cut stones piled on top of each other. So there is hope, I think, and that hope, I think, is improving our understanding and practice of software architecture[5]. And it's not like there we don't have examples in software. Let's take Bentley's challenge[6]: Don Knuth with 12 pages of (literate/web) Pascal, Doug McIllroy with 6 lines of shell. Or Nile/Gezira, a modern graphics subsystem (think Cairo or Quartz) in ~500LOC. Yes, 500LOC. So we need to figure out what makes these examples work so much better and how we can apply what we've learned to make our lives better. That's what I am doing, care to join me? [5] [1] http://queue.acm.org/detail.cfm?id=1039523 [2] https://www.youtube.com/watch?v=B5hGmXFHQS4 [3] https://www.youtube.com/watch?v=wKY-0atLDAo [4] https://www.youtube.com/watch?v=0OAtHOLIg84 [5] http://objective.st [6] http://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys-…