Neil Mitchell

> The first main disadvantage is that they require the kernel to support syscall tracing, which essentially means they only work on Linux. I have Ideas™ for how to get this working on macOS without disabling SIP, but they're still incomplete and not fully general; I may write a follow-up post about that. I don't yet have ideas for how this could work on Windows, but it seems possible. On Windows, Linux, and also macOS with SIP disabled (as implied, disabling is a bad idea), the https://github.com/jacereda/fsatrace executable exists today and can trace filesystem access. It is used by the Shake build system. In particular, https://neilmitchell.blogspot.com/2020/05/file-tracing.html mentions that Shake copies system binaries to temporary folders to workaround the SIP protection. That blogpost also mentions other problems and solutions (like library preloading).

The shake build system (a general-purpose build system similar-to/better-than make) has a "prune" feature for exactly this purpose: http://neilmitchell.blogspot.com/2015/04/cleaning-stale-files-with-sha… But I think the best solution (that also works with make) is to have a "make dist" target that creates a final .tar.gz archive of the result. If the rule is written properly then it won't contain any stale files. The disadvantage is for large project it may be slow, but you are not supposed to use this rule during development (where it is useless anyway), only for releases (which still can be built incrementally -- only the final .tar.gz needs to be created from scratch)

He also has two follow-up posts on his blog about build systems. They're both worth reading. Small Project Build Systems - https://neilmitchell.blogspot.com/2021/09/small-project-build-systems.… Huge Project Build Systems - https://neilmitchell.blogspot.com/2021/09/huge-project-build-systems.h…

This is in fact one part of the post that was based on an unironic kernel of truth. I've wrestled with this more than most people. In particular I tried to fit cargo into the Fuchsia build system, with only partial success. That approach has been abandoned and Fuchsia now translates Cargo manifests into GN build rules, which seems to be the prevailing approach for polyglot builds, especially with Bazel. However, the experience when doing pure Rust is night and day compared with stuff like CMake. And even in a polyglot build, the Cargo manifest is a reasonably good starting point, as argued recently by matklad[1]. Obviously, build systems and package managers at large scale are very challenging to get right, otherwise we'd have a good solution by now. I personally think Bazel is a pretty good step in the right direction, but it's very clunky and awkward for smaller projects. I found Neil Mitchell's classification[2] useful. Cargo is best-in-class for the "small" and arguably "medium" cases, but starts showing its limitations beyond that. Bazel is the reverse, one of the few really workable choices for "huge" but not a great experience otherwise. [1]: https://matklad.github.io/2023/03/28/rust-is-a-scalable-language.html [2]: https://neilmitchell.blogspot.com/2021/09/small-project-build-systems.…

> Even for someone reasonably experienced in functional programming, there are tons of new concepts you need to master to be minimally productive in the language. Mostly agree, but it does depend on how you define minimally productive. I also believe this is more of a pedagogy problem. For instance, someone can tell me what their definition of being minimally productive is in Haskell and if it is within my skills I can get them there pretty quickly. > Beyond that, it introduces a whole new problem class of space leaks that you need to reason about. I feel like "whole new problem class" is unfair since other languages have memory leaks. Also, there are techniques developing for dealing with space leaks just like other languages have techniques for memory leaks: http://neilmitchell.blogspot.com/2015/09/detecting-space-leaks.html > It's not a bad language, but it is a much more radical leap from anything else you're likely to have tried than Swift is. Perhaps I've drunk the koolaid, but it doesn't seem that radical after having learned it. > OCaml is fairly verbose Agreed, it's the main thing that has put me off of learning what is probably an awesome language. To be honest I just can't get over all the let's :/ I find this a little hypocritical since I found it easy to get over lisps/schemes parenthesis. > still doesn't have a great parallelism story AFAIK Right, but they are apparently fixing that fairly quickly: https://news.ycombinator.com/item?id=9582980 To…

I must respectfully disagree that Haskell's memory footprint is simply 'low'. This is because the memory footprint of a given Haskell program is not at all transparent, and Haskell is notorious for leaking memory in a maddeningly opaque fashion [1, 2, 3, 4]. Space leaks might be relatively straightforward to diagnose and fix for a true domain expert, but I would not want to have to rely on someone having such abstruse knowledge in a production application. It goes without saying that a space leak in a production app is a really, really bad thing. Although, I suppose one's choice of Haskell is a function of one's own risk/reward profile. Haskell and its failure modes are hard to understand. That induces extra risk that some people (myself included) might be uncomfortable with. That said, I am now enthusiastically following you guys and hope to see the proverbial averages get sorely beaten. [1] http://neilmitchell.blogspot.com/2013/02/chasing-space-leak-in-shake.h… [2] http://blog.ezyang.com/2011/05/calling-all-space-leaks/ [3] http://blog.ezyang.com/2011/05/space-leak-zoo/ [4] http://blog.ezyang.com/2011/05/anatomy-of-a-thunk-leak/

I forgot about lenses, thanks. About non-locality of macros vs. functions: Macros, being functions, provide this kind of assurance, too at the syntax tree level: you only operate on the local, current subtree. They can indeed modify the meaning of symbols in that subtree, but I think you are taking things to the extreme. Off course, an arbitrarly nasty macro "far up in the tree" (how deep are the typical functions, anyway?) could do horrible things. Also, we must ban sugar because if someone eats too much sugar, that someone dies. However, the average usage is more subtle. For example, when you DEFUN a function, you are registering a function in your current environment (side-effects) through a macro, which does not dramatically change the semantics of the body of the function, compared to, say, an anonymous LAMBDA (the only change is, I think, the ability to RETURN-FROM a named function). CL is not a functional language: you can loop over lists or across vectors, you can setf special variables and mutate strings. You have static and dynamic non-local exits (return, throw/catch, signal) and even goto's (tagbody). That does not mean that you should use them all and go against best practices and common sense. Just because a feature could be misused does not make it irrelevant. For the record, I have nothing against functional languages (or any other language, really (except of course PHP ;-)). It just seem unfair to attack macros. They should be avoided whenever…

This is bullshit FUD from a known troll. But even ignoring the source, the arguments are extremely weak. (It was also a good idea to close the question on SO. SO is not a philosophy site, it's a site for specific questions. "Why doesn't anyone like green?" is not a specific question. "How do I make links green" is. Sigh.) Anyway, let's briefly entertain each argument. Lack of compelling examples: there is a whole book, Real World Haskell, and there are 2000+ modules on HackageDB that you can read. There is darcs. There is xmonad. There is GHC. Etc., etc., etc. If you want to read real Haskell code, it's one click away on any documentation page. This argument is bullshit. Lack of success stories: I like: http://neilmitchell.blogspot.com/2007/05/does-xmonad-crash.html. What is more compelling than "I used a theorem prover to ensure that my GUI app can't crash"? I know "industry" likes stories about some happy CTO that decided to use Foo and improved revenues 8 percent. But the reality is, nobody actually cares about this. People use languages because they know them and like them. Why did I introduce Haskell at the bank where I work? Not because of some happy CTO story. I did it because I was confronted with a problem to solve, and decided that I would be best able to solve it with Haskell. I was right; the project was a success. I don't get to work on it much anymore because it never breaks. Fail! User unfriendly: apt-get install ghc6. cabal install…

> Anyway, I can't see how this would be more useful than a classic optimizing compiler, except for languages like javascript, which is interpreted by third party clients. Pretty much every language can benefit from optimizations like constant folding and inlining (even C, yes); those techniques are subsets of partial evaluation, and partial evaluation is, apparently, a subset of supercompilation. And we should expect these general classes of optimization techniques to offer speedups. We know that classic optimizing compilers (GCC?) miss a lot of opportunities; this is especially obvious when you compare C and FORTRAN numeric performance - presumably all the FORTRAN optimizations could also be done in C by a sufficiently smart compiler (they're both Turing-complete languages, after all), but the C compilers can't reliably figure out when to do them. Even a language like Haskell which practically goes out of its way to let the compiler optimize however it wants can benefit: http://neilmitchell.blogspot.com/2007/12/supercompilation-for-haskell.… (Although I've read the paper and was confused; it looked more like partial evaluation to me - inlining and rewriting at compile-time until a fixed-point is reached - than this runtime supercompilation stuff.)

There's this concept of "forward build system" that other people too have nibbled at. See: * http://neilmitchell.blogspot.com/2021/09/small-project-build-systems.h… * https://dl.acm.org/doi/abs/10.1145/3497775.3503687 Primarily trying to reduce the high learning curve of complete build systems like bazel/please/buck. The author too seems to be heading in this direction

Here's an even more confusing title https://neilmitchell.blogspot.com/2007/03/describing-haskell-to-ada-pr…

Interestingly, just recently, Neil Mitchell has published an article claiming that this is a fallacy and in practice, people are forced to back-pedal on it. Ban recursion and unbounded loops. Proclaim the language is "Turing incomplete" and that all programs terminate. Declare that Turing incomplete programs are simpler. Have non-technical people conflate terminate quickly with terminate eventually. Realise lacking recursion makes things incredibly clunky to express, turning simple problems into brain teasers. Add recursion. Realise that the everything is better. https://neilmitchell.blogspot.com/2020/11/turing-incomplete-languages.…

If you read this, you should probably also read this very recently blog post[1] by Neil Mitchell. http://neilmitchell.blogspot.com/2020/11/turing-incomplete-languages.h…

I found the Build Systems a la Carte¹ paper, and NDM’s companion write up² to be good comparison of build systems in a more general manner. Caveat: It may skewed having been written by heavy Haskell hitters, and an author of Shake. I’ll note that I didn’t notice any bias, but maybe only because it aligned with mine ;) 1. https://www.microsoft.com/en-us/research/publication/build-systems-la-… 2. http://neilmitchell.blogspot.com/2018/07/inside-paper-build-systems-la…

If you want to learn Haskell or functional language: Haskell for all- http://www.haskellforall.com/ Bryan O'Sullivan's blog- http://www.serpentine.com/blog/ Monday Morning Haskell- https://mmhaskell.com/ Jane Street- https://blog.janestreet.com/ Haskell Weekly- https://haskellweekly.news/ Neil Mitchell's Haskell Blog-http://neilmitchell.blogspot.tw/ Types & Functional Programming- http://bitemyapp.com/ Haoyi's Programming Blog- http://www.lihaoyi.com/post/WhatsFunctionalProgrammingAllAbout.html

here is a method that can help debug space leaks: http://neilmitchell.blogspot.com/2015/09/detecting-space-leaks.html

Barclays Capital: http://neilmitchell.blogspot.co.uk/2016/09/full-time-haskell-jobs-in-l…

This is a way to introduce space leak detection in your tests http://neilmitchell.blogspot.ro/2015/09/detecting-space-leaks.html so that you can sleep well and not be afraid of using Haskell in big projects :)

I find ghcid very useful. It reloads your code automatically when it detects a filechange. https://github.com/ndmitchell/ghcid http://neilmitchell.blogspot.de/2014/09/ghcid-new-ghci-based-ide-ish.h…

I'm not proficient in Haskell, just a bystander, but this isn't the sentiment I get from reading Neil Mitchell's posts on the subject: http://queue.acm.org/detail.cfm?id=2538488 http://neilmitchell.blogspot.com/2013/02/chasing-space-leak-in-shake.h… He appears to use profiling tools and "try them all and see which produces the most useful information", to me that sounds very different from reasoning.

Yes, I was searching for an HTML renderer written in Haskell. It makes me remember HsExcel http://neilmitchell.blogspot.com.ar/2007/03/hsexcel.html

I feel like DJB's ideas behind redo are actually some of his most poorly thought-out ideas. It seems that he thought, 'Make has <problem>. To solve <problem>, a new build system should have <solution>.' In other words, he seems to just used additive solutions instead of subtractive ones. [1] Most build systems seem to be like that, adding features on top of features. The end result is usually complicated and finicky. I think that's why build systems are among the software developers hate most. And I do mean hate. (The first time and only time I saw an acquaintance, a normally calm person, have an outburst was talking about a build system.) I think the ideal build system will be subtractive, not additive. So I guess this is as good a time as any to talk about my current project, a build system. :) (Most of this will be copied from two comments I made on lobste.rs.) First, there is a paper that already lays out the features needed for an optimal build system: A Sound and Optimal Incremental Build System with Dynamic Dependencies. [2] The features are: 1. Dynamic dependencies. 2. Flexible "file stamps." 3. Targets and build scripts are Turing-compete. That's it. Really. The first item is needed based on the fact that dependencies can change based on the configuration needed for the build of a package. Say you have a package that needs libcurl, but only if users enable network features. It is also needed to import targets from another build. I’ll use the libcurl…

Over the years I've collected various links on topics related to Haskell performance. I'll spare you the whole list, but here are a few that may be of some use. Haskell Performance Patters - Johan Tibell (2012) http://johantibell.com/files/haskell-performance-patterns.html#(1) Performance (Haskell Wiki) https://wiki.haskell.org/Performance Detecting Space Leaks - Neil Mitchell (2015) http://neilmitchell.blogspot.com/2015/09/detecting-space-leaks.html The Haskell performance checklist - Chris Done, et. al. (2017 - 2019) https://github.com/haskell-perf/checklist/blob/master/README.md ThreadScope (Haskell Wiki) https://wiki.haskell.org/ThreadScope Top tips and tools for optimising Haskell - Will Sewell (2015) https://making.pusher.com/top-tips-and-tools-for-optimising-haskell/