Paul Smith
make.mad-scientist.net · Personal blog · on HN since 2016
Paul Smith, maintainer of GNU make, writes about Makefile practice, including auto-dependency generation, the jobserver and metaprogramming.
What HN saysHN recommends it as a resource on GNU make, including its metaprogramming series, and links it for dependency generation in builds.
22HN people shared it
16linked it in comments
12stories submitted
165story points
1 year agolast shared
2016people per year2026
Most-shared posts
- 1C/C++ Auto-Dependency Generation for GNU make with GCC10 people · 4 points · first shared 2016 · discussion on HN
- 2The GNU make jobserver Implementation (2015)5 people · 142 points · first shared 2016 · 40 comments on HN
- 3Rules of Makefiles5 people · 19 points · first shared 2016 · 1 comments on HN
- 4Homepage1 people · first shared 2024
- 5Multi architecture builds1 people · first shared 2023
- 6Metaprogramming1 people · first shared 2019
- 7How not to use vpath1 people · first shared 2018
What HN people wrote when they linked it
matheusmoreira · 2024-09-21
More resources:
https://make.mad-scientist.net
https://www.cmcrossroads.com/users/john-graham-cumming
https://nostarch.com/gnumake
https://gmsl.jgc.org
mturmon · 2019-02-02
I also recommend this book. The series on “meta programming make” is also quite good: http://make.mad-scientist.net/category/metaprogramming/
o11c · 2025-06-20
There are some dangerous and subtle problems with this tutorial.
In order to handle long options and empty short options, when searching MAKEFLAGS you really need to do:
ifneq (,$(findstring t,$(firstword -$(MAKEFLAGS)))) If you need compatibility with the ancient version of `make` shipped by default on OS X, not that it is missing a lot of functions and features, some subtly.
Most other problems are either obvious typos or due to violating best-practice style, so I won't go into them.
Note also that `load` is actually more portable than `guile`, but make sure you use the correct compiler in case you're cross-compiling the main project.
***
Really you should just read Paul's Rules of Makefiles [1], then the GNU make manual [2]. Some of the other manuals related to GNU build tools are also relevant even if you aren't using the particular tool. I also have a demo project [3] if you just want something that works and isn't complicated.
[1]: https://make.mad-scientist.net/papers/rules-of-makefiles/
[2]: https://www.gnu.org/software/make/manual/
[3]: https://github.com/o11c/makefile-demo
epr · 2023-04-24
>> Isn’t that just Make?
> Yep! But what if we add mappings and collections?
These are called pattern rules in make:
https://www.gnu.org/software/make/manual/html_node/Pattern-Rules.html
This is the best resource I've found on make to get a low friction build system up and running. Some parts are a bit dense if you're relatively new to make or used to someone else setting it up for you, but if you have the gnu make manual side-by-side for reference you should be good to go:
https://make.mad-scientist.net/papers/advanced-auto-dependency-generat…
If you've ever been frustrated with how time consuming it is to manually write make rules then the above method will probably be a night-and-day difference from what you're used to.
I'd also strongly suggest either disabling builtin implicit rules with -r or reading up on all of them here:
https://www.gnu.org/software/make/manual/html_node/Catalogue-of-Rules.…
AceJohnny2 · 2023-12-04
Going on a tangent here, but properly integrating support for such "discovered" dependencies in GNU Make can get a bit complicated.
In particular, you probably want to use `-include` instead of `include`, to continue even a given .dep file doesn't exist (as on the first run).
The GNU Make maintainer wrote in-depth about ways to do it here:
https://make.mad-scientist.net/papers/advanced-auto-dependency-generat…
lmz · 2023-08-12
This guide, maybe? https://make.mad-scientist.net/papers/advanced-auto-dependency-generat…
dwheeler · 2023-04-25
_TwoFinger · 2021-05-17
> Not sure I will write my Makefile generator, since my project is also not that big.
gcc has a family of command-line options starting with -M than can generate dependencies as a byproduct during compilation. The generated files are in Makefile format, and only need to be included in your main Makefile.
https://make.mad-scientist.net/papers/advanced-auto-dependency-generat…
toast0 · 2021-02-09
> Correct me if I'm wrong, but don't you have to formally map out all of your inputs and outputs for partial recompilation to work?
Make needs that information, but that doesn't mean you need to be the one to catalog it. I found this article pretty useful [1]; in a nutshell, many compilation tools (including gcc, clang, and erlc) are happy to write a file that lists the compile time resources they used, and you can use that file to tell Make what the dependencies are. It's a bit meta, so it made my brain hurt a bit, but it can work pretty well, and you can use it with templated rules to really slim down your Makefile.
(I've tested this with GNU Make; don't know if it's workable with BSD Make)
[1] http://make.mad-scientist.net/papers/advanced-auto-dependency-generati…
henry_flower · 2018-03-06
yes, target-specific vars are neat.
I've thought about adding a couple of sentences about vpath (in a cautious style akin to "think twice before using it" [0]), but ultimately decided that the topic is too obscure; besides I practically never use vpath myself.
[0] http://make.mad-scientist.net/how-not-to-use-vpath/
lifthrasiir · 2017-08-18
> It's a matter of running it to generate some sort of dependency file and including it in the makefile.
This is not as simple as it sounds. Make has a fundamental limitation that dependencies cannot be generated on the fly, and current workarounds for dependency-generating dependencies are very messy [1].
[1] http://make.mad-scientist.net/papers/advanced-auto-dependency-generati…
deorder · 2017-08-18
Or you can let the compiler generate the header-dependency Makefiles for you:
http://make.mad-scientist.net/papers/advanced-auto-dependency-generati…
You already tried that? I do not find it painful to set up.
psi-squared · 2016-11-13
If I've read this correctly, the 'sem' mode lets you submit several lots of jobs with an overall limit on the total number of tasks running at a time (rather than one limit per lot of jobs).
That on its own is super useful for what I'm working on right now. But what would make it even more useful, is: can you get GNU make to use 'sem' instead of its own jobserver? That way I could run almost everything I need to under one overall task limit, and that would be really nice to have.
(For this reason, I'm a fan of the idea that every program with its own 'parallel execution' mode should be able to interact with a common jobserver. The 'make' jobserver is, as far as I know, the simplest, and should be pretty easy to support: http://make.mad-scientist.net/papers/jobserver-implementation/ )
rout39574 · 2016-04-03
Make has done 'dynamic rules' for a long, long time.
http://make.mad-scientist.net/papers/advanced-auto-dependency-generati…
opk · 2016-03-05
This actually works fine in GNU make too. The parent make acts as a job server. The implementation is ingenious, see http://make.mad-scientist.net/papers/jobserver-implementation/
chubot · 2016-06-03
OK, I see. Your implementation sounds a little like how "make -j" is actually implemented (in C).
This basically sounds like implementing a semaphore with a pipe.
http://make.mad-scientist.net/papers/jobserver-implementation/
"""
Rather than using complex schemes such as a central server, shared memory, sockets, etc. for synchronizing the various instances of make, he suggested we use the simplest, most basic UNIX construct: a pipe. Not only is this easy, but it’s also a relatively portable concept. Most operating systems these days implement some sort of pipe feature, because that paradigm is so useful.
The idea is ingeniously simple: the initial, top-level make creates a pipe and writes N one-byte tokens into the pipe. That pipe is shared between that make and all submakes, and any time any make wants to run a job it first has to read a token from the pipe. Once the job is complete, it writes the token back to the pipe. Since there are only N tokens, you know that you will never invoke more than N jobs.
"""