Hi,
Let me tell 2 real-life examples:
1. The one that made me find this issue:
My hobby project takes 45 seconds to compile, out of which the biggest .cc
file that I modify most of the time is 15 seconds. In order to understand
what's going on (it was not written by me, I'm just fixing bugs and adding
new features) I often insert statements like printf("A\n"); so that I'll
see the code is there. I copy-paste it to multiple places, replacing the
letter with B, C etc.
Once I forgot to replace the letter. Saved, launched make, and immediately
realized this mistake. So went back, fixed the file, saved, and
re-launched make (probably just pressed the Up arrow and Enter, not even
caring whether the previous make had already finished or not). I got hit
by this race, and seeing only A's in the program's output mislead me into
incorrectly thinking that B is not reached in the code, wasting a lot of
time.
I prefer uninterrupted runs of development with full focus for maybe an
hour or two at a time, re-compiling the app every minute or two in certain
phases of development. Standing up and grabbing a coffee is not a viable
option. (And yes, I do know about gdb, I just prefer printf-debugging
whenever I can.)
2. Another example that also happens to me relatively frequently:
I download a mid-size piece of free software (let's say it compiles in 10
minutes) and apply a couple of patches to it. Let's assume applying the
patches takes 5 minutes (I have to figure out the order, whether they're
-p0 or -p1, some of them slightly conflict and I have to manually merge).
If I perform these two steps sequentially, it's a total of 15 minutes. So,
instead, what I do is: Start a make (or "make -k" preferably), and
immediately afterwards start applying the patches. This way we use the CPU
time and my time in parallel. Finally run "make" once again to catch up.
This can reduce the total time in this example from 15 minutes to hardly
more than 10 minutes. Alas, chances that I'll be bitten by this race
condition is not negligible.
---
> I don't think make should be worried about this potential race; it's an
> obvious case of user error with an easy discipline for the user to apply
> to avoid any problems.
Is it really obvious? Is it documented somewhere? Do you really think
pretty much all users of make think about it? I, for one, first used make
(not just typed the command, but understood its basics, wrote a tiny
Makefile for a tiny project) about 18-20 years ago, yet this race condition
never occurred to me until recently.
Many years ago I was wondering what "make" would do if the source and
destination files had the exact timestamp. I had better things to do than
actually try it out or look it up, I just assumed that developers had put
proper thought to it and found the solution that Just Works™. Nowadays
with nanosecond timestamps I just don't care anymore.
Had I thought about it, I probably would have just guessed that I'm not the
first one with this race condition problem, would have assumed that there's
already a clever solution for this in place which, again, Just Works™.
Seeing that there isn't any, I'm kindly asking you to think about it and
come up with one.
> It's a rough-and-ready tool for developers, with
> no pretenses of being intended for use by those who don't think about
> how their tools work.
Do you really claim I should deeply think about all corner cases of the
tools before using them? Given that "make" is just a tiny little fraction
of all the tools I'm using, it's practically impossible. In fact, I think
it's quite the opposite! The creators of such tools should do their best
to make sure that the tools work reliably as expected.
I didn't think about the internals of make -- why should have I? I knew
the goal it was serving: to create an up-to-date build in as little overall
time as possible. With my particular development pattern it fails to
achieve this goal. I don't think it's me who should've thought of this,
nor that it's me who should change my workflow (touch the file after make
completes, or not save while make is running, or start drinking coffee, or
accept that I can not parallelize my time with the CPU's time, effectively
taking away long minutes of my life many times for no good reason).
In my firm opinion, it should be make's job to do whatever it can to
produce an up-to-date build under any circumstance. Hey, that's what make
is all about!
> Of course, if a file-system had a separate "up to date at" file-stamp,
> that make can set (to the newest dependency's time-stamp) and later
> read, or if make were to store this same datum in a separate database,
> it would solve your problem. It would also make it possible to have
> make rules that tentatively build the new file, compare to existing,
> replace if (meaningfully) different but only revise "up to date at"
> otherwise.
Such a tiny "database" could indeed solve this problem without race
condition. I would be more than happy to see this implemented! It
wouldn't have to be a "database", just a simple ephemeral text file that's
removed on clean exit of make, listing all the currently running rules
that'd have to be rerun if this file exists on startup (meaning that the
previous run was interrupted or crashed).
> As you note, every command that might be used in a make rule would need
> to do this "right", which is a prohibitive requirement.
>
> Requiring every command that could be used as a make rule to abide by
> this is, again, prohibitive. [...]
Some of these ideas I've thrown in might not work in practice, I agree.
Please note that I had a simple propsal: re-check the timestamps of input
files when a rule completes and re-run (moving away or dating back the
previous output file first) if any of them changed. This would not elimite
the race condition completely, but would make the window magnitudes smaller
(e.g. fraction of a millisecond instead of 15 seconds), and as such, would
already be a great improvement!
cheers,
egmont
_______________________________________________
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
