Hide Forgot
Description of problem: When I do make -j 4 -l 9, I only get one process at a time rather than the 4, limited by load average, I expect. This worked properly in Fedora 13. Version-Release number of selected component (if applicable): make-3.82-3.fc14.i686 How reproducible: Always Steps to Reproduce: 1. make -j 4 -l 9 Actual results: One job at a time Expected results: Four jobs at a time Additional info: make -j4 works correctly. make -j -l9 also seems to work correctly. It's just when I specify both a number of jobs and a load average that doesn't work.
Strange, the algorithm didn't change since before f-13. It seems to kinda-sorta work here where I'm sitting. I'm using the following makefile to test: all: $(shell rm -vf tmp > /dev/stderr) $(shell seq -f job-%g 100) job-%: @echo "1" >> tmp @echo "enter $@; $$(wc -l<tmp) jobs; load $$(cat /proc/loadavg)" @for i in $$(seq $*0); do for j in $$(seq 10); \ do for k in $$(seq 10); do :; \ done; done; done @sed -i 1d tmp @echo "exit $@; $$(wc -l<tmp) jobs; load $$(cat /proc/loadavg)" # Umm, in my locale, I have to write 1.45 as 1,45, and... $ ./make-3.82/make -j 10 -l 1,45 -f ~/tmp/load.mk удалён «tmp» # ... this just says "tmp erased". enter job-1; 4 jobs; load 0.51 1.14 1.17 7/321 18340 enter job-2; 5 jobs; load 0.51 1.14 1.17 8/321 18341 [...] The output oscillates between 1 and about 8 jobs in flight here, and the load is about 1.2 to 1.9. make takes the system-level load into account, so if you have just finished building, or someone else is occupying the machine, it will keep launching one job at a time at the beginning, before the load values get down a bit. Anyway, it seems to be doing approximately the right thing.
Yes, I seem to have misidentified what exactly wasn't working as expected; passing the -l option seems to be irrelevant to the issue. I apologize for wasting your time. The actual problem seems to be the combination of remaking the Makefile and -j N from MAKEFLAGS in the environment. Given this makefile: ---- things = thing1 thing2 thing3 thing4 all: $(things) Makefile: foo.c touch Makefile clean: touch foo.c $(things): @echo @echo $@ start MAKEFLAGS: $(MAKEFLAGS); sleep 4; echo $@ end ---- "make clean; make -j4" works fine. "make clean; MAKEFLAGS=-j make" also works fine. "make clean; MAKEFLAGS=-j4 make" breaks; the MAKEFLAGS passed down to sub-makes has no -j option, so we only get one job at a time.
I believe this is intentional, due to jobserver. From the manual: > The ‘-j’ option is a special case (see Parallel Execution). If you set it to > some numeric value ‘N’ and your operating system supports it (most any UNIX > system will; others typically won't), the parent make and all the sub-makes > will communicate to ensure that there are only ‘N’ jobs running at the same > time between them all.
Also, I can't seem to be able to reproduce it. $ cat load.mk all: batch-1 batch-2 batch-3 batch-4 erase:; $(shell rm -vf tmp > /dev/stderr) batch-%: erase sleep .$$(($$RANDOM / 32)) $(MAKE) -f load.mk $(shell seq -f job-%g $$((10 + $*)) 10 100) job-%: @echo "1" >> tmp @echo "enter $@; $$(wc -l<tmp) jobs; $(MAKEFLAGS)" @for i in $$(seq $*0); do for j in $$(seq 10); do for k in $$(seq 10); do :; \ done; done; done @sed -i 1d tmp @echo "exit $@; $$(wc -l<tmp) jobs; $(MAKEFLAGS)" $ MAKEFLAGS=-j4 make -f load.mk [...] enter job-11; 4 jobs; w --jobserver-fds=3,4 -j exit job-13; 3 jobs; w --jobserver-fds=3,4 -j enter job-23; 4 jobs; w --jobserver-fds=3,4 -j exit job-11; 3 jobs; w --jobserver-fds=3,4 -j enter job-21; 4 jobs; w --jobserver-fds=3,4 -j exit job-12; 3 jobs; w --jobserver-fds=3,4 -j enter job-22; 4 jobs; w --jobserver-fds=3,4 -j exit job-14; 3 jobs; w --jobserver-fds=3,4 -j [...] ("make -f load.mk -j 4" behaves the same way) $ rpm -q make make-3.82-3.fc14.x86_64
Your load.mk doesn't seem to rebuild itself; that seems to necessary to reproduce the bug.
Duh, I missed your example makefile :-/ It indeed reproduces the problem, and I don't think it's related to the special handling of -j anymore.
Created attachment 515601 [details] Erm... let's call it a fix I'm turning this in reluctantly. It removes the issue that you see, it doesn't break the test suite, but I had to turn off a bit of logic that someone put there for reasons that I don't understand. I'll pass this through upstream and see what they have to say.
It is longterm annoying, could there be a Fedora backport before a new upstream release? (F-16 would be enough for me.)
make-3.82-8.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/make-3.82-8.fc16
Package make-3.82-8.fc16: * should fix your issue, * was pushed to the Fedora 16 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing make-3.82-8.fc16' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2011-15358 then log in and leave karma (feedback).
make-3.82-8.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.