Created attachment 1223944 [details] reproducer.sh Description of problem: rpm-build 4.13.0-1 broke recursive parallel builds using GNU make. Minimal reproducer is attached. This bug manifests itself only if make is called with multiple jobs, e.g. -j16, and when rpmbuild itself is called from parallel make. Version-Release number of selected component (if applicable): rpm-build-4.13.0-1.fc24.x86_64 (broken) rpm-build-4.13.0-0.rc1.27.fc24.x86_64 (latest functional version) How reproducible: 100 % Steps to Reproduce: 0. Prepare environment with rpmbuild and GNU make. 1. Download attached reproducer.sh and all other files into one working directory. 2. Run reproducer.sh Actual results: Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.11YS3o + umask 022 + cd /home/pspacek/rpmbuild/BUILD + cp /home/pspacek/rpmbuild/SOURCES/Makefile /home/pspacek/rpmbuild/BUILD/ + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.OjLUrR + umask 022 + cd /home/pspacek/rpmbuild/BUILD + /usr/bin/make -O -j8 make[1]: Entering directory '/home/pspacek/rpmbuild/BUILD' sleep "1" echo "1" 1 make[1]: Leaving directory '/home/pspacek/rpmbuild/BUILD' make[1]: *** write jobserver: Bad file descriptor. Stop. make[1]: *** Waiting for unfinished jobs.... make[1]: Entering directory '/home/pspacek/rpmbuild/BUILD' sleep "2" echo "2" 2 make[1]: Leaving directory '/home/pspacek/rpmbuild/BUILD' make[1]: *** write jobserver: Bad file descriptor. Stop. error: Bad exit status from /var/tmp/rpm-tmp.OjLUrR (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.OjLUrR (%build) Makefile:2: recipe for target 'doit' failed Expected results: It should work as it worked before, with rpm-build-4.13.0-0.rc1.27.fc24.x86_64. Additional information: This broke FreeIPA build workflow and costs us time becaus we have to run builds sequentially.
Created attachment 1223945 [details] Makefile
Created attachment 1223946 [details] test.src.rpm
This is log from RPM transaction which upgrades rpm-build to the broken version: Packages Altered: Upgraded rpm-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-build-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-build-libs-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-devel-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-libs-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-plugin-selinux-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-plugin-systemd-inhibit-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-python-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates Upgraded rpm-python3-4.13.0-0.rc1.27.fc24.x86_64 @fedora Upgrade 4.13.0-1.fc24.x86_64 @updates
[brain@x1carbon tpm]$ rpmbuild --rebuild ./test-1.0-1.fc24.src.rpm Installing ./test-1.0-1.fc24.src.rpm warning: user pspacek does not exist - using root warning: group pspacek does not exist - using root warning: user pspacek does not exist - using root warning: group pspacek does not exist - using root Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.OKdH5n + umask 022 + cd /home/brain/rpmbuild/BUILD + cp /home/brain/rpmbuild/SOURCES/Makefile /home/brain/rpmbuild/BUILD/ + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.gm5skv + umask 022 + cd /home/brain/rpmbuild/BUILD + /usr/bin/make -O -j4 sleep "1" echo "1" 1 sleep "2" echo "2" 2 sleep "3" echo "3" 3 + exit 0 Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/brain/rpmbuild/BUILDROOT/test-1.0-1.fc26.x86_64 Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.wBRgUK + umask 022 + cd /home/brain/rpmbuild/BUILD + /usr/bin/rm -rf /home/brain/rpmbuild/BUILDROOT/test-1.0-1.fc26.x86_64 + exit 0 Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.EsHUx0 + umask 022 + cd /home/brain/rpmbuild/BUILD + exit 0 I don't see any error. Please provide exact reproducer. Regarding "-O" flag, it is just for synchronizing output. I guess you are doing something wrong in makefile. """ -O[type], --output-sync[=type] When running multiple jobs in parallel with -j, ensure the output of each job is collected together rather than interspersed with output from other jobs. If type is not specified or is target the output from the entire recipe for each target is grouped together. If type is line the output from each command line within a recipe is grouped together. If type is recurse output from an entire recursive make is grouped together. If type is none output synchronization is disabled. """
Yes I can reproduce this. If make breaks with added -O then it is a bug in make, switching component. Running without -O I make is warning about the construct, so what you're doing might not be that hot an idea to begin with: > Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.reWm3s > + umask 022 > + cd /home/pmatilai/rpmbuild/BUILD > + /usr/bin/make -j4 > make[1]: warning: -jN forced in submake: disabling jobserver mode. Anyway, this is TRIVIAL to work around as you're just as free to call "make" in whatever manner you wish from %build as you've always been, so complaining how this is costing you time is a bit thick really. As in: -O is treated as last-option-wins case so you can just use eg: "%make_build -Onone" Or -Orecursive which appears to work too in this case. Or just do the old-fashioned thing and call it "make %{?_smp_mflags}" like you had to do with older rpm versions.
As for the bug in make, the warning about forced -j and disabling jobserver is ominuously missing when -O is added. And then there's the error about bad file descriptor about jobserver: make[1]: *** write jobserver: Bad file descriptor. Stop.
@Igor, the very exact reproducer is already provided in bug description. To reproduce the issue you have to use script reproducer.sh. It is not sufficient to call rpmbuild --rebuild on the srpm. "%make_build -Onone" works as workaround. It breaks as soon as I try make -j2 -Oline. Thank you for debugging this!
[brain@x1carbon tpm]$ curl https://bugzilla.redhat.com/attachment.cgi?id=1223944 -o reproducer.sh % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 18 100 18 0 0 12 0 0:00:01 0:00:01 --:--:-- 12 [brain@x1carbon tpm]$ curl https://bugzilla.redhat.com/attachment.cgi?id=1223945 -o Makefile % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 53 100 53 0 0 41 0 0:00:01 0:00:01 --:--:-- 41 [brain@x1carbon tpm]$ curl https://bugzilla.redhat.com/attachment.cgi?id=1223946 -o test.src.rpm % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5993 100 5993 0 0 4133 0 0:00:01 0:00:01 --:--:-- 4135 [brain@x1carbon tpm]$ bash -x reproducer.sh + make -j16 breakit rpmbuild --rebuild test-1.0-1.fc24.src.rpm error: cannot open test-1.0-1.fc24.src.rpm: No such file or directory make: *** [Makefile:2: breakit] Error 1 [brain@x1carbon tpm]$ mv test.src.rpm test-1.0-1.fc24.src.rpm [brain@x1carbon tpm]$ bash -x reproducer.sh + make -j16 breakit rpmbuild --rebuild test-1.0-1.fc24.src.rpm Installing test-1.0-1.fc24.src.rpm warning: user pspacek does not exist - using root warning: group pspacek does not exist - using root warning: user pspacek does not exist - using root warning: group pspacek does not exist - using root Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.70A2uM + umask 022 + cd /home/brain/rpmbuild/BUILD + cp /home/brain/rpmbuild/SOURCES/Makefile /home/brain/rpmbuild/BUILD/ + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.yFJEbd + umask 022 + cd /home/brain/rpmbuild/BUILD + /usr/bin/make -O -j4 make[1]: Entering directory '/home/brain/rpmbuild/BUILD' make[1]: warning: -jN forced in submake: disabling jobserver mode. make[1]: Leaving directory '/home/brain/rpmbuild/BUILD' make[1]: Entering directory '/home/brain/rpmbuild/BUILD' sleep "1" echo "1" 1 make[1]: Leaving directory '/home/brain/rpmbuild/BUILD' make[1]: Entering directory '/home/brain/rpmbuild/BUILD' sleep "2" echo "2" 2 make[1]: Leaving directory '/home/brain/rpmbuild/BUILD' make[1]: Entering directory '/home/brain/rpmbuild/BUILD' sleep "3" echo "3" 3 make[1]: Leaving directory '/home/brain/rpmbuild/BUILD' + exit 0 Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/brain/rpmbuild/BUILDROOT/test-1.0-1.fc26.x86_64 Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.Fei2dM + umask 022 + cd /home/brain/rpmbuild/BUILD + /usr/bin/rm -rf /home/brain/rpmbuild/BUILDROOT/test-1.0-1.fc26.x86_64 + exit 0 Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.NUQfll + umask 022 + cd /home/brain/rpmbuild/BUILD + exit 0 [brain@x1carbon tpm]$ make --version GNU Make 4.2.1 Built for x86_64-redhat-linux-gnu Copyright (C) 1988-2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
Looks like its been fixed upstream then, good. Fedora 24 (for which this bug is filed) has make 4.1 though, and with that this is reproducable.
Okay. I confirm that make-1:4.1-5.fc24.x86_64 is broken and that %make_build works fine with make-1:4.2.1-1.fc26.x86_64. Thank you for you help, guys!