Description of problem: Can't build 'nosrc' srpms from JPackage for a 32-bit target using mock, on a 64-bit host. Version-Release number of selected component (if applicable): 0.9.13-1.fc10 How reproducible: Every time Steps to Reproduce: 1. Download a 'nosrc' srpm from JPackage, e.g. 'java-1.6.0-sun': 'wget http://mirrors.dotsrc.org/jpackage/5.0/generic/non-free/SRPMS/java-1.6.0-sun-1.6.0.11-1jpp.nosrc.rpm' 2. Download the 32-bit 'bin' file from Sun, corresponding to the version of the 'nosrc' rpm: 'wget -O jdk-6u11-linux-i586.bin "http://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/VerifyItem-Start/jdk-6u11-linux-i586.bin?BundledLineItemUUID=qz5IBe.phWsAAAEf6CEk4zLq&OrderID=kzJIBe.pYgYAAAEf2iEk4zLq&ProductID=abtIBe.ovJUAAAEdeDBGb7Et&FileName=/jdk-6u11-linux-i586.bin"' 3. Init the mock chroot: 'mock -v -r fedora-10-i386 --init' 4. Use mock to copy in the bin file to the SOURCES directory in the chroot: 'mock -v -r fedora-10-i386 --copyin jdk-6u11-linux-i586.bin /builddir/build/SOURCES/' 5. Confirm the bin file was copied into the chroot: ls -l '/var/lib/mock/fedora-10-i386/root/builddir/build/SOURCES/' 6. Use mock to rebuild the srpm, with the '--no-clean' option: 'mock -v -r fedora-10-i386 --no-clean --rebuild java-1.6.0-sun-1.6.0.11-1jpp.nosrc.rpm' Actual results: Mock deletes the SOURCES directory so the bin file can't be found, even though the '--no-clean' option was specified: DEBUG: remove tree: /var/lib/mock/fedora-10-i386/root/builddir Expected results: Mock doesn't delete the SOURCES directory and the RPM files are built successfully. Additional info:
I can confirm this is still not fixed for mock 1.1.12 in epel. Despite using the --no-clean flag, the --rebuild command removes the mockbuild user and its home directory /builddir then recreates the user. The verbose flag doesn't show where the srpm gets copied back into the /builddir/build/originals directory. I'd like to use nosrc rpms with mock to handle my university's build of various proprietary packages that we can only legally handle as nosrc rpms (maple, mathematica, matlab) -- Chandler Wilkerson Rice University
I can confirm that this bug is still present on an up-to-date Fedora 16. I am upgrading to Fedora 17 and should be able to confirm if it is present there. As I am trying to package several nosrc RPMs, I would really appreciate this being fixed soon. It is not possible to build nosrc RPMs using mock at present.
(In reply to comment #2) > I can confirm that this bug is still present on an up-to-date Fedora 16. I > am upgrading to Fedora 17 and should be able to confirm if it is present > there. > This is still a problem in Fedora 17 (1.1.28).
Jessica, do you have a couple of the nosrc SRPMS handy? If so would you please attach them to this BZ so I can use them for testing?
Clark, For testing, you could just pick up a regular SRPM, add nosource: 1 to the .spec file (where 1 is the primary .tar.bz source file) and test with that. In my experience, nosrc rpms tend to be too large and unwieldy for testing.
(In reply to comment #4) > Jessica, > > do you have a couple of the nosrc SRPMS handy? If so would you please attach > them to this BZ so I can use them for testing? http://mirrors.dotsrc.org/jpackage/5.0/generic/non-free/SRPMS/ has several.
Created attachment 815938 [details] Keep the build directory when --no-clean specified This is a first step in trying to sort out building nosrc srpms
bump?
Fedorahosted product is retired and nearly invisible. Moving to Fedora product.
Not deleting builddir could be easily implemented in the current version of mock and would actually simplify the code (the part of code has already been isolated for the sake of having --short-circuit). But I think users expect the build directory to be deleted. At least I always thought of --no-clean as an option that let's me build the package without the need to reinstall it's buildrequires, but still having reproducible build (minus the BR part ofc). If we don't delete the build dir there will be garbage left behind the previous build that will affect the build. And mock will need to get a better way to know which files were output by rpmbuild than globbing (and I'm not sure if there is a better way). Otherwise you wouldn't be able to build two different SRPMs in a row: ERROR: Expected to find single rebuilt srpm, found 2. So if this is still necessary, I'd suggest having some other option for this than --no-clean, such as --no-delete-builddir.
Well - from man page: --rebuild If no command is specified, rebuild is assumed. Rebuilds the specified SRPM(s). The buildroot is cleaned first, unless --no-clean is specified. -n, --no-clean Do not clean chroot before building package.
The SOURCE dir is actually deleted when: > /home/msuchy/projects/mock/py/mockbuild/buildroot.py(216)_make_build_user() -> self.doChroot(['/usr/sbin/userdel', '-r', '-f', dets['user']], shell=False, raiseExc=False) because that delete home dir, which is /builddir I would use: - self.doChroot(['/usr/sbin/userdel', '-r', '-f', dets['user']], shell=False, raiseExc=False) + if self.config['clean']: + self.doChroot(['/usr/sbin/userdel', '-r', '-f', dets['user']], shell=False, raiseExc=False) + else: + self.doChroot(['/usr/sbin/userdel', '-f', dets['user']], shell=False, raiseExc=False) But additionally, this: > /home/msuchy/projects/mock/py/mockbuild/buildroot.py(360)_setup_build_dirs() -> os.chown(os.path.join(dirpath, path), self.chrootuid, -1) would fail as well, because --copyin create the files as root:root so mockbuild user can not chown them. I am not sure if better would be copyin let create file as mockbuilder user or skip _setup_build_dirs() if --no-clean was used.
Hmm but then we have problems, because right after: mock -r fedora-20-i386 --init is: $ ls -l '/var/lib/mock/fedora-20-i386/root/builddir/build/' total 28 drwxr-xr-x 2 msuchy msuchy 4096 Nov 19 17:16 BUILD drwxr-xr-x 2 msuchy msuchy 4096 Nov 19 17:16 BUILDROOT drwxr-xr-x 2 msuchy msuchy 4096 Nov 19 17:16 originals drwxr-xr-x 2 msuchy msuchy 4096 Nov 19 17:16 RPMS drwxr-xr-x 2 msuchy msuchy 4096 Nov 19 17:16 SOURCES drwxr-xr-x 2 msuchy msuchy 4096 Nov 19 17:16 SPECS drwxr-xr-x 2 msuchy msuchy 4096 Nov 19 17:16 SRPMS I guess that we should place that chown somewhere else too.
taking as I'm working on this.
Fixed in commit 49de45d. But it will not work anyway until bug 1166121 is resolved.
mock-1.2.3-1.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/mock-1.2.3-1.fc21
mock-1.2.3-1.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/mock-1.2.3-1.fc20
mock-1.2.3-1.el7 has been submitted as an update for Fedora EPEL 7. https://admin.fedoraproject.org/updates/mock-1.2.3-1.el7
mock-1.2.3-1.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/mock-1.2.3-1.el6
Package mock-1.2.3-1.fc21: * should fix your issue, * was pushed to the Fedora 21 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing mock-1.2.3-1.fc21' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-16285/mock-1.2.3-1.fc21 then log in and leave karma (feedback).
(In reply to Miroslav Suchý from comment #11) > Well - from man page: > --rebuild > If no command is specified, rebuild is assumed. Rebuilds > the specified SRPM(s). The buildroot is cleaned > first, unless --no-clean is specified. > -n, --no-clean > Do not clean chroot before building package. What's the difference between "buildroot" and "chroot"?
Good point. s/buildroot/chroot/ in commit 95849c2
mock-1.2.3-1.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.2.3-1.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.2.3-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.2.3-1.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.
Has this actually been fixed? I ran into this bug in 1.2.14 on CentOS 6.7. This is what I see in buildroot.py (in function _make_build_user) checked out from mock tip if self.config['clean']: self.doChroot(['/usr/sbin/userdel', '-r', '-f', dets['user']], shell=False, raiseExc=False, nosync=True) else: self.doChroot(['/usr/sbin/userdel', '-f', dets['user']], shell=False, raiseExc=False, nosync=True) self.doChroot(['/usr/sbin/userdel', '-r', '-f', dets['user']], shell=False, raiseExc=False, nosync=True) Note that the third "userdel -r" is executed unconditionally.
Just to make sure I'm barking up the right bugzilla, what I want to have happen is for "/builddir/build/BUILD" under the chroot to persist between two calls to "mock --buildsrpm --no-clean" (I've run "mock --init" prior to the two buildsrpm calls). The lines of code I pointed to in the previous are not what's deleting the builddir (although the last "userdel -r" call seems to be bug too), but it's the line immediately before that. Line 248 of buildroot.py is util.rmtree(self.make_chroot_path(self.homedir), selinux=self.selinux, exclude=excluded) That unconditionally deletes /builddir under the chroot. It should probably be if self.config['clean']: util.rmtree(self.make_chroot_path(self.homedir), selinux=self.selinux, exclude=excluded) I can create a new bug report if that's the appropriate thing to do.
(In reply to praetorian from comment #28) > Just to make sure I'm barking up the right bugzilla, what I want to have > happen is for "/builddir/build/BUILD" under the chroot to persist between > two calls to "mock --buildsrpm --no-clean" (I've run "mock --init" prior to > the two buildsrpm calls). > > The lines of code I pointed to in the previous are not what's deleting the > builddir (although the last "userdel -r" call seems to be bug too), but it's > the line immediately before that. > > Line 248 of buildroot.py is > > util.rmtree(self.make_chroot_path(self.homedir), > selinux=self.selinux, exclude=excluded) Please note the "excluded" argument - it doesn't remove the whole tree, but keeps selected files/dirs present. It's configured in config_opts['exclude_from_homedir_cleanup'] - default to ['build/SOURCES'], so this should make nosrc rpm builds possible. Why do you want to keep BUILD directory?
(In reply to Michael Simacek from comment #29) > Please note the "excluded" argument - it doesn't remove the whole tree, but > keeps selected files/dirs present. It's configured in > config_opts['exclude_from_homedir_cleanup'] - default to ['build/SOURCES'], > so this should make nosrc rpm builds possible. Thank you Michael! Setting "config_opts['exclude_from_homedir_cleanup'] = ['build/SOURCES', 'build/BUILD']" in my mock config file does what I want. While looking for information on that, I also realized that site-defaults.cfg has documentation for all parameters, that should come in handy. > Why do you want to keep BUILD directory? I'm trying to create a single software collection (scl) from the latest fc24 SRPMs for LLVM and clang. Turns out if I want to run "make check" on the clang build, I need the original LLVM build tree because that contains a unit test framework that clang uses too. Preserving the BUILD directory seems like the easiest way to do this. Sorry to keep bugging you about this, but isn't the third call to "userdel" in comment #27 a bug? In the first two calls, you conditionally call "userdel -r" or "userdel" depending on "config['clean']", but then you call "userdel -r" anyway right afterwards.
(In reply to praetorian from comment #30) > (In reply to Michael Simacek from comment #29) > > Please note the "excluded" argument - it doesn't remove the whole tree, but > > keeps selected files/dirs present. It's configured in > > config_opts['exclude_from_homedir_cleanup'] - default to ['build/SOURCES'], > > so this should make nosrc rpm builds possible. > > Thank you Michael! Setting "config_opts['exclude_from_homedir_cleanup'] = > ['build/SOURCES', 'build/BUILD']" in my mock config file does what I want. > While looking for information on that, I also realized that > site-defaults.cfg has documentation for all parameters, that should come in > handy. > > > Why do you want to keep BUILD directory? > > I'm trying to create a single software collection (scl) from the latest fc24 > SRPMs for LLVM and clang. Turns out if I want to run "make check" on the > clang build, I need the original LLVM build tree because that contains a > unit test framework that clang uses too. Preserving the BUILD directory > seems like the easiest way to do this. The correct way to do that would be to package that test framework in a subpackage and buildrequire it in clang. > > Sorry to keep bugging you about this, but isn't the third call to "userdel" > in comment #27 a bug? Kind of. It always fails because the user has been deleted already, so it does nothing. I'll prepare a patch that removes it.