Bug 511158

Summary: Yum no longer works inside a mock chroot
Product: [Fedora] Fedora Reporter: Jonathan Underwood <jonathan.underwood>
Component: mockAssignee: David Cantrell <dcantrell>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 11CC: cwhuang, dcantrell, kdudka, mebrown, williams
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-07-13 23:37:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jonathan Underwood 2009-07-13 23:08:24 UTC
Description of problem:
On an F-11 box:

$ mock -r fedora-11-x86_64 init
[snip]
$ mock -r fedora-11-x86_64 install yum
[snip]
$ mock -r fedora-11-x86_64 shell
INFO: mock.py version 0.9.16 starting...
State Changed: init plugins
State Changed: start
State Changed: lock buildroot
mock-chroot> yum install wget

error: cannot open Packages index using db3 - No such file or directory (2)
error: cannot open Packages database in /var/lib/rpm
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 309, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 157, in main
    base.getOptionsConfig(args)
  File "/usr/share/yum-cli/cli.py", line 189, in getOptionsConfig
    self.conf
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 652, in <lambda>
    conf = property(fget=lambda self: self._getConfig(),
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 239, in _getConfig
    self._conf = config.readMainConfig(startupconf)
  File "/usr/lib/python2.6/site-packages/yum/config.py", line 794, in readMainConfig
    yumvars['releasever'] = _getsysver(startupconf.installroot, startupconf.distroverpkg)
  File "/usr/lib/python2.6/site-packages/yum/config.py", line 867, in _getsysver
    idx = ts.dbMatch('provides', distroverpkg)
TypeError: rpmdb open failed


Version-Release number of selected component (if applicable):
mock-0.9.16-1.fc11
yum-3.2.23-3.fc11

How reproducible:
Everytime

Comment 1 Jonathan Underwood 2009-07-13 23:13:43 UTC
I should add that the host is running F-11 x86_64.

Comment 2 Jonathan Underwood 2009-07-13 23:31:58 UTC
This issue is fixed by doing this inside the chroot:
rm /var/lib/rpm/__db* 

However, surely, since the rpm inside the chroot, and the rpm used to create the chroot are (in this case) the same (F-11 x86_64) surely this shouldn't be necessary?

And, wouldn't it be a good idea for mock to rm /var/lib/rpm/__db* at the end of init?

Comment 3 Jesse Keating 2009-07-13 23:37:36 UTC
Mock isn't really meant to do anything but build chroots for building packages.  Period.  Anything else is really beyond the use case and gets into feature creep.  Yum isn't meant to be useable inside the chroot.  Automatically rming the files isn't necessary for the operation of building packages.  If you're using your chroots for something else, you'll need to do the work necessary to make them useful for that.

Comment 4 Jonathan Underwood 2009-07-13 23:41:54 UTC
Well, that's a shame, because it's a great way to develop packages against other versions of the distro without having to install them. If it's really not meant for this kind of work, I wonder why it has a --shell command line option. Point taken though.

Comment 5 Jesse Keating 2009-07-14 01:09:06 UTC
Shell is there to investigate a failed build to see what happened.

I realize that mock is useful for making chroots for other things.  I use it myself to do pungi composes.  What mock likely needs is a plugin system so that those who want to use it for non-building can write plugins to assist with it, sort of how yum has plugins to do things that are beyond what the main devs wish to support and maintain.

Comment 6 Clark Williams 2009-07-14 14:00:14 UTC
There is a plugin system in place. Unfortunately, it's not well documented and currently only used for mock internal plugins.

The current plugins are:

   bind_mount
   ccache
   root_cache
   tmpfs
   yum_cache

I believe that you can insert plugins with some judicious editing of the config files. That may be a way for Jonathan to do some prebuild cleanups, etc.

Comment 7 Jonathan Underwood 2009-07-14 14:03:15 UTC
Thanks for that info, I'll have a dig into that when I have a few spare moments.

Comment 8 Huang, Chih-Wei 2009-08-23 12:54:25 UTC
Hi Jesse,
I tested mock+pungi on F11 these two days, and it didn't work,
as described in this bug. I struggled in these days, and finally
found it already been reported here.
But you said it is not a bug?
I totally cannot agree.

There are many reasons to consider it as a bug:
* This is a documented way to use mock and pungi:
https://fedorahosted.org/pungi/wiki/PungiDocs/RunningPungiInMock

* This worked before, at least on Fedora 10, 9, 8.
I'm sure since I used frequently before.

* You said mock should just been used to build rpms,
and --shell is just for debug purpose. But how about --install?
This is a valid option of mock. People will use it and crash the rpm db.
Thus rpm, yum in chroot envrionemt is unusable anymore.
Any normal user will consider it a bug.


Consequently, either fix this big, or fix the doc, at least.

Comment 9 Huang, Chih-Wei 2009-08-23 15:54:45 UTC
Hi Jesse,
Now I found more serious problems. Even
--rebuild didn't work since rpm db crashed.

mock --init
mock -v --no-clean --rebuild asterisk-codecs-1.0-beta9.src.rpm

The log is long, I noticed it tried to do yum update
DEBUG: /usr/bin/yum --installroot /var/mock/ippbx/root/  update
DEBUG: Executing command: /usr/bin/yum --installroot /var/mock/ippbx/root/  update
DEBUG: ================================================================================
DEBUG:  Package             Arch      Version                    Repository       Size
DEBUG: ================================================================================
DEBUG: Installing:
DEBUG:  xz-lzma-compat      i586      4.999.8-0.7.beta.fc11      updates-11       16 k
DEBUG:      replacing  lzma.i586 4.32.7-2.fc11
DEBUG: Transaction Summary
DEBUG: ================================================================================
DEBUG: Install      1 Package(s)         
DEBUG: Update       0 Package(s)         
DEBUG: Remove       0 Package(s)         
DEBUG: Total download size: 16 k
DEBUG: Installed:
DEBUG:   xz-lzma-compat.i586 0:4.999.8-0.7.beta.fc11                                   
DEBUG: Replaced:
DEBUG:   lzma.i586 0:4.32.7-2.fc11                 

<skipped>

State Changed: setup
DEBUG: Executing command: ['rpm', '-Uvh', '--nodeps', '/builddir/build/originals/asterisk-codecs-1.0-beta9.src.rpm']
DEBUG: asterisk-codecs             ##################################################
DEBUG: Child returncode was: 0
INFO: ENTER do(['bash', '--login', '-c', 'rpmbuild -bs --target i586 --nodeps builddir/build/SPECS/asterisk-codecs.spec'], False, '/var/mock/ippbx/root/', None, 0, True, 0, 0, 490, None, logger=<mock.trace_decorator.getLog object at 0x8db244c>)
DEBUG: Executing command: ['bash', '--login', '-c', 'rpmbuild -bs --target i586 --nodeps builddir/build/SPECS/asterisk-codecs.spec']
DEBUG: warning: Could not canonicalize hostname: cwhuang.asus.com.tw
DEBUG: Building target platforms: i586
DEBUG: Building for target i586
DEBUG: Wrote: /builddir/build/SRPMS/asterisk-codecs-1.0-beta9.src.rpm
DEBUG: Child returncode was: 0
INFO: LEAVE do --> 

DEBUG: /usr/bin/yum --installroot /var/mock/ippbx/root/  resolvedep  ccache 
DEBUG: Executing command: /usr/bin/yum --installroot /var/mock/ippbx/root/  resolvedep  ccache 
DEBUG: Traceback (most recent call last):
DEBUG:   File "/usr/bin/yum", line 29, in <module>
DEBUG:     yummain.user_main(sys.argv[1:], exit_code=True)
DEBUG:   File "/usr/share/yum-cli/yummain.py", line 309, in user_main
DEBUG:     errcode = main(args)
DEBUG:   File "/usr/share/yum-cli/yummain.py", line 157, in main
DEBUG:     base.getOptionsConfig(args)
DEBUG:   File "/usr/share/yum-cli/cli.py", line 189, in getOptionsConfig
DEBUG:     self.conf
DEBUG:   File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 652, in <lambda>
DEBUG:     conf = property(fget=lambda self: self._getConfig(),
DEBUG:   File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 239, in _getConfig
DEBUG:     self._conf = config.readMainConfig(startupconf)
DEBUG:   File "/usr/lib/python2.6/site-packages/yum/config.py", line 794, in readMainConfig
DEBUG:     yumvars['releasever'] = _getsysver(startupconf.installroot, startupconf.distroverpkg)
DEBUG:   File "/usr/lib/python2.6/site-packages/yum/config.py", line 873, in _getsysver
DEBUG:     hdr = idx.next()
DEBUG: StopIteration

See? Even --rebuild doesn't work.

I believe the key point is the command
/usr/bin/yum --installroot ...
will crash rpm db in the chroot environment,
so any further rpm or yum command cannot execute.
(manually rm /var/lib/rpm/__db* in chroot fix the issue,
but how can I do it inside --rebuild command?)

Can you admit it is a bug now?

Comment 10 Huang, Chih-Wei 2009-08-23 16:33:32 UTC
Ok, I found another bug 507309
that described the same issue.
Actually, this bug is directly resulted from that one.

Comment 11 Jesse Keating 2009-09-09 20:43:18 UTC
Once you've gone into a chroot and used rpm from within the chroot, the rpm database cache will be in a format used by the version of rpm inside the chroot.  This may conflict with the version of rpm outside the chroot.  Therefor if you do anything inside, you should clear any cache created by any use outside.  And if you're going to do anything outside after you've done something inside, you'll need to clear the cache as well, before you exit the chroot.

Comment 12 Huang, Chih-Wei 2009-09-10 01:40:44 UTC
Hi Jesse,
Seems you didn't get my point.
I didn't do inside/outside chroot by hand
as you described, What I did are just

mock --init
mock --rebuild xxx.src.rpm

and it failed.

The reason is mock --rebuild will do 
yum --installroot /var/mock/ippbx/root update
(this is did outside, right?)
so later rebuild inside chroot will fail
since the cache is not cleared.

Unless I modify mock script itself,
I didn't see any way to clear the cache
during the internal of mock --rebuild command.

Comment 13 Jesse Keating 2009-09-10 19:43:02 UTC
That sounds like a different bug that should be filed on its own.