Description of problem: I'm pretty sure I've got the details correct, but atm. the only reproducer I have is running a 20 minute "pungi" command. Anyway, what appears to be happening is that we do this: cached_ro_ts = yb.rpmdb.ts action_ts = yb.ts # Install a bunch of things for h in cached_ro_ts.dbMatch('name', 'blah'): print "CTS", h['name'], h['epochnum'], h['version'], h['release'], h['arch'] for h in action_ts.dbMatch('name', 'blah'): print "ATS", h['name'], h['epochnum'], h['version'], h['release'], h['arch'] new_ts = initReadOnlyTS(root=self.root) for h in action_ts.dbMatch('name', 'blah'): print "NTS", h['name'], h['epochnum'], h['version'], h['release'], h['arch'] ...and we _only_ get answers from "ATS".
Note that this apparently worked with F15 GA of rpm/yum/pungi ... but fails now with rawhide. As I said, the test takes like 20 minutes, and you'll need a bunch of space on a VM etc. I'm testing a horrendous workaround atm. which is to do: diff --git a/yum/__init__.py b/yum/__init__.py index e9bc1f9..73155b8 100644 --- a/yum/__init__.py +++ b/yum/__init__.py @@ -1564,6 +1564,7 @@ class YumBase(depsolve.Depsolve): resultobject.return_code): self.rpmdb.dropCachedData() else: + self.rpmdb._ts = self.ts self.rpmdb.dropCachedDataPostTransaction(list(self.tsInfo)) self.plugins.run('posttrans') ...I'll be mildly surprised if this doesn't break something, but at least the rpm lookups will work. And I assume pungi/yum will quit soon enough after, anyway I'll see what happens in about 20 minutes :).
Ok, so the rpmdb._ts should be rpmdb.ts ... but apart from that it "works" ... except then pungi/yum dies because it can't find "nfs-utils-lib", but that appears to not actually have got installed ... it has a txmbr, but no callback and rpm/yum can't find it after pungi fails.
The thing is, you shouldn't be using more than one transaction set on a single db from a process. There's no good reason to do so, and rpm (sometime in the future) is likely to prevent that forcefully. (I've said this on yum-devel too but might've been missed by everybody :)
Ok, so after going through the post F15 changes and not seeing what could be causing this ... I decided to check, and the F15 GA versions of yum and rpm also fail.
Yeh, I know, and patches accepted ;).
Also note that I tried doing "del self.ts" ... and that didn't change anything, so the summary here is wrong (or, at least, incomplete).
Ok, so I downgraded to: yum from F14 rpm from F15 ...and now it "works", but only because yum isn't doing the checks anymore. After pungi has finished nfs-utils-lib is still not installed.
Ok, so testing with the rawhide yum/rpm again ... if I do this: diff --git a/yum/__init__.py b/yum/__init__.py index 12c2bdd..8c573c2 100644 --- a/yum/__init__.py +++ b/yum/__init__.py @@ -1564,6 +1564,7 @@ class YumBase(depsolve.Depsolve): resultobject.return_code): self.rpmdb.dropCachedData() else: + self.rpmdb.ts = self.ts self.rpmdb.dropCachedDataPostTransaction(list(self.tsInfo)) self.plugins.run('posttrans') diff --git a/yum/rpmsack.py b/yum/rpmsack.py index ef6fbd5..cb0d089 100644 --- a/yum/rpmsack.py +++ b/yum/rpmsack.py @@ -375,6 +375,9 @@ class RPMDBPackageSack(PackageSackBase): pkg = self.searchNevra(n, e, v, r, a) if not pkg: # Wibble? + if True: + print "ERROR: ****:", txmbr + continue self._deal_with_bad_rpmdbcache("dCDPT(pkg checksums): %s" % txmbr) continue ...then pungi "succeeds", even though nfs-utils-lib isn't installed. Dennis, I'm guessing you'll want to do that locally while we try to get to the bottom of the two-ish bugs: 1. For some reason nfs-utils-lib isn't being installed, even though it's in the transaction. This is almost certainly an rpm thing, and maybe a packaging thing ... but nobody has noticed before yum started complaining loudly. 2. Something changed so that we can't see the effects of the "action ts" from the other tses we have ... appears to happen even if we close the action ts. Yum is at least partly to blame here because 3.2.28-5 in F14 seems to be fine, which is only like 9 months of changes :). There have been changes where we added more tses in that time (like to fix the auto_close snafu), and removed them (Panu patches :). Any help, ideas etc. would be welcome. To "test" this you need a rawhide VM with a few GB of free space, install pungi and then get: http://www.and.org/pungi-test-data ...changing the URL to whatever mirror is best for you (for rawhide). Put the second part of the file in temp.ks, and run the first part. About 20 minutes later it'll fail.
Ok, so I've found the #2 problem ... and it's nothing to do with rpm or yum. What happens is that pylorax/yumhelper.py does: installroot = joinpaths(self.ybo.conf.installroot, "installroot") os.makedirs(installroot) self.ybo.conf.installroot = installroot ...but this is _after_ the rpmdb has been setup and told that .conf.installroot is the previous value. So the "action ts" is looking in /path/oldroot/installroot and all the other tses are looking in /path/oldroot ... shocker that they don't see the same data. If you _really_ need to do this, we'll probably have to add a .changeInstallRoot() API ... but that's going to be painful.
Ok, just confirmed that doing this change to pylorax/yumhelper fixes #2 ... # After: self.ybo.conf.installroot = installroot self.ybo.rpmdb.root = installroot self.ybo.rpmdb.ts.close() ...that's not a useful upstream fix, as self.ybo.rpmdb._persistdir and cachedir (and maybe other things) are still broken. But it at least confirms the problem.
Ok, I've split these up now ... assiging #2 to pungi.
My guess for the reason nfs-utils-lib is not getting installed is that it's obsoleted by libnfsidmap.
You're right, this is probably a yum bug where we aren't removing the nfs-utils-lib package from the install ... for some reason. Somewhat unusual, as packages which are obsoleted don't tend to stay around ... so this probably doesn't get tested much (saying that, I can't make it happen with plain yum ... yet).
Bah I didn't include the "reproducer. So, on F15, you can kind of test this with: % sudo yum in nfs-utils-lib Loaded plugins: PT, aliases, apps, auto-update-debuginfo, keys, noop, ps, : security, tmprepo, updater, verify, versionlock Setting up Install Process Package nfs-utils-lib-1.1.5-5.fc15.x86_64 is obsoleted by libnfsidmap-0.24-5.fc15.x86_64 which is already installed Nothing to do % sudo yum --setopt=obsoletes=false in nfs-utils-lib Loaded plugins: PT, aliases, apps, auto-update-debuginfo, keys, noop, ps, : security, tmprepo, updater, verify, versionlock Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package nfs-utils-lib.x86_64 0:1.1.5-5.fc15 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: nfs-utils-lib x86_64 1.1.5-5.fc15 fedora 61 k Transaction Summary ================================================================================ Install 1 Package Total download size: 61 k Installed size: 130 k Is this ok [y/N]: ...where I'm assuming rpm will tell us to FOAD if we run the transaction (certainly so if we are installing libnfsidmap at the same time -- which is also possible).
I've done some testing, and it seems like the patch in comment 8 does what's needed. If the package A obsoletes the package B, then doing sudo yum install A B --setopts=obsoletes=false fails gracefully with an error message. Also, if A is already installed, sudo yum install B --setopts=obsoletes=false will similarly fail gracefully. However, if A is not installed, then the above seems to install B successfully.
The patch in comment 8 was a hack to work around the installroot thing, I don't see how it can help the obsoletes problem. I've create a few of unit tests though now, so it should be easier to test (module mock'd test object problems :).
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle. Changing version to '19'. (As we did not run this process for some time, it could affect also pre-Fedora 19 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19
This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.