Red Hat Bugzilla – Bug 137812
up2date terminates entire transaction on RPMCALLBACK_UNPACK_ERROR leaving rpmdb in undefined state
Last modified: 2007-11-30 17:07:04 EST
Description of problem:
If rpm triggers the RPMCALLBACK_UNPACK_ERROR, up2date aborts the entire
transaction. In one case, a customer had /usr/src mounted from an NFS share
read-only. They then tried to go from RHEL3-U1 to RHEL3-U2 using up2date. The
transaction ordering was such that when the kernel-source upgrade failed the
redhat-release package, along with at least a dozen others, had already been
removed. Any further invocations of up2date say "Could not determine what
version of Red Hat Linux you are running."
Version-Release number of selected component (if applicable):
Always with proper setup
Steps to Reproduce:
1. Install RHEL3-U1 including kernel-source
2. Make /usr/src a read-only NFS mountpoint
3. run up2date -u
up2date stops midway through the installation of packages, complaining
that the installation of kernel-source failed.
It then stops the installation of packages and exits.
If you then try to run up2date again after fixing the problem, it complains
that it cannot figure out what version of Red Hat is running.
All packages except kernel-source updated properly. Alternatively, if up2date
stops mid-way, it should be able to restart and continue installing packages.
As a reference, rpm's CLI either skips to the next transaction element or does
nothing. The scenario described above is not reproducible with rpm CLI.
Also, I see nothing in the newer up2date sources to suggest this would not also
happen going from U2->U3.
How does one recover from this?
For whatever reason, I got the following error (on two different systems):
"There was a fatal RPM install error. The message was:
There was a rpm unpack error installing the package:
rpm --rebuilddb doesn't seem to help. Both systems are pretty hosed.
I should point out that I'm seeing this and I don't have /usr/src
Here is that error that caused the problem:
% sudo rpm -Uvh sendmail-8.12.11-4.RHEL3.1.i386.rpm
1:sendmail warning: /etc/aliases created as
warning: /etc/mail/Makefile created as /etc/mail/Makefile.rpmnew
warning: /etc/mail/access created as /etc/mail/access.rpmnew
warning: /etc/mail/domaintable created as /etc/mail/domaintable.rpmnew
warning: /etc/mail/helpfile created as /etc/mail/helpfile.rpmnew
warning: /etc/mail/local-host-names created as
warning: /etc/mail/mailertable created as /etc/mail/mailertable.rpmnew
warning: /etc/mail/sendmail.cf created as /etc/mail/sendmail.cf.rpmnew
warning: /etc/mail/sendmail.mc created as /etc/mail/sendmail.mc.rpmnew
warning: /etc/mail/submit.cf created as /etc/mail/submit.cf.rpmnew
warning: /etc/mail/submit.mc created as /etc/mail/submit.mc.rpmnew
warning: /etc/mail/trusted-users created as /etc/mail/trusted-users.rpmnew
warning: /etc/mail/virtusertable created as /etc/mail/virtusertable.rpmnew
warning: /etc/pam.d/smtp.sendmail created as
warning: /etc/rc.d/init.d/sendmail saved as
warning: /etc/sysconfig/sendmail created as /etc/sysconfig/sendmail.rpmnew
warning: /usr/lib/sasl/Sendmail.conf saved as
error: unpacking of archive failed on file
cpio: open failed - Not a directory
In RHEL3 U1, we were using /var/log/mail as a regular file to hold
syslog output for the mail facility. Sometime between then and U3 it
is required to be a directory. This causes the unpack to fail.
Looks like the RPMCALLBACK_UNPACK_ERROR
might be a new error for rpm update in rhel3u4.
this seems to be handled in the rhel4 code, which is being backported for
rhel3u6 (aka, up2date-4.4.x).
This bug is considered MustFix for RHEL 3 U6 by RHN Engineering.
Dev & PM ACKs for U6
need a test plan
get a rhel3 u1 box
make sure it has the kernel-source package installed
make /usr/src readonly
( to closer reproduce the original report, it should be NFS mounted, easiest
ugh, what happened to the rest of my test plan... bugzilla is being weird...
make /usr/src/ readonly
(to closer reproduce the original report, it should be NFS mounted,
an easier approach might be to do the install so that there is a
seperate /usr/src partion, and mount it read-only. Another way
might be a normal install and then something like `chmod -R a-w /usr/src`)
It should still error out, but more gracefully now.
Note: it is probably quicker/easier to just run `up2date kernel-source`,
but the above is closer to the original report.
Alright, I tried this. You can't run up2date as non-root, but if you're root,
then /usr/src will not be read-only. I think I'm missing some crucial element
of knowledge here.
Here's what I did:
1- kickstarted test07.rhndev to rhel 3 u1
2- installed kernel-source
3- ran 'chmod -R a-w /usr/src' on test07
4- created a non-root user on test07
5- logged in as the non-root user and verified I cannot write to /usr/src
6- tried to run up2date, but had to run it as root
Hmm, indeed about /usr/src being writeable by root. I didn't think about
Sounds like you need to do the NFS or seperate partion thing then, since those
remain read-only even for root (at least, if you change
the mount options for /usr/src to be ro in fstab).
retested on rhel 3 u1 AS i386 box registered to proxy -> webqa
kernel* removed off the packageSkipList
made /usr/src read-only using the following command:
chattr -R +i /usr/src
ran the rpm -Uvh sendmail command and sendmail was successfully updated
ran up2date -u and it, too, worked (updated all packages)... except for the test
install of lilo, which failed:
"Error installing lilo.conf The message was:
test install of lilo failed"
Finally, /var/log/mail is a directory in rhel 3 u1
This should not have been cloned for RHEL 4 U2. It was RHEL 3 specific.
This is re: comment 20
I should have noticed this earlier. AFAICT nothing has been done to fix this,
nor was the non-existent fix tested properly.
In order for /usr/src being read-only to be relevant, you must be updating a
package that contains files in /usr/src via up2date. sendmail does not meet this
criteria while kernel-source does.
To be sure you are actually recreating the problem scenario you should see a
message indicating that the install of kernel-source failed. Once you've seen
this the important part is that up2date doesn't leave the system in a mess (ie:
packages whose new versions are not installed at the time of the failure should
either still have the old version installed or else go ahead and install the new
one in spite of the kernel-source failure).
My testing went as follows:
I started with RHEL3-U5, subscribed to U6-Beta channel, upgraded up2date. Then,
I made /usr/src read-only (nfs mount of localhost:/opt) and ran 'up2date -u'.
The read-only /usr/src did cause a fatal rpm unpack error and it did leave the
system in an inconsistent state. Although redhat-release was updated (as a
result of it being early in the transaction order) all the packages that had not
yet been installed at the time of the failure are gone according to rpm:
[root@localhost root]# rpm -q rpm
package rpm is not installed
This bug is still present in U6.
I believe the above also implies that this bug is present in RHEL4.
PROD_READY is a deprecated state. This bug is now RELEASE_PENDING.
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.