Bug 502221 - mkinitrd fails with: "cp: cannot stat `/usr/lib64//libparted-1.8.so.8.0.0': No such file or directory"
mkinitrd fails with: "cp: cannot stat `/usr/lib64//libparted-1.8.so.8.0.0': N...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: mkinitrd (Show other bugs)
11
All Linux
low Severity medium
: ---
: ---
Assigned To: Peter Jones
Fedora Extras Quality Assurance
:
: 482964 488183 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-05-22 11:57 EDT by Carl Roth
Modified: 2009-07-11 13:20 EDT (History)
7 users (show)

See Also:
Fixed In Version: 6.0.87-1.fc11
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-07-11 13:08:49 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Output from mkinitrd (304.48 KB, text/plain)
2009-05-22 14:36 EDT, Carl Roth
no flags Details
Fixed mkinitrd-functions file (10.03 KB, text/plain)
2009-05-25 05:38 EDT, Hans de Goede
no flags Details

  None (edit)
Description Carl Roth 2009-05-22 11:57:24 EDT
Description of problem:

See BUG488183.  I'm re-opening it because I claim it's a mkinitrd bug.

The mkinitrd step fails for kernel-PAE-2.6.29.3-140.fc11.i686 becuase it cannot traverse the symlinks set up by parted-devel.

Parted sets up in '/lib':

lrwxrwxrwx. 1 root root     22 2009-05-09 09:38 /lib/libparted-1.8.so.8 -> libparted-1.8.so.8.0.0*
-rwxr-xr-x. 1 root root 435884 2009-05-08 08:29 /lib/libparted-1.8.so.8.0.0*

whereas parted-devel sets up in '/usr/lib':

lrwxrwxrwx. 1 root root 32 2009-05-09 09:39 libparted.so -> ../../lib/libparted-1.8.so.8.0.0*
lrwxrwxrwx. 1 root root 12 2009-04-14 08:45 libparted-1.8.so.8 -> libparted.so*

Note that the libparted.so symlink is owned by parted-devel, but the libparted-1.8.so.8 symlink is un-owned (generated by ldconfig).

mkinitrd stumbles across the non-local symlinks in /usr/lib and fails.  

Removing parted-devel and re-running mkinitrd allows mkintrd to succeed.

Version-Release number of selected component (if applicable):

parted-1.8.8-17.fc11.i586
mkinitrd-6.0.84-1.fc11.i586
kernel-PAE-2.6.29.3-140.fc11.i686

How reproducible:

Always

Steps to Reproduce:
1. Install mkinitrd, parted, parted-devel
2. Update to kernel-PAE-2.6.29.3-140.fc11.i686
3. See that %post/new-kernel-pkg/mkinitrd fails
4. remove parted-devel
5. re-run mkinitrd.

Actual results:


Expected results:


Additional info:

Offending mkinitrd command in this case is

/sbin/mkinitrd --allow-missing -f /boot/initrd-2.6.29.3-140.fc11.i686.PAE.img 2.6.29.3-140.fc11.i686.PAE
Comment 1 Hans de Goede 2009-05-22 13:42:58 EDT
Hmm,

I'm afraid even with your instructions I cannot reproduce this. I've checked that
libparted has the same symlink structure as you report, but mkinitrd still functions fine.

Can you run mkinitrd like this (as root):
bash -x /sbin/mkinitrd test.img $(uname -r) &> log

And then attach the resulting log file ?

Thanks.

Note to self: related bug 482964
Comment 2 Carl Roth 2009-05-22 14:35:30 EDT
Attaching output.

Note from the output that 'cp' still fails.  The resulting initrd is missing the libparted.so file, so the boot fails (I have tried this too).
Comment 3 Carl Roth 2009-05-22 14:36:32 EDT
Created attachment 345138 [details]
Output from mkinitrd
Comment 4 Hans de Goede 2009-05-25 04:24:24 EDT
*** Bug 488183 has been marked as a duplicate of this bug. ***
Comment 5 Hans de Goede 2009-05-25 04:24:28 EDT
*** Bug 482964 has been marked as a duplicate of this bug. ***
Comment 6 Hans de Goede 2009-05-25 04:28:18 EDT
Ok, I can reproduce this now, I think you somehow have ended up with /usr/lib being listed in /etc/ld.so.conf or a file under /etc/ld.so.conf.d. Can you please
check if this is the case and in which file it is listed ?

Then do:
rpm -qf /etc/<bad-ld.so.conf-file>

To get the name of the package which owns that file and then do:
rpm --verify <packagename>

To see if this is a local modification, or some packaging bug in
some package (which I don't have installed). Once you've done that, as a work
around to the problem you've reported, you can remove the offending line and re-run mkinitrd, 

In the mean time I'll try to fix mkinitrd to handle things properly
when /usr/lib is listed in ld.so.conf
Comment 7 Hans de Goede 2009-05-25 04:29:05 EDT
Erm make that /usr/lib64 ended up in a ld.so.conf file, sorry.
Comment 8 Hans de Goede 2009-05-25 04:33:00 EDT
(In reply to comment #7)
> Erm make that /usr/lib64 ended up in a ld.so.conf file, sorry.  

Hmm, the summary for the attached bug has lib64 in it, but the attached log uses
lib, anyways looks for /usr/lib (for 32 bit installs) and/or /usr/lib64 (for 64 bit installs) in the ld.so.conf files.
Comment 9 Hans de Goede 2009-05-25 05:38:13 EDT
Created attachment 345296 [details]
Fixed mkinitrd-functions file

This functions file fixes this issue. Copy this file to
/usr/libexec/initrd-functions, and then re-run mkinitrd, after this
this problem should be gone. Please let me know if this fixes things
for you. This will work for both F-10 and F-11.

p.s.
I would still like to know how /usr/lib[64] got into ld.so.conf, as if
that is done by a package that is another (minor) bug which ought to be
fixed.
Comment 10 Carl Roth 2009-05-25 13:03:01 EDT
(In reply to comment #8)
> (In reply to comment #7)
> > Erm make that /usr/lib64 ended up in a ld.so.conf file, sorry.  
> 
> Hmm, the summary for the attached bug has lib64 in it, but the attached log
> uses
> lib, anyways looks for /usr/lib (for 32 bit installs) and/or /usr/lib64 (for 64
> bit installs) in the ld.so.conf files.  

Sorry for that confusion, the subject like should rather read "/usr/lib".  I was transcribing the subject line from BUG48813.  The current behavior that I am seeing is on an i386 system.
Comment 11 Carl Roth 2009-05-25 13:31:18 EDT
(In reply to comment #6)
> Ok, I can reproduce this now, I think you somehow have ended up with /usr/lib
> being listed in /etc/ld.so.conf or a file under /etc/ld.so.conf.d. Can you
> please
> check if this is the case and in which file it is listed ?
> 
> Then do:
> rpm -qf /etc/<bad-ld.so.conf-file>
> 
> To get the name of the package which owns that file and then do:
> rpm --verify <packagename>
> 
> To see if this is a local modification, or some packaging bug in
> some package (which I don't have installed). Once you've done that, as a work
> around to the problem you've reported, you can remove the offending line and
> re-run mkinitrd, 
> 
> In the mean time I'll try to fix mkinitrd to handle things properly
> when /usr/lib is listed in ld.so.conf  

This doesn't seem to be the case for me:

# egrep ^/ /etc/ld.so.conf /etc/ld.so.conf.d/*
/etc/ld.so.conf.d/freetype-freeworld-i386.conf:/usr/lib/freetype-freeworld
/etc/ld.so.conf.d/mysql-i386.conf:/usr/lib/mysql
/etc/ld.so.conf.d/qt-i386.conf:/usr/lib/qt-3.3/lib
/etc/ld.so.conf.d/xulrunner-32.conf:/usr/lib/xulrunner-1.9.1

# rpm -qf /etc/ld.so.conf.d/* | sort | uniq
freetype-freeworld-2.3.9-2.fc11.i586
kernel-PAE-2.6.29.1-111.fc11.i686
kernel-PAE-2.6.29.2-126.fc11.i686
kernel-PAE-2.6.29.3-140.fc11.i686
mysql-libs-5.1.32-1.fc11.i586
qt3-3.3.8b-25.fc11.i586
xulrunner-1.9.1-0.20.beta4.fc11.i586

I also dumped all of the settings from LD_LIBRARY_PATH but the mkinitrd script still failed.
Comment 12 Carl Roth 2009-05-25 13:45:30 EDT
(In reply to comment #9)
> Created an attachment (id=345296) [details]
> Fixed mkinitrd-functions file
> 
> This functions file fixes this issue. Copy this file to
> /usr/libexec/initrd-functions, and then re-run mkinitrd, after this
> this problem should be gone. Please let me know if this fixes things
> for you. This will work for both F-10 and F-11.
> 
> p.s.
> I would still like to know how /usr/lib[64] got into ld.so.conf, as if
> that is done by a package that is another (minor) bug which ought to be
> fixed.  

I tried this out and it does the trick:

# /sbin/mkinitrd --allow-missing -f /tmp/foo.img 2.6.29.3-140.fc11.i686.PAE

# gzip -dc /tmp/foo.img | cpio -t | grep parted
gzip -dc /tmp/foo.img | cpio -t | grep parted
lib/libparted-1.8.so.8.0.0
lib/libparted-1.8.so.8
usr/lib/libparted-1.8.so.8
16200 blocks
Comment 13 Hans de Goede 2009-05-25 13:46:16 EDT
> 
> This doesn't seem to be the case for me:
> 
> # egrep ^/ /etc/ld.so.conf /etc/ld.so.conf.d/*
> /etc/ld.so.conf.d/freetype-freeworld-i386.conf:/usr/lib/freetype-freeworld
> /etc/ld.so.conf.d/mysql-i386.conf:/usr/lib/mysql
> /etc/ld.so.conf.d/qt-i386.conf:/usr/lib/qt-3.3/lib
> /etc/ld.so.conf.d/xulrunner-32.conf:/usr/lib/xulrunner-1.9.1
> 
> # rpm -qf /etc/ld.so.conf.d/* | sort | uniq
> freetype-freeworld-2.3.9-2.fc11.i586
> kernel-PAE-2.6.29.1-111.fc11.i686
> kernel-PAE-2.6.29.2-126.fc11.i686
> kernel-PAE-2.6.29.3-140.fc11.i686
> mysql-libs-5.1.32-1.fc11.i586
> qt3-3.3.8b-25.fc11.i586
> xulrunner-1.9.1-0.20.beta4.fc11.i586
> 
> I also dumped all of the settings from LD_LIBRARY_PATH but the mkinitrd script
> still failed.  

Hmm, what is the output of ldd /sbin/nash ?

If that points to /usr/lib for libparted-1.8.so.8, then somehow ld.so ends up
searching /usr/lib before it searches /usr, which is not normal (see man ld.so),
anyways I suspect that it will point to /usr/lib and in that case the attached
updated mkinitrd-functions file should fix things.

I still wonder why ld.so ends up finding libparted in /usr/lib before it checks
/lib though, maybe you did have a buggy ld.so.conf.d file and it got updated?
What you can do (after the ldd /sbin/nash check please) is try running ldconfig
as root, and then look at the output of /sbin/nash again.
Comment 14 Carl Roth 2009-05-25 14:16:07 EDT
(In reply to comment #13)

> 
> Hmm, what is the output of ldd /sbin/nash ?
> 
> If that points to /usr/lib for libparted-1.8.so.8, then somehow ld.so ends up
> searching /usr/lib before it searches /usr, which is not normal (see man
> ld.so),
> anyways I suspect that it will point to /usr/lib and in that case the attached
> updated mkinitrd-functions file should fix things.
> 
> I still wonder why ld.so ends up finding libparted in /usr/lib before it checks
> /lib though, maybe you did have a buggy ld.so.conf.d file and it got updated?
> What you can do (after the ldd /sbin/nash check please) is try running ldconfig
> as root, and then look at the output of /sbin/nash again.  

I think the culprit may be a mis-guided LD_LIBRARY_PATH:

$ echo $LD_LIBRARY_PATH
/usr/lib:/lib:...

$ ldd /sbin/nash | grep parted
	libparted-1.8.so.8 => /usr/lib/libparted-1.8.so.8 (0x00c1b000)

$ (unset LD_LIBRARY_PATH; ldd /sbin/nash | grep parted)
	libparted-1.8.so.8 => /lib/libparted-1.8.so.8 (0x00c1b000)
Comment 15 Hans de Goede 2009-05-25 14:31:21 EDT
(In reply to comment #14)
> I think the culprit may be a mis-guided LD_LIBRARY_PATH:
> 
> $ echo $LD_LIBRARY_PATH
> /usr/lib:/lib:...
> 
> $ ldd /sbin/nash | grep parted
>  libparted-1.8.so.8 => /usr/lib/libparted-1.8.so.8 (0x00c1b000)
> 
> $ (unset LD_LIBRARY_PATH; ldd /sbin/nash | grep parted)
>  libparted-1.8.so.8 => /lib/libparted-1.8.so.8 (0x00c1b000)  

Yes that explains, so is this a local modification you made, or is some
package dropping files in /etc/profile.d which set that funky
library path ?
Comment 16 Thomas Quinn 2009-05-25 14:43:02 EDT
(In reply to comment #13)
> > 
> > This doesn't seem to be the case for me:
> > 
> > # egrep ^/ /etc/ld.so.conf /etc/ld.so.conf.d/*
> > /etc/ld.so.conf.d/freetype-freeworld-i386.conf:/usr/lib/freetype-freeworld
> > /etc/ld.so.conf.d/mysql-i386.conf:/usr/lib/mysql
> > /etc/ld.so.conf.d/qt-i386.conf:/usr/lib/qt-3.3/lib
> > /etc/ld.so.conf.d/xulrunner-32.conf:/usr/lib/xulrunner-1.9.1
> > 
> > # rpm -qf /etc/ld.so.conf.d/* | sort | uniq
> > freetype-freeworld-2.3.9-2.fc11.i586
> > kernel-PAE-2.6.29.1-111.fc11.i686
> > kernel-PAE-2.6.29.2-126.fc11.i686
> > kernel-PAE-2.6.29.3-140.fc11.i686
> > mysql-libs-5.1.32-1.fc11.i586
> > qt3-3.3.8b-25.fc11.i586
> > xulrunner-1.9.1-0.20.beta4.fc11.i586
> > 
> > I also dumped all of the settings from LD_LIBRARY_PATH but the mkinitrd script
> > still failed.  
> 
> Hmm, what is the output of ldd /sbin/nash ?
> 
> If that points to /usr/lib for libparted-1.8.so.8, then somehow ld.so ends up
> searching /usr/lib before it searches /usr, which is not normal (see man
> ld.so),
> anyways I suspect that it will point to /usr/lib and in that case the attached
> updated mkinitrd-functions file should fix things.
> 
> I still wonder why ld.so ends up finding libparted in /usr/lib before it checks
> /lib though, maybe you did have a buggy ld.so.conf.d file and it got updated?
> What you can do (after the ldd /sbin/nash check please) is try running ldconfig
> as root, and then look at the output of /sbin/nash again.  

In my case, my ld.so.conf had /usr/lib and not /lib.  This seems to be left over from an old version of fedora (predating 2004).  Moving the ld.so.conf.rpmnew (dated 5/11/2004!) over changed the reference to the /lib version.
Comment 17 Carl Roth 2009-05-25 16:56:06 EDT
(In reply to comment #15)
> (In reply to comment #14)
> > I think the culprit may be a mis-guided LD_LIBRARY_PATH:
> > 
> > $ echo $LD_LIBRARY_PATH
> > /usr/lib:/lib:...
> > 
> > $ ldd /sbin/nash | grep parted
> >  libparted-1.8.so.8 => /usr/lib/libparted-1.8.so.8 (0x00c1b000)
> > 
> > $ (unset LD_LIBRARY_PATH; ldd /sbin/nash | grep parted)
> >  libparted-1.8.so.8 => /lib/libparted-1.8.so.8 (0x00c1b000)  
> 
> Yes that explains, so is this a local modification you made, or is some
> package dropping files in /etc/profile.d which set that funky
> library path ?  

Yes this was a local configuration in .profile.
Comment 18 Onno Hommes 2009-05-26 10:13:57 EDT
Why is the priority so low and the severity only medium. On Fedora fc10 the mkinitrd during a kernel upgrade does not prevent the new kernel update nor does it stop and so after the upgrade you have a broken initrd.img and you get a kernel panic during your next boot. So I would say the severity is at least higher then medium and the priority should also be higher.
Comment 19 Hans de Goede 2009-05-27 13:34:17 EDT
Ok, I've prepared an update for F-10, which will go to updates-testing first. Once it has reached updates-testing the updates system will post instructions here on how to test it. Feedback appreciated.

There won't be an update for F-11 yet, as F-11 is frozen for the upcoming release
and this bug is not critical enough to break the freeze. As soon as F-11 gold gets
out the door I'll do an update for F-11 for this.
Comment 20 Fedora Update System 2009-05-27 13:36:57 EDT
mkinitrd-6.0.71-6.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/mkinitrd-6.0.71-6.fc10
Comment 21 Fedora Update System 2009-05-28 03:57:41 EDT
mkinitrd-6.0.71-6.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mkinitrd'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-5542
Comment 22 Bug Zapper 2009-06-09 12:21:47 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 23 Fedora Update System 2009-06-16 08:33:11 EDT
mkinitrd-6.0.87-1.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/mkinitrd-6.0.87-1.fc11
Comment 24 Fedora Update System 2009-06-18 07:55:28 EDT
mkinitrd-6.0.87-1.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mkinitrd'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F11/FEDORA-2009-6652
Comment 25 Fedora Update System 2009-07-11 13:08:40 EDT
mkinitrd-6.0.71-6.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 26 Fedora Update System 2009-07-11 13:20:37 EDT
mkinitrd-6.0.87-1.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.