Does libuuid exist on your installation media?  Can you attach the complete traceback?

(In reply to comment #1)
> Does libuuid exist on your installation media? 

Not sure if I answer your question completely - there's no libuuid package, but it looks like the installation image contains just files. find(1) tells me this:

./lib64/libuuid.so.1.3.0
./lib64/libuuid.so.1
./proc/sys/kernel/random/uuid
./bin/dbus-uuidgen
./sys/devices/virtual/dmi/id/product_uuid
./usr/lib64/python2.7/uuid.py
./usr/lib64/python2.7/uuid.pyc


> Can you attach the complete
> traceback?

What I pasted /is/ the complete traceback, just reproduced with python -c "import cryptsetup" because I couldn't find it in /tmp/anaconda.log

I'm attaching a screenshot of the traceback so we're on the same page.

Created attachment 484229 [details]
installation screenshot

(In reply to comment #3)
> Created attachment 484229 [details]
> installation screenshot

To help me reproduce your failure, can you provide some information on how you are hitting this issue?

1. What version of anaconda are you using to install: F-15-Alpha, nightly, custom?
2. How are you booting the installer: DVD, netinst.iso, PXEboot?
3. What boot parameters are you adding: any ks= or similar?
4. If using kickstart, can you provide the ks.cfg?
5. If doing manual, can you detail the manual selections you make that contribute to this failure?

Additionally, I'm wondering if there's media read errors going on here, since the code path isn't protected at all.  That is, I think just about everyone would be hitting this problem since the import's at the very top of kickstart.py and should always be happening.  Can you grab /tmp/syslog?

(In reply to comment #4)
> (In reply to comment #3)
> > Created attachment 484229 [details]
> > installation screenshot
> 
> To help me reproduce your failure, can you provide some information on how you
> are hitting this issue?
> 
> 1. What version of anaconda are you using to install: F-15-Alpha, nightly,
> custom?

Whatever is in http://download.fedora.redhat.com/pub/fedora/linux/development/15/x86_64/os/ (or rather our internal mirror of it)

> 2. How are you booting the installer: DVD, netinst.iso, PXEboot?

koan --virt, the tree is available via NFS. I pointed cobbler at vmlinuz and initrd from the mirror, not sure what magic does koan do after that.

> 3. What boot parameters are you adding: any ks= or similar?

yep, a custom kickstart

The full kernel command line looks like:
method=$url ksdevice=link lang= text ks=http://$cobbler_server/cblr/svc/op/ks/pr
ofile/Fedora-15-x86_64-minimal selinux=0 dhcpclass=ipa-lab-vms kssendmac 

> 4. If using kickstart, can you provide the ks.cfg?

OK

(In reply to comment #6)
> > 2. How are you booting the installer: DVD, netinst.iso, PXEboot?
> 
> koan --virt, the tree is available via NFS. I pointed cobbler at vmlinuz and
> initrd from the mirror, not sure what magic does koan do after that.

Sorry, not NFS, but rather HTTP.

Created attachment 484248 [details]
sanitized kickstart

(In reply to comment #5)
> Additionally, I'm wondering if there's media read errors going on here, since
> the code path isn't protected at all.  That is, I think just about everyone
> would be hitting this problem since the import's at the very top of
> kickstart.py and should always be happening.  Can you grab /tmp/syslog?

Since this is a network install, I doubt it, but sure, I'll attach.

Can it be that the mirror is in some kind of odd state?

Created attachment 484249 [details]
/tmp/syslog

*** Bug 684847 has been marked as a duplicate of this bug. ***

Created attachment 484257 [details]
anaconda-logs.tgz

Reproduced with F-15-Alpha (anaconda-15.20.1-1) using the provided ks.cfg.  Attaching log files.

-rw-r--r-- root/root      5872 2011-03-14 13:24 tmp/anaconda.log
-rw-r--r-- root/root         0 2011-03-14 13:24 tmp/ifcfg.log
-rw-r--r-- root/root         0 2011-03-14 13:24 tmp/program.log
-rw-r--r-- root/root      2609 2011-03-14 13:24 tmp/storage.log
-rw-r--r-- root/root     50101 2011-03-14 13:24 tmp/syslog

I was *unable* to reproduce the bug using anaconda-15.22-1 along with the provided ks.cfg.

(In reply to comment #12)
> I was *unable* to reproduce the bug using anaconda-15.22-1 along with the
> provided ks.cfg.

How do I test a new anaconda version? I see that 15.22 was built in koji, do I need to regenerate any of the images?

Thank you for testing, James!

(In reply to comment #13)
> (In reply to comment #12)
> > I was *unable* to reproduce the bug using anaconda-15.22-1 along with the
> > provided ks.cfg.
> 
> How do I test a new anaconda version? I see that 15.22 was built in koji, do I
> need to regenerate any of the images?

F-15 installation images are created daily at http://download.fedoraproject.org/pub/fedora/linux/development/15/x86_64/os

However, those images use only content that is in the base F-15 (stable) repo.  anaconda-15.22.1 is not yet in the stable repository.  Until then, I have created a custom boot.iso (and pxeboot images) that you can point virt-install (or koan) to.

http://jlaska.fedorapeople.org/updates/anaconda-15.22-1/x86_64/os/

If you are able to boot anaconda-15.22-1, can you please add positive karma feedback to https://admin.fedoraproject.org/updates/anaconda-15.22-1.fc15

(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #12)
> > > I was *unable* to reproduce the bug using anaconda-15.22-1 along with the
> > > provided ks.cfg.
> > 
> > How do I test a new anaconda version? I see that 15.22 was built in koji, do I
> > need to regenerate any of the images?
> 
> F-15 installation images are created daily at
> http://download.fedoraproject.org/pub/fedora/linux/development/15/x86_64/os
> 
> However, those images use only content that is in the base F-15 (stable) repo. 
> anaconda-15.22.1 is not yet in the stable repository.  Until then, I have
> created a custom boot.iso (and pxeboot images) that you can point virt-install
> (or koan) to.
> 
> http://jlaska.fedorapeople.org/updates/anaconda-15.22-1/x86_64/os/
> 
> If you are able to boot anaconda-15.22-1, can you please add positive karma
> feedback to https://admin.fedoraproject.org/updates/anaconda-15.22-1.fc15

Works for me, too. Karma++. Thanks for providing the images!

Shorter reproducer, from python on tty2:

>>> from pyanaconda.storage import *

This avoids all the path weirdness we do in the anaconda script, so maybe it helps narrow down what's going on.

Reopening. 

I hit this again with 15.22 now that it got pushed into repos. Exactly the same traceback, reproducible with:

python -c "from pyanaconda.storage import *"

I'm not sure why I didn't hit it with James' custom initrd - maybe during the initrd build he rebuilt the bindings that seem to be failing?

*** Bug 688524 has been marked as a duplicate of this bug. ***

*** Bug 688514 has been marked as a duplicate of this bug. ***

Try:

  $ readelf -a /lib64/libuuid.so.1 | grep uuid_generate

you have to see the symbol in the output, something like:

  63: 0000003e50e02640    41 FUNC    GLOBAL DEFAULT   12 uuid_generate@@UUID_1.0

Do you really have libuuid from util-linux package?

There was libuuid in e2fsprogs package. The newly compiled binaries (with verioned ABI) cannot be executed against the old library.

A simpler reproducer:

import parted
try:
    import volume_key
except ImportError as e:
    print e
    print 'no volume key'
    volume_key = None

import pycryptsetup

Note the 'import parted' part is necessary. Why?

Patch adding missing library (presence of which makes the problem go away) was sent for a review: https://www.redhat.com/archives/anaconda-devel-list/2011-March/msg00222.html

Note we still don't know the root cause of the import failures.

This is not Anaconda specific, I am able to reproduce this on an installed system:
1) use the f15 alpha netinst to install the system, boot it
2) yum update
3) switch for the current f15 devel repo, e.g. http://download.fedora.redhat.com/pub/fedora/linux/development/15/x86_64/os/
4) yum install python-cryptsetup python-volume_key pyparted
5) hide away or remove /usr/lib64/libassuan.*
6) run the code in comment 21

the results are the volume_key complaining about a missing libassuan (that makes sense) but then right after cryptsetup complaining about missing uuid_generate (that makes no sense). 

a strange thing also is pycryptsetup can still be imported if the import of the other two packages is not attempted first. the order also matters. this is very weird and points to either

a) one of the packages playing nasty tricks with the import mechanism

b) more fundamental problem with dynamic libraries load mechanism.

David, have you ever seen anything like that? Who would you talk to (note I doubt it is specifically a python problem)?

(forgot in the reproducer description: one needs to run ldconfig as root after step 5).

*** Bug 688682 has been marked as a duplicate of this bug. ***

I can also provide access to the virtual machine where everything is set up.

I tried with an f14 kernel 2.6.35.6-45.fc14.x86_64 with no different result and then with an f14 glibc (glibc-2.12.90-17.x86_64) and the problem is gone (pycryptsetup can still be imported after a failed volume_key import).

Reassigning to glibc. Glibc mainteiner: I'll be happy to share the virtual machine where this is reproducible. Please do not let this sit, it seems like a serious, sneaky issue.

I doubt this is util-linux-ng problem.

In short: when we import few (unrelated?) python modules (which load dynamic libraries) and one of those modules legitimately fails to load do to a missing .so then pycryptsetup also fails to load (with: ImportError: /lib64/libcryptsetup.so.1: undefined symbol: uuid_generate. note the lib and the symbol ARE present). If the python interpreter is restarted then pycryptsetup can be loaded without a problem. 

And reverting glibc makes this go away.

(In reply to comment #14)

> However, those images use only content that is in the base F-15 (stable) repo. 
> anaconda-15.22.1 is not yet in the stable repository.  Until then, I have
> created a custom boot.iso (and pxeboot images) that you can point virt-install
> (or koan) to.
> 
> http://jlaska.fedorapeople.org/updates/anaconda-15.22-1/x86_64/os/
> 
> If you are able to boot anaconda-15.22-1, can you please add positive karma
> feedback to https://admin.fedoraproject.org/updates/anaconda-15.22-1.fc15

This fix in comment #14 has worked for me too. 
If it helps to narrow down why it is not being seen more, I am updating my local copy infrequently, this bug did not occur in the copy I made on March 10th, and I was experiencing the problem on March 16th (first reported here though 13th, so that means 3 days really).

Discussed at 2011-03-18 blocker review meeting. We agreed to make this bug specific to the lorax bug which the underlying glibc bug causes, and make it a Beta blocker. We cloned this bug off for the glibc issue, so that on doesn't get lost. This bug can be closed once the workaround is implemented and pushed.

The clone for the glibc issue is https://bugzilla.redhat.com/show_bug.cgi?id=688990 .

*** Bug 689159 has been marked as a duplicate of this bug. ***

Fixed by eefd77e1b116a6821b402013983f64a1ae86d23b on master and c0a8d7deff4b5a928873249c5436f453ad4a8341 on f15-branch.

lorax-0.3.2-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/lorax-0.3.2-1.fc15

Verified this fix in a private boot.iso build using the updated lorax and anaconda packages.

Verified this fix using a private boot.iso in a QEMU/KVM x86_64 VM.

lorax-0.3.2-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

*** Bug 690180 has been marked as a duplicate of this bug. ***

*** Bug 690742 has been marked as a duplicate of this bug. ***