Cloning - I would like to investigate this problem more closely. +++ This bug was initially created as a clone of Bug #661682 +++ Created attachment 467729 [details] Anaconda logs Description of problem: Uprade from f13->f14 (i386) got stuck at middle of installing packages. I could switch to console, but not back to GUI, since GUI was stuck. Version-Release number of selected component (if applicable): I don't know the runtime version of anaconda, I did preupgrade from f13, whichever was the version it happened to use at the time of yesterday. How reproducible: I don't know. Steps to Reproduce: 1. I updated F13 2. I run preupgrade to download the packages and rebooted 3. At boot I had to tell anaconda to use proxies while it asked for it 4. Just let run until got stuck Actual results: Upgrade GUI got stuck with the latest error message: 15:13:35,562 WARN anaconda: /usr/lib/python2.7/site-packages/pyanaconda/iw/progress_gui.py:67: GtkWarning: Failed to set text from markup due to error parsing markup: Error on line 2 char 32: Element 'markup' was closed, but the currently open element is 'NAME' self.infolabel.set_markup(txt) Expected results: Upgrade to finnish the package installs. Additional info: See attachments for more logs --- Additional comment from clumens on 2010-12-09 15:16:02 CET --- You appear to be hung up on the package scriptlet for this package: Upgrading openldap-servers-2.4.23-4.fc14.i686 bdb_db_open: warning - no DB_CONFIG file found in directory /var/lib/ldap: (2). Expect poor performance for suffix "dc=my-domain,dc=com". How long did you let it sit? There's really nothing anaconda can do here (except redo the UI to actually refresh, but ugh). --- Additional comment from ilkka.tengvall on 2010-12-10 14:43:39 CET --- It was sitting there over night. I haven't even configured openldap-servers in my system, too bad the default settings fail the upgrade.
Ilkka, was your F13 up-to-date before the upgrade? Do you know which version of openldap you had? I haven't encountered any problem when upgrading from openldap-2.4.21-11.fc13 to openldap-servers-2.4.23-4.fc14. I tried upgrading with database filled with quite a lot of data, with empty database and even with not-created database. Everything seems to be fine. Jan
I still have the /var lvm snapshot still available. It seems the last version of updated ldap was: Aug 24 15:49:29 Updated: openldap-2.4.21-10.fc13.x86_64 Aug 24 15:50:21 Updated: openldap-clients-2.4.21-10.fc13.x86_64 Aug 24 15:50:45 Updated: openldap-2.4.21-10.fc13.i686 Aug 24 15:52:38 Updated: openldap-devel-2.4.21-10.fc13.x86_64 Hmmm, that's weird, I seem to have both 64/32 bit versions installed. I have Wind River environment on this PC, and it requires certain packages for 32 bit also. I don't see another reason for that. and yes, the F13 was up-to-date. And I had nothing in ldap database, it was unused. I installed ldap stuff to debug something, but I was not using the server for anything real, never did anything with the database.
I managed to reproduce this by damaging the database a little bit before the upgrade. (You might not even know, if you weren't using openldap-servers. Unclean server shutdown is also possible.) In fact it is the same problem as here (bug 667768). It seems that slapd tools (slaptest, slapcat, slapadd, ...) get stuck if the database is inconsistent. Unfortunately it can not be fixed as easily as the previous bug. There is no way to check that database is OK before doing the database export (which is done during the upgrade). This is definitely a bug, not very serious one. I suppose that nobody with damaged ldap database will run package upgrade knowingly. I will investigate this more and try to find some solution. Ilkka, Thanks for reporting. :)
heh, thanks, good to know it wasn't only in between my ears after all :D !
Can be fixed by adding 'db_recover' before database migration.
Fixed in openldap-2.4.23-7.fc14
openldap-2.4.23-7.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/openldap-2.4.23-7.fc14
openldap-2.4.23-7.fc14 has been pushed to the Fedora 14 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update openldap'. You can provide feedback for this update here: https://admin.fedoraproject.org/updates/openldap-2.4.23-7.fc14
openldap-2.4.23-8.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/openldap-2.4.23-8.fc14
Package openldap-2.4.21-12.fc13: * should fix your issue, * was pushed to the Fedora 13 updates-testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing openldap-2.4.21-12.fc13' as soon as you are able to, then reboot. Please go to the following url: https://admin.fedoraproject.org/updates/openldap-2.4.21-12.fc13 then log in and leave karma (feedback).
Package openldap-2.4.23-9.fc14: * should fix your issue, * was pushed to the Fedora 14 updates-testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing openldap-2.4.23-9.fc14' as soon as you are able to, then reboot. Please go to the following url: https://admin.fedoraproject.org/updates/openldap-2.4.23-9.fc14 then log in and leave karma (feedback).
thanks for solving this. I don't though know what to put into karma, since I don't want to test it by re-installing old f13 over my existing f14 and upgrade it back to f14. I can test it though if you tell me what you did to break the db in purpose to get it stuck, and a way to repeat the error. Only if you want me to, I trust your tests though.
Sorry, I was too hasty... The upgrade got stuck: --&<----------------------------------------------------------- [itengval@pikkud ~]$ sudo yum update --enablerepo=updates-testing openldap-2.4.23-9.fc14 Loaded plugins: presto, refresh-packagekit updates-testing/metalink | 17 kB 00:00 updates-testing | 4.7 kB 00:00 updates-testing/primary_db | 790 kB 00:00 Setting up Update Process Resolving Dependencies --> Running transaction check --> Processing Dependency: openldap = 2.4.23-4.fc14 for package: openldap-devel-2.4.23-4.fc14.i686 --> Processing Dependency: openldap = 2.4.23-4.fc14 for package: openldap-clients-2.4.23-4.fc14.i686 --> Processing Dependency: openldap = 2.4.23-4.fc14 for package: openldap-servers-2.4.23-4.fc14.i686 ---> Package openldap.i686 0:2.4.23-9.fc14 set to be updated --> Running transaction check ---> Package openldap-clients.i686 0:2.4.23-9.fc14 set to be updated ---> Package openldap-devel.i686 0:2.4.23-9.fc14 set to be updated ---> Package openldap-servers.i686 0:2.4.23-9.fc14 set to be updated --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Updating: openldap i686 2.4.23-9.fc14 updates-testing 256 k Updating for dependencies: openldap-clients i686 2.4.23-9.fc14 updates-testing 157 k openldap-devel i686 2.4.23-9.fc14 updates-testing 1.1 M openldap-servers i686 2.4.23-9.fc14 updates-testing 2.0 M Transaction Summary ================================================================================ Upgrade 4 Package(s) Total download size: 3.5 M Is this ok [y/N]: y Downloading Packages: Setting up and reading Presto delta metadata updates-testing/prestodelta | 22 kB 00:00 Processing delta metadata Package(s) data still to download: 3.5 M (1/4): openldap-2.4.23-9.fc14.i686.rpm | 256 kB 00:00 (2/4): openldap-clients-2.4.23-9.fc14.i686.rpm | 157 kB 00:00 (3/4): openldap-devel-2.4.23-9.fc14.i686.rpm | 1.1 MB 00:00 (4/4): openldap-servers-2.4.23-9.fc14.i686.rpm | 2.0 MB 00:00 -------------------------------------------------------------------------------- Total 5.5 MB/s | 3.5 MB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Updating : openldap-2.4.23-9.fc14.i686 1/8 Updating : openldap-clients-2.4.23-9.fc14.i686 2/8 Updating : openldap-servers-2.4.23-9.fc14.i686 3/8 --&<----------------------------------------------------------- Looking simultaneously at the yum log --&<----------------------------------------------------------- Mar 01 14:41:05 Updated: ruby-libs-1.8.7.334-1.fc14.i686 Mar 01 14:41:05 Installed: nss-softokn-freebl-devel-3.12.9-5.fc14.i686 Mar 01 14:41:05 Updated: nss-softokn-devel-3.12.9-5.fc14.i686 Mar 01 14:41:05 Updated: nss-devel-3.12.9-8.fc14.i686 Mar 02 11:43:09 Updated: gpxe-roms-qemu-1.0.1-3.fc14.noarch Mar 02 11:43:09 Updated: 1:NetworkManager-glib-0.8.3.996-1.fc14.i686 Mar 02 11:43:13 Updated: 1:NetworkManager-0.8.3.996-1.fc14.i686 Mar 02 11:43:24 Updated: 1:NetworkManager-gnome-0.8.3.996-1.fc14.i686 Mar 02 11:44:13 Updated: openldap-2.4.23-9.fc14.i686 Mar 02 11:44:13 Updated: openldap-clients-2.4.23-9.fc14.i686 --&<----------------------------------------------------------- So there is no log of openldap-servers. It's been 20 minutes now stuck there. And DB should be rather empty, I don't use it for anything. You can here see from ps awfux what it's doing: --&<----------------------------------------------------------- root 4191 0.0 0.0 8332 1596 pts/0 S+ 11:43 0:00 | \_ sudo yum update --enablerepo=updates-testing openldap-2.4.23-9.fc14 root 4192 0.9 3.2 124196 100104 pts/0 S+ 11:43 0:11 | \_ /usr/bin/python /usr/bin/yum update --enablerepo=updates-testing openldap-2.4.23 root 4248 0.0 0.0 5084 1092 pts/0 S+ 11:44 0:00 | \_ /bin/sh /home/itengval/rpmbuild/rpm/tmp/rpm-tmp.5uZ0N8 2 root 4252 0.0 0.0 5748 1136 pts/0 S+ 11:44 0:00 | \_ runuser -m -s /usr/sbin/slapadd -- ldap -q -l /var/lib/ldap/upgrade.ldif ldap 4253 0.0 0.1 13832 3300 pts/0 Sl+ 11:44 0:00 | \_ slapadd -q -l /var/lib/ldap/upgrade.ldif --&<----------------------------------------------------------- and here is what the ldap is having open (lsof -p 4253): --&<----------------------------------------------------------- COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME slapadd 4253 ldap cwd DIR 253,1 4096 2 / slapadd 4253 ldap rtd DIR 253,1 4096 2 / slapadd 4253 ldap txt REG 253,1 2241560 26958 /usr/sbin/slapd slapadd 4253 ldap mem REG 253,1 1577412 26181 /lib/libdb-4.8.so slapadd 4253 ldap mem REG 253,1 105200 18360 /lib/libresolv-2.13.so slapadd 4253 ldap mem REG 253,1 1274636 14000 /usr/lib/libnss3.so slapadd 4253 ldap mem REG 253,1 107528 70518 /usr/lib/libnssutil3.so slapadd 4253 ldap mem REG 253,1 12164 70427 /lib/libplds4.so slapadd 4253 ldap mem REG 253,1 16676 69396 /lib/libplc4.so slapadd 4253 ldap mem REG 253,1 240412 17322 /lib/libnspr4.so slapadd 4253 ldap mem REG 253,1 133344 18349 /lib/libpthread-2.13.so slapadd 4253 ldap mem REG 253,1 19776 14990 /lib/libdl-2.13.so slapadd 4253 ldap mem REG 253,1 84848 17743 /lib/libz.so.1.2.5 slapadd 4253 ldap mem REG 253,1 169060 14538 /usr/lib/libsmime3.so slapadd 4253 ldap mem REG 253,1 216536 14705 /usr/lib/libssl3.so slapadd 4253 ldap mem REG 253,1 298084 12682 /lib/libfreebl3.so slapadd 4253 ldap mem REG 253,1 36132 19914 /lib/libcrypt-2.13.so slapadd 4253 ldap mem REG 253,1 137160 14144 /lib/ld-2.13.so slapadd 4253 ldap mem REG 253,1 1847224 16957 /lib/libc-2.13.so slapadd 4253 ldap mem REG 253,1 102740 20866 /usr/lib/libsasl2.so.2.0.23 slapadd 4253 ldap mem REG 253,1 36852 61664 /usr/lib/libltdl.so.7.2.2 slapadd 4253 ldap 0r FIFO 0,8 0t0 164117 pipe slapadd 4253 ldap 1w CHR 1,3 0t0 4032 /dev/null slapadd 4253 ldap 2u REG 253,1 138 17497 /tmp/tmpcAgZ4K slapadd 4253 ldap 3r REG 253,1 0 218516 /var/lib/ldap/upgrade.ldif --&<----------------------------------------------------------- this is the temp file content: --&<----------------------------------------------------------- $ sudo cat /tmp/tmpcAgZ4K bdb_db_open: warning - no DB_CONFIG file found in directory /var/lib/ldap: (2). Expect poor performance for suffix "dc=my-domain,dc=com". --&<----------------------------------------------------------- and notice the file upgrade.ldif is empty Any additional info you would like to get from the environment (my work laptop)? br, it
If upgrade.ldif is empty, then the recovery tool was unable to fix the broken database (1). Or: the database is in the old format which cannot be opened by current version of db4 tools (2). The first possibility is very unlikely. But the second one probably happened. I found another nasty bug in the upgrade script - the database is not migrated if we the upgrade is done from OpenLDAP with embedded DB4 to some newer version. If you were not using openldap-servers, you probably didn't notice it. And the problem appeared now (after some improvements in the upgrade script). I rewrote the script to be able to handle this situation without damaging the data and without a freeze during the package upgrade. I will push the update soon.
Hm... I was not entirely right. The problem didn't disappear entirely. I found this issue: http://www.openldap.org/its/index.cgi?findid=6853 And upstream fix seems to work. :-)
Fixed in openldap-2.4.23-10.fc14
openldap-2.4.23-10.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/openldap-2.4.23-10.fc14
openldap-2.4.24-2.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/openldap-2.4.24-2.fc15
openldap-2.4.24-2.fc15 has been pushed to the Fedora 15 stable repository. If problems still persist, please make note of it in this bug report.
openldap-2.4.23-10.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report.