Description of problem: While rebuilding FreeIPA server using Fedora 19 beta ipa-server-install fails. Version-Release number of selected component (if applicable): ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: bind x86_64 32:9.9.3-3.P1.fc19 updates-testing 1.7 M bind-dyndb-ldap x86_64 3.3-1.fc19 updates-testing 90 k freeipa-server x86_64 3.2.0-2.fc19 fedora 1.1 M How reproducible: Steps to Reproduce: 1. Build a new Fedora 19 Beta VM 2. yum install --assumeyes bind bind-dyndb-ldap freeipa-server 3. ipa-server-install \ --admin-password adminpassword \ --domain hunter.org \ --ds-password dspassword \ --hostname ipa.hunter.org \ --idstart 128000000 \ --mkhomedir \ --no-forwarders \ --no-ntp \ --realm HUNTER.ORG \ --setup-dns \ --ssh-trust-dns \ --unattended Actual results: ... Configuring Kerberos KDC (krb5kdc): Estimated time 30 seconds [1/10]: adding sasl mappings to the directory [2/10]: adding kerberos container to the directory [3/10]: configuring KDC [4/10]: initialize kerberos container Failed to initialize the realm container [5/10]: adding default ACIs [6/10]: creating a keytab for the directory [7/10]: creating a keytab for the machine [8/10]: adding the password extension to the directory [9/10]: starting the KDC [10/10]: configuring KDC to start on boot Done configuring Kerberos KDC (krb5kdc)Unexpected error - see /var/log/ipaserver-install.log for details: CalledProcessError: Command '/bin/systemctl restart kadmin.service' returned non-zero exit status 1 . Configuring kadmin .... Expected results: No errors Additional info:
Created attachment 757755 [details] ipaserver-install.log
There is regression in libbsd that caused this issue. Please downgrade and see https://admin.fedoraproject.org/updates/FEDORA-2013-10062/libbsd-0.5.1-1.fc19
I'll try to find the problem in libbsd. In the mean time I've unpushed the update. Thanks for providing the detailed instructions to reproduce the problem. I've tried it, and I get a failure before the one you describe, at step 6 of "Configuring Kerberos KDC": Configuring Kerberos KDC (krb5kdc): Estimated time 30 seconds [1/10]: adding sasl mappings to the directory [2/10]: adding kerberos container to the directory [3/10]: configuring KDC [4/10]: initialize kerberos container Failed to initialize the realm container [5/10]: adding default ACIs [6/10]: creating a keytab for the directory Unexpected error - see /var/log/ipaserver-install.log for details: CalledProcessError: Command 'kadmin.local -q addprinc -randkey ldap/ipa.z00.us -x ipa-setup-override-restrictions' returned non-zero exit status 1 This is probably some sort of operator error on my part, but I'm not sure how to resolve it, so any advice will be appreciated.
Created attachment 757779 [details] ipaserver-install.log showing failure at step 6 of KDC setup I think this is probably not actually related to this bug, but I need to know how to get past this in order to reproduce the actual problem in libbsd-0.5.1.
The failure occurs when krb5_util is run. It segfaults when creating the Kerberos database which then causes kadmin.local to fail later. We should probably be catching this failure in the IPA installer so it fails at the right place (the "Failed to initialize realm container" message above. The only way we've been able to get past this failure so far is to downgrade libbsd to 0.4.
I understand that, but following the given instructions doesn't even get me that far. I get a failure at step 6 of KDC setup, not step 10. That's why I was requesting info on how to get past that.
The error does not occur in step 10. As Rob noted there is a bug in ipa-server-install such that it does not stop when the failure occurs in step 4 or perhaps even earlier.
I don't understand why the output you included from the command shows steps up through 10, while the output I get stops at 6 with a different error reported.
I do not understand either. But the error I am reporting is in step 4.
Eric, I think we found the issue. IPA KDB driver is linked against Samba libraries. These libraries, in turn, are linked against Samba's internal library that provides replacement for functions known to be broken or not implemented on the platform. Some code in that library (libreplace) uses libbsd. libbsd 0.5.1 has setproctitle() replacement. The code in src/setproctitle.c sets up spt_init() as a constructor that is called on library initialization. spt_init() calls internally environ handling code, to copy it. spt_copyenv() calls to spt_clearenv(). spt_clearenv() calls system-provided clearenv() if that is there. Here is where the problem comes. Glibc's clearenv() frees environment memory if that was allocated by the glibc (as in our case). It also always returns 0. Any allocation for the new environment will happen at first setenv() call. At this point spt_copyenv() attempts to access memory which was freed by Glibc's clearenv() and may or may not crash. For Nalin under valgrind it crashed, so we've got following backtrace (I'm cutting down to interesting parts): #0 __strchr_sse2 () at ../sysdeps/x86_64/strchr.S:32 #1 0x00007fffec389cdb in spt_copyenv (oldenv=0x55555577ec10) at setproctitle.c:94 #2 spt_init (argc=8, argv=0x7fffffffe448, envp=0x55555577ec10) at setproctitle.c:172 #3 0x00007ffff7deb4f3 in call_init (env=0x55555577ec10, argv=0x7fffffffe448, argc=8, l=<optimized out>) at dl-init.c:82 #4 _dl_init (main_map=main_map@entry=0x5555557baac0, argc=8, argv=0x7fffffffe448, env=0x55555577ec10) at dl-init.c:131 #5 0x00007ffff7defb04 in dl_open_worker (a=a@entry=0x7fffffffda98) at dl-open.c:566 #6 0x00007ffff7deb304 in _dl_catch_error ( objname=objname@entry=0x7fffffffda88, errstring=errstring@entry=0x7fffffffda90, mallocedp=mallocedp@entry=0x7fffffffda80, operate=operate@entry=0x7ffff7def770 <dl_open_worker>, args=args@entry=0x7fffffffda98) at dl-error.c:177 #7 0x00007ffff7def21b in _dl_open ( file=0x5555557baa40 "/usr/lib64/krb5/plugins/kdb/ipadb.so", mode=-2147483646, caller_dlopen=0x7ffff6a143cd <krb5int_open_plugin+349>, nsid=-2, argc=8, argv=<optimized out>, env=0x55555577ec10) at dl-open.c:656 #8 0x00007ffff733502b in dlopen_doit (a=a@entry=0x7fffffffdca0) at dlopen.c:66 So, spt_clearenv() needs to be smarter when it uses system-provided clearenv() and spt_copyenv() should be careful -- oldenv will likely be deleted before first use.
Moving to libbsd.
I reported the bug upstream. See bug #971513 for further info.
libbsd-0.5.1-2.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/libbsd-0.5.1-2.fc19
libbsd-0.5.1-2.fc19 resolves the problem for me: yum install --assumeyes createrepo koji mkdir /root/local cd /root/local koji download-build --arch x86_64 libbsd-0.5.1-2.fc19 cd - createrepo /root/local cat >/etc/yum.repos.d/local.repo <<EOD [local] baseurl=file:///root/local enabled=0 gpgcheck=0 name=Local Updates $releasever - $basearch EOD yum install --assumeyes --enablerepo local bind bind-dyndb-ldap freeipa-server ... Updating for dependencies: bind-libs x86_64 32:9.9.3-3.P1.fc19 updates-testing 978 k bind-libs-lite x86_64 32:9.9.3-3.P1.fc19 updates-testing 693 k bind-license noarch 32:9.9.3-3.P1.fc19 updates-testing 77 k libbsd x86_64 0.5.1-2.fc19 local 61 k libldb x86_64 1.1.15-3.fc19 updates-testing 119 k libtdb x86_64 1.2.12-1.fc19 updates-testing 38 k Transaction Summary ....
Just for my education, is there a way to tell koji where to download files instead of the current directory?
Package libbsd-0.5.1-2.fc19: * should fix your issue, * was pushed to the Fedora 19 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing libbsd-0.5.1-2.fc19' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-10259/libbsd-0.5.1-2.fc19 then log in and leave karma (feedback).
libbsd-0.5.2-1.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/libbsd-0.5.2-1.fc19
libbsd-0.5.2-1.fc19 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report.