Created attachment 1441093 [details] ipaclient install log Description of problem: In a freshly update rawhide i did an ipa-client-install Very thing go well I even got the "Client uninstall complete." message but then... Boom! the install failed. Version-Release number of selected component (if applicable): freeipa-common-4.6.90.pre2-3.fc29.noarch freeipa-client-common-4.6.90.pre2-3.fc29.noarch freeipa-client-4.6.90.pre2-3.fc29.x86_64 How reproducible: 100% Steps to Reproduce: 1.ipa-client-install Additional info: Attaching the ipaclient-install.log and the screen output
Created attachment 1441095 [details] screen output
2018-05-24T14:58:45Z DEBUG Starting external process 2018-05-24T14:58:45Z DEBUG args=['/usr/sbin/ipa-join', '-s', 'ipa.boston.devel.redhat.com', '-b', 'dc=boston,dc=devel,dc=redhat,dc=com', '-h', 'fedora.boston.devel.redhat.com'] 2018-05-24T14:58:46Z DEBUG Process finished, return code=11 2018-05-24T14:58:46Z DEBUG stdout= 2018-05-24T14:58:46Z DEBUG stderr=Failed to add key to the keytab child exited with 11 so ipa-join failed with SIGSEGV. Interesting. Could you please enable gathering coredumps and retry?
(In reply to Alexander Bokovoy from comment #2) > 2018-05-24T14:58:45Z DEBUG Starting external process > 2018-05-24T14:58:45Z DEBUG args=['/usr/sbin/ipa-join', '-s', > 'ipa.boston.devel.redhat.com', '-b', 'dc=boston,dc=devel,dc=redhat,dc=com', > '-h', 'fedora.boston.devel.redhat.com'] > 2018-05-24T14:58:46Z DEBUG Process finished, return code=11 > 2018-05-24T14:58:46Z DEBUG stdout= > 2018-05-24T14:58:46Z DEBUG stderr=Failed to add key to the keytab > child exited with 11 > > so ipa-join failed with SIGSEGV. Interesting. Could you please enable > gathering coredumps and retry? I just looked under /var/spool/abrt and I didn't see in core dump. I see the abrt-ccpp.service and abrt-journal-core.service are running... what else need to happen to have a core drop?
If it is reproducible, then kinit admin.REDHAT.COM gdb /usr/sbin/ipa-join gdb> run -s ipa.boston.devel.redhat.com -b dc=boston,dc=devel,dc=redhat,dc=com -h fedora.boston.devel.redhat.com would crash as well and you'd be able to collect the stacktrace.
(In reply to Alexander Bokovoy from comment #4) > If it is reproducible, then > > kinit admin.REDHAT.COM > gdb /usr/sbin/ipa-join > gdb> run -s ipa.boston.devel.redhat.com -b > dc=boston,dc=devel,dc=redhat,dc=com -h fedora.boston.devel.redhat.com > > would crash as well and you'd be able to collect the stacktrace. Sorry it took so long to get back to this... gdb) run -s ipa.boston.devel.redhat.com -b dc=boston,dc=devel,dc=redhat,dc=com -h fedora.boston.devel.redhat.com Starting program: /usr/sbin/ipa-join -s ipa.boston.devel.redhat.com -b dc=boston,dc=devel,dc=redhat,dc=com -h fedora.boston.devel.redhat.com warning: Loadable section ".note.gnu.property" outside of ELF segments [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". warning: Loadable section ".note.gnu.property" outside of ELF segments [New Thread 0x7fffe68a3700 (LWP 1717)] [Thread 0x7fffe68a3700 (LWP 1717) exited] HTTP POST to URL 'https://ipa.boston.devel.redhat.com:443/ipa/xml' failed. libcurl failed even to execute the HTTP transaction, explaining: error setting certificate verify locations: CAfile: /etc/ipa/ca.crt CApath: none [Inferior 1 (process 1707) exited with code 021] (gdb) bt No stack.
ipa-join exiting with retcode=11 means 'Failed to add key to keytab', I don't think that the process exited with SIGSEGV in this case. ipa-join is internally calling ipa-getkeytab and this process is returning 11. The issue looks like https://pagure.io/freeipa/issue/4607, is the file /etc/krb5.keytab is symlink to another file?
(In reply to Florence Blanc-Renaud from comment #6) > ipa-join exiting with retcode=11 means 'Failed to add key to keytab', I > don't think that the process exited with SIGSEGV in this case. > ipa-join is internally calling ipa-getkeytab and this process is returning > 11. > > The issue looks like https://pagure.io/freeipa/issue/4607, is the file > /etc/krb5.keytab is symlink to another file? No it is not. fedora$ ll /etc/krb5.keytab 394164 -rw-r--r--. 1 root root 0 Jan 16 11:58 /etc/krb5.keytab But it does exist with a zero length.
Bug analysis ------------ Valid bug: yes Regression: no Regression introduction: N/A Affected versions: RHEL 7.5+, FreeIPA 4.5.4+ Use cases (reproduction steps): * Create an empty /etc/krb5.keytab and launch ipa-client-install: rm /etc/krb5.keytab touch /etc/krb5.keytab ipa-client-install --domain $DOMAIN --realm $REALM --principal admin --password pwd -U Cause: empty /etc/krb5.keytab file Consequence: ipa-client-install fails to purge the keytab and fails to add new keys to the keytab Workaround: rm /etc/krb5.keytab before launching ipa-client-install Fix complexity: ?
Hi, I noticed in the logs that the installer is trying to purge old principals from the existing keytab but fails: 2018-05-24T14:58:21Z DEBUG stderr=Failed to open keytab '/etc/krb5.keytab': Unsupported key table format version number This happens because the file exists but is empty. I reproduced the same behavior with an empty /etc/krb5.keytab. Can you re-try ipa-client-install after removing /etc/krb5.keytab?
For what it's worth, krb5 doesn't think an empty file is a valid keytab. Among other things, it's missing the file format header (0x05 0x02, normally, but it can also be 0x05 0x01) and the end marker (0x00 0x00 0x00 0x00). For more information, see http://web.mit.edu/kerberos/krb5-latest/doc/formats/keytab_file_format.html What caused an empty file to be at /etc/krb5.keytab?
(In reply to Florence Blanc-Renaud from comment #9) > Hi, > I noticed in the logs that the installer is trying to purge old principals > from the existing keytab but fails: > 2018-05-24T14:58:21Z DEBUG stderr=Failed to open keytab '/etc/krb5.keytab': > Unsupported key table format version number > > This happens because the file exists but is empty. I reproduced the same > behavior with an empty /etc/krb5.keytab. > > Can you re-try ipa-client-install after removing /etc/krb5.keytab? The install was successful when I removed the keytab... I notice I got a similar error when I did a simple klist -k so it appears the keytab got corrupted
(In reply to Robbie Harwood from comment #10) > For what it's worth, krb5 doesn't think an empty file is a valid keytab. > Among other things, it's missing the file format header (0x05 0x02, > normally, but it can also be 0x05 0x01) and the end marker (0x00 0x00 0x00 > 0x00). For more information, see > http://web.mit.edu/kerberos/krb5-latest/doc/formats/keytab_file_format.html > > What caused an empty file to be at /etc/krb5.keytab? IDK... maybe when I uninstalled the ipa client?
I think we can close this... unless having an zero length keytab is a bug?
Upstream ticket: https://pagure.io/freeipa/issue/7625
This issue is difficult to diagnose and may prevent client installation. Let's keep this BZ and implement the following improvement: in the installer, we should check if the keytab is already existing and with 0-length. If it is the case, the installer should delete the keytab.
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle. Changing version to '29'.
Fixed upstream master: https://pagure.io/freeipa/c/cf1301fb064fc230c780c4bc5eeccb723899f7b6 ipa-4-7: https://pagure.io/freeipa/c/fc32cbb686c9d6c40607ddc48342403c7ed7bd4a
Test case: master: https://pagure.io/freeipa/c/c049992c6f1a21ec084d6371c8ea287a53e194c0
Test case: ipa-4-7: https://pagure.io/freeipa/c/e9b05971749bcbfc927eab4f50ab3974cd9a2861
Fixed in 4.7.2