Bug 1351368

Summary: IPA: client component of ipa-server install failed with httplib.BadStatusLine
Product: Red Hat Enterprise Linux 7 Reporter: Vasavi Gupta <vchindal>
Component: mod_auth_gssapiAssignee: Simo Sorce <ssorce>
Status: CLOSED DUPLICATE QA Contact: BaseOS QE - Apps <qe-baseos-apps>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: nsoman, pvoborni, rcritten, ssorce, vchindal
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-30 17:23:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vasavi Gupta 2016-06-29 22:06:04 UTC
Description of problem:
client component of ipa server installation failed with httplib.BadStatusLine 

Version-Release number of selected component (if applicable):
ipa-server-4.2.0-15.el7_2.18.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Use quickinstall to install ipa-server-4.2.0-15.el7_2.18.x86_64

Actual results:
DEBUG The ipa-server-install command failed, exception: SystemExit: Configuration of client side components failed!

Expected results:
Successfull installation of all components

Additional info:

2016-06-29T14:56:36Z DEBUG args='/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'testrelm.test' '--server' 'hp-xw6600-02.testrelm.test' '--realm' 'TESTRELM.TEST' '--hostname' 'hp-xw6600-02.testrelm.test'
2016-06-29T14:56:46Z DEBUG Process finished, return code=1
2016-06-29T14:56:46Z DEBUG stdout=

2016-06-29T14:56:46Z DEBUG stderr=Using existing certificate '/etc/ipa/ca.crt'.
Client hostname: hp-xw6600-02.testrelm.test
Realm: TESTRELM.TEST
DNS Domain: testrelm.test
IPA Server: hp-xw6600-02.testrelm.test
BaseDN: dc=testrelm,dc=test
Skipping synchronizing time with NTP server.
New SSSD config will be created
Configured sudoers in /etc/nsswitch.conf
Configured /etc/sssd/sssd.conf
trying https://hp-xw6600-02.testrelm.test/ipa/json
Forwarding 'ping' to json server 'https://hp-xw6600-02.testrelm.test/ipa/json'
Traceback (most recent call last):
  File "/usr/sbin/ipa-client-install", line 3102, in <module>
    sys.exit(main())
  File "/usr/sbin/ipa-client-install", line 3083, in main
    rval = install(options, env, fstore, statestore)
  File "/usr/sbin/ipa-client-install", line 2780, in install
    api.Backend.rpcclient.forward('ping')
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 888, in forward
    return self._call_command(command, params)
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 865, in _call_command
    return command(*params)
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 1016, in _call
    return self.__request(name, args)
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 985, in __request
    verbose=self.__verbose >= 3,
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python2.7/site-packages/ipalib/rpc.py", line 574, in single_request
    return SSLTransport.single_request(self, host, handler, request_body, verbose)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1303, in single_request
    response = h.getresponse(buffering=True)
  File "/usr/lib64/python2.7/httplib.py", line 1051, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 415, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 379, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

2016-06-29T14:56:46Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 311, in run
    cfgr.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 281, in run
    self.execute()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 303, in execute
    for nothing in self._executor():
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 343, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 365, in _handle_exception
    util.raise_exc_info(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 333, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 87, in run_generator_with_yield_from
    raise_exc_info(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 65, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 539, in _configure
    executor.next()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 343, in __runner
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 421, in _handle_exception
    self.__parent._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 365, in _handle_exception
    util.raise_exc_info(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 418, in _handle_exception
    super(ComponentBase, self)._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 365, in _handle_exception
    util.raise_exc_info(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 333, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 87, in run_generator_with_yield_from
    raise_exc_info(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 65, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install
    for nothing in self._installer(self.parent):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 1614, in main
    install(self)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 263, in decorated
    func(installer)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/install.py", line 899, in install
    "ipa-client-install returned: " + str(e))

2016-06-29T14:56:46Z DEBUG The ipa-server-install command failed, exception: SystemExit: Configuration of client side components failed!
ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'testrelm.test' '--server' 'hp-xw6600-02.testrelm.test' '--realm' 'TESTRELM.TEST' '--hostname' 'hp-xw6600-02.testrelm.test'' returned non-zero exit status 1
2016-06-29T14:56:46Z ERROR Configuration of client side components failed!
ipa-client-install returned: Command ''/usr/sbin/ipa-client-install' '--on-master' '--unattended' '--domain' 'testrelm.test' '--server' 'hp-xw6600-02.testrelm.test' '--realm' 'TESTRELM.TEST' '--hostname' 'hp-xw6600-02.testrelm.test'' returned non-zero exit status 1

Comment 2 Petr Vobornik 2016-06-30 10:37:00 UTC
Could you attach httpd access and error log from the installation time?

Does it work with prior versions of following packages?
   ipa-4.2.0-15.el7_2.18
   mod_auth_gssapi-1.3.1-2.el7_2

I.e., is it a regression introduced in one of these builds?

Comment 4 Petr Vobornik 2016-06-30 16:14:12 UTC
httpd error log:

[Thu Jun 30 11:52:07.131596 2016] [core:notice] [pid 18186] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Thu Jun 30 11:52:07.227218 2016] [suexec:notice] [pid 18186] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Jun 30 11:52:07.227270 2016] [:warn] [pid 18186] NSSSessionCacheTimeout is deprecated. Ignoring.
[Thu Jun 30 11:52:07.460744 2016] [auth_digest:notice] [pid 18186] AH01757: generating secret for digest authentication ...
[Thu Jun 30 11:52:07.461543 2016] [lbmethod_heartbeat:notice] [pid 18186] AH02282: No slotmem from mod_heartmonitor
[Thu Jun 30 11:52:07.461565 2016] [:warn] [pid 18186] NSSSessionCacheTimeout is deprecated. Ignoring.
[Thu Jun 30 11:52:07.470177 2016] [mpm_prefork:notice] [pid 18186] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) mod_auth_gssapi/1.3.1 mod_nss/2.4.6 NSS/3.19.1 Basic ECC mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Thu Jun 30 11:52:07.470203 2016] [core:notice] [pid 18186] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Jun 30 11:52:09.560110 2016] [:error] [pid 18192] ipa: INFO: *** PROCESS START ***
[Thu Jun 30 11:52:09.582429 2016] [:error] [pid 18191] ipa: INFO: *** PROCESS START ***
[Thu Jun 30 11:53:40.924273 2016] [mpm_prefork:notice] [pid 18186] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Jun 30 11:53:42.547502 2016] [core:notice] [pid 19587] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Thu Jun 30 11:53:42.548160 2016] [suexec:notice] [pid 19587] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Jun 30 11:53:42.548180 2016] [:warn] [pid 19587] NSSSessionCacheTimeout is deprecated. Ignoring.
[Thu Jun 30 11:53:42.848803 2016] [auth_digest:notice] [pid 19587] AH01757: generating secret for digest authentication ...
[Thu Jun 30 11:53:42.849482 2016] [lbmethod_heartbeat:notice] [pid 19587] AH02282: No slotmem from mod_heartmonitor
[Thu Jun 30 11:53:42.849496 2016] [:warn] [pid 19587] NSSSessionCacheTimeout is deprecated. Ignoring.
[Thu Jun 30 11:53:42.856977 2016] [mpm_prefork:notice] [pid 19587] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) mod_auth_gssapi/1.3.1 mod_nss/2.4.6 NSS/3.19.1 Basic ECC mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Thu Jun 30 11:53:42.856994 2016] [core:notice] [pid 19587] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Thu Jun 30 11:53:44.873538 2016] [:error] [pid 19602] ipa: INFO: *** PROCESS START ***
[Thu Jun 30 11:53:45.040349 2016] [:error] [pid 19601] ipa: INFO: *** PROCESS START ***
[Thu Jun 30 11:53:47.870005 2016] [core:notice] [pid 19587] AH00051: child pid 19604 exit signal Segmentation fault (11), possible coredump in /etc/httpd
[Thu Jun 30 11:53:47.870096 2016] [core:notice] [pid 19587] AH00051: child pid 19605 exit signal Segmentation fault (11), possible coredump in /etc/httpd

Comment 6 Petr Vobornik 2016-06-30 16:33:52 UTC
Moving to mod_auth_gssapi:

part of backtrace
{   "signal": 11
,   "executable": "/usr/sbin/httpd"
,   "stacktrace":
      [ {   "crash_thread": true
        ,   "frames":
              [ {   "address": 140710523253632
                ,   "build_id": "0835a91f48f7ba1ba7236048640cff353831bbb2"
                ,   "build_id_offset": 25472
                ,   "function_name": "mag_auth"
                ,   "file_name": "/etc/httpd/modules/mod_auth_gssapi.so"
                }
              , {   "address": 140710751086464
                ,   "build_id": "2127abafa6b15b048cf32290d5c0aa65555d8c0e"
                ,   "build_id_offset": 219008
                ,   "function_name": "ap_run_check_user_id"
                ,   "file_name": "/usr/sbin/httpd"
                }
              , {   "address": 140710751098770
                ,   "build_id": "2127abafa6b15b048cf32290d5c0aa65555d8c0e"
                ,   "build_id_offset": 231314
                ,   "function_name": "ap_process_request_internal"
                ,   "file_name": "/usr/sbin/httpd"
                }

Comment 7 Simo Sorce 2016-06-30 17:17:43 UTC
I think I found the issue, patch soon

Comment 9 Petr Vobornik 2016-06-30 17:23:11 UTC
This is a regression in a patch for bug 1347175, therefore closing this one and it should be fixed in bug 1347175.

*** This bug has been marked as a duplicate of bug 1347175 ***