Bug 1455541
| Summary: | after upgrade login from web ui breaks | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Petr Vobornik <pvoborni> | ||||
| Component: | ipa | Assignee: | IPA Maintainers <ipa-maint> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Nikhil Dehadrai <ndehadra> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 7.4 | CC: | ksiddiqu, mbabinsk, mvarun, ndehadra, pvoborni, rcritten, slaznick, spoore, sumenon, tscherf | ||||
| Target Milestone: | rc | Keywords: | Regression | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | ipa-4.5.0-15.el7 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-08-01 09:51:24 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1452216 | ||||||
| Attachments: |
|
||||||
|
Description
Petr Vobornik
2017-05-25 12:23:08 UTC
Upstream ticket: https://pagure.io/freeipa/issue/6973 Fixed upstream master: https://pagure.io/freeipa/c/9c3fad9cef7785a65c795f1b4fc3f94e50af9db2 ipa-4-5: https://pagure.io/freeipa/c/db7967061b9b7d001c923ce3da9d6c6036627253 IPA Version: ipa-server-4.5.0-13.el7.x86_64 Tested that after upgrading IPA server to latest version (In my case RHEL 7.3.z to Rhel 7.4): 1) IPA web UI is accessible. 2) We are able to log in to the server. Fixed upstream master: https://pagure.io/freeipa/c/3b6892783ee6ed6dac9c4f328cc89ae030ce10a7 ipa-4-5: https://pagure.io/freeipa/c/37be8e9ac3b46d6d31199227216b5a5a8d5d5614 IPA-server: ipa-server-4.5.0-15.el7.x86_64 Tested the bug for WEBUI login to ipa server after upgrade( in my case RHEL 7.3.z > 7.4), but the login still fails with same error message "Login failed due to an unknown reason. " Refer following logs: [root@vm-idm-029 ~]# rpm -q ipa-server ipa-server-4.5.0-15.el7.x86_64 [root@vm-idm-029 ~]# tail -f /var/log/httpd/error_log [Wed Jun 07 15:03:51.010984 2017] [:error] [pid 29940] [remote 10.67.116.122:212] mod_wsgi (pid=29940): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'. [Wed Jun 07 15:03:51.011104 2017] [:error] [pid 29940] [remote 10.67.116.122:212] Traceback (most recent call last): [Wed Jun 07 15:03:51.011155 2017] [:error] [pid 29940] [remote 10.67.116.122:212] File "/usr/share/ipa/wsgi.py", line 51, in application [Wed Jun 07 15:03:51.011241 2017] [:error] [pid 29940] [remote 10.67.116.122:212] return api.Backend.wsgi_dispatch(environ, start_response) [Wed Jun 07 15:03:51.011271 2017] [:error] [pid 29940] [remote 10.67.116.122:212] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 262, in __call__ [Wed Jun 07 15:03:51.011315 2017] [:error] [pid 29940] [remote 10.67.116.122:212] return self.route(environ, start_response) [Wed Jun 07 15:03:51.011335 2017] [:error] [pid 29940] [remote 10.67.116.122:212] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 274, in route [Wed Jun 07 15:03:51.011372 2017] [:error] [pid 29940] [remote 10.67.116.122:212] return app(environ, start_response) [Wed Jun 07 15:03:51.011393 2017] [:error] [pid 29940] [remote 10.67.116.122:212] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 923, in __call__ [Wed Jun 07 15:03:51.011429 2017] [:error] [pid 29940] [remote 10.67.116.122:212] self.kinit(user_principal, password, ipa_ccache_name) [Wed Jun 07 15:03:51.011449 2017] [:error] [pid 29940] [remote 10.67.116.122:212] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 959, in kinit [Wed Jun 07 15:03:51.011483 2017] [:error] [pid 29940] [remote 10.67.116.122:212] pkinit_anchors=[paths.KDC_CERT, paths.KDC_CA_BUNDLE_PEM], [Wed Jun 07 15:03:51.011506 2017] [:error] [pid 29940] [remote 10.67.116.122:212] File "/usr/lib/python2.7/site-packages/ipalib/install/kinit.py", line 122, in kinit_armor [Wed Jun 07 15:03:51.011566 2017] [:error] [pid 29940] [remote 10.67.116.122:212] run(args, env=env, raiseonerr=True, capture_error=True) [Wed Jun 07 15:03:51.011594 2017] [:error] [pid 29940] [remote 10.67.116.122:212] File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 498, in run [Wed Jun 07 15:03:51.011637 2017] [:error] [pid 29940] [remote 10.67.116.122:212] raise CalledProcessError(p.returncode, arg_string, str(output)) [Wed Jun 07 15:03:51.011732 2017] [:error] [pid 29940] [remote 10.67.116.122:212] CalledProcessError: Command '/usr/bin/kinit -n -c /var/run/ipa/ccaches/armor_29940 -X X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt -X X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem' returned non-zero exit status 1 Thus on the basis of bove observations, changing status to "ASSIGNED". Can you please switch on framework debugging in /etc/ipa/default.conf, re-run the test and then post the httpd error_log? also please check audit log for AVCs although these should be fixed in selinux-policy. Which brings me to the question whether you use selinux-policy-3.13.1-159.el7.noarch which was built as part of https://bugzilla.redhat.com/show_bug.cgi?id=1436689, can you check that? (In reply to Martin Babinsky from comment #14) > Can you please switch on framework debugging in /etc/ipa/default.conf, > re-run the test and then post the httpd error_log? also please check audit > log for AVCs although these should be fixed in selinux-policy. [root@vm-idm-001 ~]# rpm -qa ipa-server ipa-server-4.5.0-15.el7.x86_64 /var/log/httpd/error_log [Wed Jun 07 17:51:47.350374 2017] [:error] [pid 16997] [remote 10.67.116.109:4] mod_wsgi (pid=16997): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'. [Wed Jun 07 17:51:47.350481 2017] [:error] [pid 16997] [remote 10.67.116.109:4] Traceback (most recent call last): [Wed Jun 07 17:51:47.350513 2017] [:error] [pid 16997] [remote 10.67.116.109:4] File "/usr/share/ipa/wsgi.py", line 51, in application [Wed Jun 07 17:51:47.350570 2017] [:error] [pid 16997] [remote 10.67.116.109:4] return api.Backend.wsgi_dispatch(environ, start_response) [Wed Jun 07 17:51:47.350601 2017] [:error] [pid 16997] [remote 10.67.116.109:4] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 262, in __call__ [Wed Jun 07 17:51:47.350640 2017] [:error] [pid 16997] [remote 10.67.116.109:4] return self.route(environ, start_response) [Wed Jun 07 17:51:47.350653 2017] [:error] [pid 16997] [remote 10.67.116.109:4] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 274, in route [Wed Jun 07 17:51:47.350671 2017] [:error] [pid 16997] [remote 10.67.116.109:4] return app(environ, start_response) [Wed Jun 07 17:51:47.350682 2017] [:error] [pid 16997] [remote 10.67.116.109:4] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 923, in __call__ [Wed Jun 07 17:51:47.350701 2017] [:error] [pid 16997] [remote 10.67.116.109:4] self.kinit(user_principal, password, ipa_ccache_name) [Wed Jun 07 17:51:47.350712 2017] [:error] [pid 16997] [remote 10.67.116.109:4] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 959, in kinit [Wed Jun 07 17:51:47.350729 2017] [:error] [pid 16997] [remote 10.67.116.109:4] pkinit_anchors=[paths.KDC_CERT, paths.KDC_CA_BUNDLE_PEM], [Wed Jun 07 17:51:47.350743 2017] [:error] [pid 16997] [remote 10.67.116.109:4] File "/usr/lib/python2.7/site-packages/ipalib/install/kinit.py", line 122, in kinit_armor [Wed Jun 07 17:51:47.350782 2017] [:error] [pid 16997] [remote 10.67.116.109:4] run(args, env=env, raiseonerr=True, capture_error=True) [Wed Jun 07 17:51:47.350793 2017] [:error] [pid 16997] [remote 10.67.116.109:4] File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 498, in run [Wed Jun 07 17:51:47.350815 2017] [:error] [pid 16997] [remote 10.67.116.109:4] raise CalledProcessError(p.returncode, arg_string, str(output)) [Wed Jun 07 17:51:47.350854 2017] [:error] [pid 16997] [remote 10.67.116.109:4] CalledProcessError: Command '/usr/bin/kinit -n -c /var/run/ipa/ccaches/armor_16997 -X X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt -X X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem' returned non-zero exit status 1 selinux-policy-3.13.1-160.el7.noarch [root@vm-idm-001 ~]# ausearch -m AVC -ts today <no matches> [root@vm-idm-001 ~]# cat /var/log/audit/audit.log |audit2allow Nothing to do [root@vm-idm-001 ~]# I've managed to reproduce with clean install. The issue was that kinit which obtains armor required pre-auth. After restarting gssproxy and then krb5kdc it no longer required it and Web UI login started to work again. I'm not sure if both services needs to be restarted or only on of them. I did not manage to reproduce after un-installation and re-instalaltion on the same host. Note the initial installation was with older selinux policy so maybe my reproducer did not match this scenario, but the work-around is worth trying. FYI, I tried restarting gssproxy then krb5kdc with no luck. I thought I'd try again with SELinux in permissive mode and that worked:
Afterwards I did see these AVC denials in the log:
time->Wed Jun 7 14:53:44 2017
type=SYSCALL msg=audit(1496865224.775:130): arch=c000003e syscall=2 success=yes exit=3 a0=563698bc2ed5 a1=0 a2=1b6 a3=24 items=0 ppid=2515 pid=2584 auid=4294967295 uid=387 gid=387 euid=387 suid=387 fsuid=387 egid=387 sgid=387 fsgid=387 tty=(none) ses=4294967295 comm="kinit" exe="/usr/bin/kinit" subj=system_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(1496865224.775:130): avc: denied { open } for pid=2584 comm="kinit" path="/var/lib/ipa-client/pki/kdc-ca-bundle.pem" dev="dm-0" ino=4520247 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:realmd_var_lib_t:s0 tclass=file
type=AVC msg=audit(1496865224.775:130): avc: denied { read } for pid=2584 comm="kinit" name="kdc-ca-bundle.pem" dev="dm-0" ino=4520247 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:realmd_var_lib_t:s0 tclass=file
----
time->Wed Jun 7 14:53:44 2017
type=SYSCALL msg=audit(1496865224.775:131): arch=c000003e syscall=5 success=yes exit=0 a0=3 a1=7fff570e49d0 a2=7fff570e49d0 a3=0 items=0 ppid=2515 pid=2584 auid=4294967295 uid=387 gid=387 euid=387 suid=387 fsuid=387 egid=387 sgid=387 fsgid=387 tty=(none) ses=4294967295 comm="kinit" exe="/usr/bin/kinit" subj=system_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(1496865224.775:131): avc: denied { getattr } for pid=2584 comm="kinit" path="/var/lib/ipa-client/pki/kdc-ca-bundle.pem" dev="dm-0" ino=4520247 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:realmd_var_lib_t:s0 tclass=file
Seems like these policies slipped between our fingers, I already reported these denials in the massive IPA SELinux bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1436689#c109 IPA-server: ipa-server-4.5.0-16.el7.x86_64 SELINUX: selinux-policy-3.13.1-161.el7.noarch Verified the bug for WEBUI login to ipa server after upgrade (in my case RHEL 7.3.z > 7.4), the login is now successful. Refer attached screenshot. Created attachment 1287162 [details]
Login-success-after-upgrade
Login-success-after-upgrade
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2304 |