Bug 809262

Summary: IPA Upgrade Web UI failure with internal server error
Product: Red Hat Enterprise Linux 6 Reporter: Scott Poore <spoore>
Component: ipaAssignee: Rob Crittenden <rcritten>
Status: CLOSED ERRATA QA Contact: IDM QE LIST <seceng-idm-qe-list>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.3CC: jgalipea, mkosek
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ipa-2.2.0-8.el6 Doc Type: Bug Fix
Doc Text:
No documentation needed.
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-20 13:26:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Scott Poore 2012-04-02 21:50:04 UTC
Description of problem:

After upgrading from 2.1.3-9 in RHEL6.2 to 2.2.0-5, Web UI shows an Internal Server Error after login.  This was also seen on 2.1.3-9 -> 2.2.0-5 -> 2.2.0-7.  This appears related (at least somewhat) to bug 783592.   The SELinux httpd_manage_ipa boolean appears to be set to off after the upgrade.  Setting it to true/on fixes the problem.

Version-Release number of selected component (if applicable):
ipa-server-2.2.0-5.el6.x86_64
selinux-policy-3.7.19-142.el6.noarch


How reproducible:
very.  

Steps to Reproduce:
1.  <setup IPA server on RHEL6.2>
2.  kinit admin
3.  <install firefox and xauth if necessary>
4.  firefox https://$MASTER/ipa/ui
5.  <follow steps to configure firefox for single sign-on to IPA>

  
Actual results:

IPA returns an Internal Server Error in the browser.

6.  setsebool httpd_manage_ipa=on
7.  <select retry in browser>

IPA returns expected user page

Expected results:

Should not need to manually turn on the httpd_manage_ipa boolean.

Additional info:

Can check this entirely from the command line with this:

kinit admin 

jsonfile=/tmp/jsoninput

echo '{"method":"user_find","params":[[],{"sizelimit":0,"pkey_only":true}]}' > $jsonfile

sessionid=$(curl -v --negotiate -u: https://$MASTER/ipa/session/login_kerberos --cacert /etc/ipa/ca.crt 2>&1 |grep ipa_session 2>&1|sed 's/^.*ipa_session=\([0-Z]*\).*$/\1/')

curl  -H "Content-Type:application/json" -H "Referer: https://$MASTER/ipa/xml" -H "Accept:application/json"  -H "Accept-Language:en" --cacert /etc/ipa/ca.crt -d  @$jsonfile -X POST -b "ipa_session=$sessionid; httponly; Path=/ipa; secure" https://$MASTER/ipa/session/json 2>&1|grep "dn.*uid="

It will return html for the user list or an Internal Server Error page depending on how httpd_manage_ipa is set.

This is what the failure looks like:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
 root@localhost and inform them of the time the error occurred,
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
<hr>
<address>Apache/2.2.15 (Red Hat) Server at storm.testrelm.com Port 443</address>
</body></html>

And we see AVCs:

# ausearch -m avc -ts 17:47
----
time->Mon Apr  2 17:48:01 2012
type=SYSCALL msg=audit(1333403281.476:373): arch=c000003e syscall=2 success=no exit=-13 a0=7f0ebc115310 a1=241 a2=1b6 a3=0 items=0 ppid=20402 pid=20540 auid=0 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=6 comm="httpd" exe="/usr/sbin/httpd" subj=unconfined_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(1333403281.476:373): avc:  denied  { write } for  pid=20540 comm="httpd" name="ipa_memcached" dev=dm-0 ino=394187 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:memcached_var_run_t:s0 tclass=dir

And here's the error_log traceback:

[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68] mod_wsgi (pid=20422): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68] Traceback (most recent call last):
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]   File "/usr/share/ipa/wsgi.py", line 49, in application
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]     return api.Backend.wsgi_dispatch(environ, s
tart_response)
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]   File "/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py", line 229, in __call__
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]     return self.route(environ, start_response)
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]   File "/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py", line 241, in route
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]     return app(environ, start_response)
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]   File "/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py", line 792, in __call__
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]     ipa_ccache_name = bind_ipa_ccache(ccache_data)
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]   File "/usr/lib/python2.6/site-packages/ipalib/session.py", line 1228, in bind_ipa_ccache
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68]     dst = open(name, 'w')
[Mon Apr 02 17:48:01 2012] [error] [client 10.16.96.68] IOError: [Errno 13] Permission denied: '/var/run/ipa_memcached/krbcc_20422'

Comment 2 Martin Kosek 2012-04-03 07:54:18 UTC
This SELinux boolean is set only for new installs. We need to set it for upgraded installs as well. I will open a ticket.

Comment 3 Martin Kosek 2012-04-03 07:54:50 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/2603

Comment 4 Rob Crittenden 2012-04-04 15:30:17 UTC
Fixed upstream

master: 17a0738d2d352f9c3d73167b3fb22cd566fd98d4

ipa-2-2: 56196b28085b346b86b43662a1ba7fdaf7a2454b

Comment 6 Scott Poore 2012-04-09 19:03:10 UTC
Verified.

Version :: ipa-server-2.2.0-8.el6.x86_64

Automated Test Results ::

Beaker job results not yet available but, manual run of automated test is:

# upgrade_bz_809262 

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: upgrade_bz_809262: IPA Upgrade Web UI failure with internal server error
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [13:59:45] ::  Machine in recipe is MASTER
:: [13:59:45] ::  Checking SELinux Boolean httpd_manage_ipa
:: [   PASS   ] :: SELinux Boolean httpd_manage_ipa is enabled
:: [13:59:46] ::  Checking Web UI
:: [13:59:46] ::  Prepare json query in file
:: [13:59:46] ::  Getting Session ID with:  curl -v --negotiate -u: https://spoore-dvm1.testrelm.com/ipa/session/login_kerberos --cacert /etc/ipa/ca.crt
:: [   PASS   ] :: Running 'curl  -H "Content-Type:application/json" -H "Referer: https://spoore-dvm1.testrelm.com/ipa/xml" -H "Accept:application/json"  -H "Accept-Language:en" --cacert /etc/ipa/ca.crt -d  @/tmp/jsoninput -X POST -b "ipa_session=871822b06caf17d6e3b5c75df1144dd7; httponly; Path=/ipa; secure" https://spoore-dvm1.testrelm.com/ipa/session/json > /tmp/errormsg.out 2>&1'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
120   776    0   776    0    69    543     48 --:--:--  0:00:01 --:--:--   513
{
    "error": null, 
    "id": null, 
    "principal": "admin", 
    "result": {
        "count": 3, 
        "result": [
            {
                "dn": "uid=admin,cn=users,cn=accounts,dc=testrelm,dc=com", 
                "uid": [
                    "admin"
                ]
            }, 
            {
                "dn": "uid=jack,cn=users,cn=accounts,dc=testrelm,dc=com", 
                "uid": [
                    "jack"
                ]
            }, 
            {
                "dn": "uid=jill,cn=users,cn=accounts,dc=testrelm,dc=com", 
                "uid": [
                    "jill"
                ]
            }
        ], 
        "summary": "3 users matched", 
        "truncated": false
    }, 
    "version": "2.1.90.rc1"
}:: [   PASS   ] :: Running 'cat /tmp/errormsg.out'
:: [13:59:48] ::  Checking /tmp/errormsg.out for "Internal Server Error"
:: [13:59:48] ::  Internal Server Error Not Found
:: [   PASS   ] :: BZ 809262 not found...WebUI did not return Internal Server 
Error
result_server not set, assuming developer mode.
Setting 192.168.122.101 to state upgrade_bz_809262.36
:: [   PASS   ] :: Running 'rhts-sync-set -s 'upgrade_bz_809262.36' -m 192.168.122.101'

#

Comment 8 Martin Kosek 2012-04-25 11:22:05 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
No documentation needed.

Comment 10 errata-xmlrpc 2012-06-20 13:26:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0819.html