Bug 1152236 - RHEV Admin Portal Unreachable After Upgrading to 3.4.0 due to mod_proxy_ajp not loading
Summary: RHEV Admin Portal Unreachable After Upgrading to 3.4.0 due to mod_proxy_ajp n...
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.4.0
Hardware: x86_64
OS: Linux
Target Milestone: ovirt-3.6.0-rc
: 3.6.0
Assignee: Yedidyah Bar David
QA Contact: Jiri Belka
Depends On:
TreeView+ depends on / blocked
Reported: 2014-10-13 16:33 UTC by Robert McSwain
Modified: 2019-09-12 08:02 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2016-03-09 20:41:09 UTC
oVirt Team: Integration
Target Upstream Version:
ylavi: Triaged+

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:0376 0 normal SHIPPED_LIVE Red Hat Enterprise Virtualization Manager 3.6.0 2016-03-10 01:20:52 UTC
oVirt gerrit 34232 0 master MERGED packaging: setup: Load mod_proxy_ajp.so if not loaded 2020-04-19 13:32:06 UTC

Description Robert McSwain 2014-10-13 16:33:26 UTC
Description of problem:
RHEV Admin Portal Unreachable After Upgrading to 3.4 due to mod_proxy_ajp not loading

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization 3.4 (immediately after running ovirt-engine-upgrade)

How reproducible:

Steps to Reproduce:
1. Upgrade to RHEV 3.4
2. Upgrade appears successful until attempting to visit Admin Portal
3. Browser returns: 
Not Found
The requested URL /ovirt-engine/ was not found on this server.

Actual results:
1. RHEV-M Admin Portal fails to start mod_proxy_ajp after upgrade, 
2. service httpd restart returns 
Starting httpd: Warning: DocumentRoot [/var/www/does/not/exist] does not exist

Expected results:
RHEV-M Admin Portal works as expected

Additional info:
Issue is resolved via the following workaround

In httpd config, Apache was unable to proxy pass the requests from :443 to ovirt-engine, simply because the proxy_ajp_module didn't load.

    <IfModule mod_proxy_ajp.c>
    LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

For some reason, ifModule mod_proxy_ajp.c failed, thus LoadModule was skipped. When the if clause is commented out, everything works.

Comment 9 Yedidyah Bar David 2014-10-20 10:24:05 UTC
To summarize an internal investigation:

1. The file /etc/httpd/conf.d/proxy_ajp.conf had bad content due to an internal error, and not due to any known (to me) bug in a released product (rhev or other).

2. I posted a fix to prevent such an error in future similar cases, although I am not sure we really need it. As always, we can't fix everything - if a user somehow unloaded the ajp proxy module, merely adding LoadModule might not be enough - e.g. the user can also remove the actual module file. nvm.

Comment 10 Yedidyah Bar David 2014-11-03 09:10:57 UTC
moving to Modified, as the patch was merged to upstream master. Can quite easily be cloned and backported to older releases, not that important imo.

Comment 11 Yaniv Lavi 2015-03-11 10:13:07 UTC
Can you please add a updated steps to reproduce ?

Comment 12 Yedidyah Bar David 2015-03-22 10:29:04 UTC
(In reply to Yaniv Dary from comment #11)
> Can you please add a updated steps to reproduce ?

Something like that should do:

1. On a clean system (el or fedora), yum install httpd

2. Find in /etc/httpd all lines with 'LoadModule proxy_ajp_module' and deactivate them (remove or comment out).

3. install and setup engine.

4. Try opening the web interface with a web browser.

To reproduce, install and setup a version without the fix (say, 3.5.z), it will fail.

To verify, install and setup a fixed version (current master/3.6+), it should work.

For extra points, find and disable mod_proxy_ajp.so (by renaming or removing it) and restart httpd. The fixed version will hopefully cause apache to write to the error log a line which will point you to our conf file, which will have somewhat-helpful comment.

Comment 13 Jiri Belka 2015-11-06 15:49:14 UTC
ok, rhevm-

so, if anything was broken while loading the module via other files it will be caught by "last" conf file - ie. _z_ -ovirt-engine-proxy.conf ('z-' to be loaded last) and this conf file tries its best to load the module...

# grep -vE '^[ \t]*#|^$' /etc/httpd/conf.d/z-ovirt-engine-proxy.conf 
<IfModule !proxy_ajp_module>
    LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
<IfModule proxy_ajp_module>
    <IfModule headers_module>
        RequestHeader unset Expect early
    <LocationMatch ^/(ovirt-engine($|/)|api($|/)|RHEVManagerWeb/|OvirtEngineWeb/|ca.crt$|engine.ssh.key.txt$|rhevm.ssh.key.txt$)>
        ProxyPassMatch ajp:// timeout=3600 retry=5
        <IfModule deflate_module>
            AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml

Comment 16 errata-xmlrpc 2016-03-09 20:41:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.