Bug 1181729 - Error 500 from JBOSS engine API
Summary: Error 500 from JBOSS engine API
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine-restapi
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 3.6.0
Assignee: Juan Hernández
QA Contact: Gonza
Whiteboard: integration
Depends On: 920126 1195123
TreeView+ depends on / blocked
Reported: 2015-01-13 16:48 UTC by Pavel Novotny
Modified: 2015-04-20 09:07 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of: 1168405
Last Closed: 2015-04-20 09:06:33 UTC
oVirt Team: ---
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
oVirt gerrit 35652 None None None Never
oVirt gerrit 35667 None None None Never

Description Pavel Novotny 2015-01-13 16:48:35 UTC
+++ This bug was initially created as a clone of Bug #1168405 +++

Description of problem:

When installing hosted engine on RHEVH3.5 over el7  with hosted engine based on Centos 7.

After running engine-setup on centos 7 , works produces 500 error

Version-Release number of selected component (if applicable):

Ovirt node 3.5
Centos 7
ovirt-engine 3.5

How reproducible:

100% (3 out of 3)

Steps to Reproduce:

Names and definitions:

    <hosted engine ip> please choose free ip to hold the engine.
    <hosted engine iname> the DNS name that will be connected to the mentioned before ip


    RHEVH install image with hosted engine plugin.
    Centos 7 install image Download - CentOS Wiki
    NFS server
    HTTP server
    Be familiar with using SSH to connect the RHEVH
    Be familiar with the F2 option in RHEVH (Rescue shell)

Setting the RHEVH

    Install RHEVH , if installing on virtual machine make sure nested virtualization is enabled and cpu configuration is copied from the hosting machine
    Setup networking and security pages to allow ssh connection
    Login using ssh into the RHEVH machine - ip can be found in the status page.
    Create local DNS resolving for engine if you dont have real DNS server.
        edit /etc/hosts
        add the  <hosted eng ip> <hosted engine name>  (for example my-engine.com )
    Or create real DNS entry in your DNS

Setting NFS server

    Install nfs server on the desired computer. (yum install nfs)
    Allow nfs , mountd , rpc-bind accss in you firewall
    start the nfs server and observe for errors.
    Using the /etc/exports file allow access for the network segment with the RHEVH and Engine will be located.
    systemctl start nfs
    in RHEVH console test the NFS server is available using showmount -e <nfs server ip>

Setting the http server

    Install the httpd server (  yum install httpd )
    Allow httpd in firewall.
    Upload the Centos7 iso into the httpd root directory.
    The httpd root directory can be found in /etc/httpd/conf/httpd.conf under DocumentRoot tag most likely (/var/www/html)
    systemctl start httpd
    in RHEVH console download some file from the httpd server using wget


Installing hosted engine in RHEVH

    In the hosted engine page provide the path to the iso located on the http server
    run the setup
    setup will fail on installation media
    re run setup
    using remote viewer login into <rhevh ip>:5900 , the password for the login is on the ssh console.


Setting up Centos 7 machine:

    Install the machine from iso.
    In the network tab please set the Ethernet connection , set the host name to <hosted engine ip> and the static ip to <hosted engine ip>

In RHEVH setup console

    Select 1 (continue setup - VM installation is complete)
    Make new RHEVH ssh session.
            hosted-engine --vm-start
            hosted-engine --add-console-password

Continue engine setup in Centos7 (The guest VM )

    Connect back to Centos machine like in previous step or by ssh connection
    Install net-tools
    and run ifconfig to verify <hosted engine ip>
    install ovirt repositories
        yum localinstall http://resources.ovirt.org/pub/yum-repo/ovirt-release35.rpm
        yum localinstall http://resources.ovirt.org/pub/yum-repo/ovirt-release35-snapshot.rpm
    yum install ovirt-engine
    run engine-setup as root when failed re run .
    From external pc please make sure you can login to the engine.

In RHEVH console with setup

Select 1 to complete install.

Actual results:
The hosted engine setup complains about 500 error , after some invetigation and puttin it the 
Error 500 was observed all logs attached.

Expected results:
Expected the setup to finish succefully

Additional info:

--- Additional comment from Anatoly Litovsky on 2014-11-26 21:22:36 CET ---

--- Additional comment from Anatoly Litovsky on 2014-11-26 21:23:16 CET ---

--- Additional comment from Anatoly Litovsky on 2014-11-26 21:26:46 CET ---

--- Additional comment from Anatoly Litovsky on 2014-11-26 21:27:16 CET ---

--- Additional comment from Anatoly Litovsky on 2014-11-26 21:27:56 CET ---

--- Additional comment from Anatoly Litovsky on 2014-11-26 21:28:22 CET ---

--- Additional comment from Douglas Schilling Landgraf on 2014-11-26 21:57:47 CET ---

Hi Anatoly,

Could you please provided the engine.log, server.log and others on /var/log/ovirt-engine/ ?


--- Additional comment from Anatoly Litovsky on 2014-11-27 07:53:57 CET ---

--- Additional comment from Anatoly Litovsky on 2014-11-27 07:55:05 CET ---

--- Additional comment from Anatoly Litovsky on 2014-11-27 07:56:48 CET ---

The bug can also be reproduced by instaling ovirt-engine over plain centos7 and accesing it from http://<server>/ovirt-engine/api

If needed have one installed and can share it over net.

--- Additional comment from Juan Hernández on 2014-11-27 11:35:16 CET ---

This is the relevant error message from server.log:

Caused by: java.lang.ClassNotFoundException: org.apache.avalon.framework.configuration.ConfigurationException from [Module "org.apache.xmlgraphics.fop:main" from local module loader @516e9cbe (roots: /var/lib/ovirt-engine/jboss_runtime/modules/00-ovirt-engine-modules,/var/lib/ovirt-engine/jboss_runtime/modules/01-ovirt-engine-jboss-as-modules)]
	at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)
	at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)
	at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
	at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)
	... 31 more

It means that the Avalon Framework module isn't defined correctly. It isn't because in RHEL6 and CentOS 6 it contains a single .jar file, but in Fedora it contains two files. We tweaked the .spec to take care of this difference, but didn't take into account that this tweak doesn't work for RHEL7 or CentOS7.

To workaround the issue you can do the following:

  # cd /usr/share/ovirt-engine/modules/org/apache/avalon/framework/main
  # ln -sf /usr/share/java/avalon-framework-api.jar .
  # ln -sf /usr/share/java/avalon-framework-impl.jar .
  # systemctl restart ovirt-engine

--- Additional comment from Gonzalo Rafuls on 2015-01-06 01:02:37 CET ---

Just to be sure, is this issue only occuring when installing a hosted engine on RHEVH3.5 over el7 with a hosted engine based on Centos 7? Has anyone tried other scenarios?

Regarding comment 10, is this also happening when installing the engine over el7?

--- Additional comment from Juan Hernández on 2015-01-06 11:53:15 CET ---

This happened when installing the engine on EL7 (RHEL7 or Centos7), no matter if it is hosted engine or plain vanilla engine.

--- Additional comment from Pavel Novotny on 2015-01-12 15:39:38 CET ---

To verify this bug properly also in RHEVM, we need RHEL7 support for it, which is planned in RHEVM 3.6, according to RFE bug 920126.

Juan, is it possible to re-target this bug also to 3.6? Because until then QA cannot verify it.

--- Additional comment from Juan Hernández on 2015-01-12 15:42:17 CET ---

This is a question for Sandro, but my understanding is that this must be targeted for 3.5.0 because we want EL7 support in ovirt-engine 3.5.1.

--- Additional comment from Sandro Bonazzola on 2015-01-13 14:33:24 CET ---

Just to clarify, oVirt is meant to support EL7 in 3.5.1 using ovirt-engine-jboss-as.

We're not supposed to verify this bug in RHEV-M 3.5. RFE bug 920126 is not yet approved so it may not make it for 3.6.

Comment 1 Sandro Bonazzola 2015-04-15 13:06:29 UTC
.el7 support has been re-targeted to 4.0.

Comment 3 Juan Hernández 2015-04-16 08:08:12 UTC
Note that this is already fixed, but it can't be verified with RHEV-M until there is a EL 7 build, and that will not happen with 3.6.

Comment 4 Sandro Bonazzola 2015-04-20 09:06:33 UTC
Closing upstream.

Note You need to log in before you can comment on or make changes to this bug.