Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1350397

Summary: OpenJDK-1.8 missing in required packages
Product: [oVirt] ovirt-engine Reporter: Andreas Bleischwitz <ableisch>
Component: Packaging.rpmAssignee: Yedidyah Bar David <didi>
Status: CLOSED CURRENTRELEASE QA Contact: Jiri Belka <jbelka>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.0.0CC: ableisch, bugs, didi, lsvaty, sbonazzo, ylavi
Target Milestone: ovirt-4.0.2Flags: rule-engine: ovirt-4.0.z+
rule-engine: blocker+
ylavi: planning_ack+
sbonazzo: devel_ack+
gklein: testing_ack+
Target Release: 4.0.2.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The 3.6 engine could use either java 1.7 or 1.8. 4.0 requires 1.8. engine-setup checks for correct java version at validation stage, prior to upgrading packages. Consequence: Upgrade from 3.6 that had only java 1.7 installed fails. Fix: The engine-specific plugin package of engine-setup was made to require java 1.8 Result: java 1.8 will be pulled in by upgrading the setup packages and engine-setup will find it and not fail.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-12 14:24:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Andreas Bleischwitz 2016-06-27 10:42:33 UTC 
Description of problem:
Upgrade from oVirt-3.6.6 to 4.0.0 fails because of non-installed java-1.8.0-openjdk.x86_64.

Version-Release number of selected component (if applicable):
ovirt-engine-setup-base-4.0.0.6-1.el7.centos.noarch

How reproducible:
Install oVirt-3.6.6 and follow the description to upgrade.

Steps to Reproduce:
1. Install oVirt-3.6.6
2. yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release40.rpm
3. yum update "ovirt-engine-setup*"
4. engine-setup
5. get following execption

Actual results:
/var/log/ovirt-engine/setup/ovirt-engine-setup.log:

2016-06-27 09:01:09 DEBUG otopi.context context._executeMethod:128 Stage validation METHOD otopi.plugins.ovirt_engine_common.ovirt_engine_common.config.java.Plugin._validation
2016-06-27 09:01:10 DEBUG otopi.context context._executeMethod:142 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/ovirt-engine-common/config/java.py", line 67, in _validation
    ] = java.Java().getJavaHome()
  File "/usr/lib/python2.7/site-packages/ovirt_engine/java.py", line 58, in getJavaHome
    error='(%s)' % stderr if stderr else '',
RuntimeError: Cannot get JAVA_HOME make sure supported JRE is installed
2016-06-27 09:01:10 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Setup validation': Cannot get JAVA_HOME make sure supported JRE is installed
2016-06-27 09:01:10 DEBUG otopi.context context.dumpEnvironment:760 ENVIRONMENT DUMP - BEGIN
2016-06-27 09:01:10 DEBUG otopi.context context.dumpEnvironment:770 ENV BASE/error=bool:'True'
2016-06-27 09:01:10 DEBUG otopi.context context.dumpEnvironment:770 ENV BASE/exceptionInfo=list:'[(<type 'exceptions.RuntimeError'>, RuntimeError('Cannot get JAVA_HOME make sure supported JRE is installed',), <traceback ob
ject at 0x6975098>)]'


Expected results:
Installation/upgrade without errors

Additional info:
After "yum -y install java-1.8.0-openjdk.x86_64" engine-setup ran through without error.
Comment 1 Yedidyah Bar David 2016-06-27 12:19:38 UTC 
Please provide output of these commands:

ls -l /usr/lib/jvm
grep java /var/log/yum.log*
rpm -qa | grep java
for jre in /usr/lib/jvm/*; do echo; echo Checking JRE "${jre}"; rpm -qf "${jre}"; "${jre}/bin/java" -version; done
Comment 2 Andreas Bleischwitz 2016-06-27 12:29:27 UTC 
(In reply to Yedidyah Bar David from comment #1)
> Please provide output of these commands:
> 
See the posts between afer your requests.

> ls -l /usr/lib/jvm
[root@ovirtmgr ~]# ls -l /usr/lib/jvm
total 16
lrwxrwxrwx. 1 root root   26 Apr 22 15:45 java -> /etc/alternatives/java_sdk
lrwxrwxrwx. 1 root root   32 Apr 22 15:45 java-1.7.0 -> /etc/alternatives/java_sdk_1.7.0
lrwxrwxrwx. 1 root root   40 Apr 22 15:45 java-1.7.0-openjdk -> /etc/alternatives/java_sdk_1.7.0_openjdk
drwxr-xr-x. 8 root root 4096 Apr 22 15:45 java-1.7.0-openjdk-1.7.0.101-2.6.6.1.el7_2.x86_64
drwxr-xr-x. 3 root root 4096 Mar 28 11:51 java-1.7.0-openjdk-1.7.0.95-2.6.4.0.el7_2.x86_64
drwxr-xr-x. 2 root root 4096 Apr 22 15:45 java-1.7.0-openjdk-1.7.0.99-2.6.5.0.el7_2.x86_64
drwxr-xr-x. 3 root root 4096 Jun 27 10:27 java-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64
lrwxrwxrwx. 1 root root   34 Apr 22 15:45 java-openjdk -> /etc/alternatives/java_sdk_openjdk
lrwxrwxrwx. 1 root root   21 Jun 27 10:28 jre -> /etc/alternatives/jre
lrwxrwxrwx. 1 root root   27 Apr 22 15:45 jre-1.7.0 -> /etc/alternatives/jre_1.7.0
lrwxrwxrwx. 1 root root   35 Apr 22 15:45 jre-1.7.0-openjdk -> /etc/alternatives/jre_1.7.0_openjdk
lrwxrwxrwx. 1 root root   53 Apr 22 15:45 jre-1.7.0-openjdk-1.7.0.101-2.6.6.1.el7_2.x86_64 -> java-1.7.0-openjdk-1.7.0.101-2.6.6.1.el7_2.x86_64/jre
lrwxrwxrwx. 1 root root   27 Jun 27 10:27 jre-1.8.0 -> /etc/alternatives/jre_1.8.0
lrwxrwxrwx. 1 root root   35 Jun 27 10:27 jre-1.8.0-openjdk -> /etc/alternatives/jre_1.8.0_openjdk
lrwxrwxrwx. 1 root root   50 Jun 27 10:27 jre-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64 -> java-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64/jre
lrwxrwxrwx. 1 root root   29 Jun 27 10:27 jre-openjdk -> /etc/alternatives/jre_openjdk


> grep java /var/log/yum.log*
[root@ovirtmgr ~]# grep java /var/log/yum.log*
Mar 01 16:46:32 Installed: tzdata-java-2016a-1.el7.noarch
Mar 01 16:46:51 Installed: python-javapackages-3.4.1-11.el7.noarch
Mar 01 16:46:53 Installed: javapackages-tools-3.4.1-11.el7.noarch
Mar 01 17:28:27 Installed: 1:java-1.7.0-openjdk-1.7.0.95-2.6.4.0.el7_2.x86_64
Mar 01 17:29:56 Installed: 1:java-1.7.0-openjdk-headless-1.7.0.95-2.6.4.0.el7_2.x86_64
Mar 01 17:31:37 Installed: 1:java-1.8.0-openjdk-headless-1.8.0.71-2.b15.el7_2.x86_64
Mar 01 17:32:08 Installed: 1:java-1.7.0-openjdk-devel-1.7.0.95-2.6.4.0.el7_2.x86_64
Mar 01 17:32:15 Installed: javamail-1.4.6-8.el7.noarch
Mar 01 17:32:21 Installed: otopi-java-1.4.1-1.el7.centos.noarch
Mar 01 17:32:21 Installed: ovirt-host-deploy-java-1.4.1-1.el7.centos.noarch
Mar 01 17:32:55 Installed: javassist-3.16.1-10.el7.noarch
Mar 01 17:32:56 Installed: vdsm-jsonrpc-java-1.1.9-1.el7.centos.noarch
Mar 01 17:46:01 Installed: 1:java-1.8.0-openjdk-1.8.0.71-2.b15.el7_2.x86_64
Mar 28 11:51:16 Updated: tzdata-java-2016c-1.el7.noarch
Mar 28 11:51:25 Updated: 1:java-1.7.0-openjdk-headless-1.7.0.99-2.6.5.0.el7_2.x86_64
Mar 28 11:51:25 Updated: 1:java-1.7.0-openjdk-1.7.0.99-2.6.5.0.el7_2.x86_64
Mar 28 11:51:30 Updated: 1:java-1.8.0-openjdk-headless-1.8.0.77-0.b03.el7_2.x86_64
Mar 28 11:51:30 Updated: 1:java-1.8.0-openjdk-1.8.0.77-0.b03.el7_2.x86_64
Mar 28 11:51:32 Updated: 1:java-1.7.0-openjdk-devel-1.7.0.99-2.6.5.0.el7_2.x86_64
Mar 28 17:00:53 Erased: 1:java-1.8.0-openjdk-1.8.0.77-0.b03.el7_2.x86_64
Mar 28 17:00:53 Erased: 1:java-1.8.0-openjdk-headless-1.8.0.77-0.b03.el7_2.x86_64
Apr 22 15:45:30 Updated: 1:java-1.7.0-openjdk-1.7.0.101-2.6.6.1.el7_2.x86_64
Apr 22 15:45:35 Updated: 1:java-1.7.0-openjdk-headless-1.7.0.101-2.6.6.1.el7_2.x86_64
Apr 22 15:45:37 Updated: 1:java-1.7.0-openjdk-devel-1.7.0.101-2.6.6.1.el7_2.x86_64
Apr 27 08:15:49 Updated: tzdata-java-2016d-1.el7.noarch
May 30 17:01:07 Updated: vdsm-jsonrpc-java-1.1.12-1.el7.centos.noarch
Jun 27 08:18:18 Updated: tzdata-java-2016e-1.el7.noarch
Jun 27 08:52:29 Updated: otopi-java-1.5.0-1.el7.centos.noarch
Jun 27 10:27:12 Installed: 1:java-1.8.0-openjdk-headless-1.8.0.91-1.b14.el7_2.x86_64
Jun 27 10:27:12 Installed: 1:java-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64
Jun 27 10:31:17 Updated: vdsm-jsonrpc-java-1.2.3-1.el7.centos.noarch
Jun 27 10:31:20 Updated: ovirt-host-deploy-java-1.5.0-1.el7.centos.noarch
Jun 27 10:31:20 Installed: xz-java-1.3-3.el7.noarch

> rpm -qa | grep java
[root@ovirtmgr ~]# rpm -qa | grep java
java-1.8.0-openjdk-headless-1.8.0.91-1.b14.el7_2.x86_64
java-1.7.0-openjdk-headless-1.7.0.101-2.6.6.1.el7_2.x86_64
ovirt-host-deploy-java-1.5.0-1.el7.centos.noarch
vdsm-jsonrpc-java-1.2.3-1.el7.centos.noarch
javamail-1.4.6-8.el7.noarch
python-javapackages-3.4.1-11.el7.noarch
java-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64
java-1.7.0-openjdk-1.7.0.101-2.6.6.1.el7_2.x86_64
xz-java-1.3-3.el7.noarch
javassist-3.16.1-10.el7.noarch
javapackages-tools-3.4.1-11.el7.noarch
tzdata-java-2016e-1.el7.noarch
otopi-java-1.5.0-1.el7.centos.noarch
java-1.7.0-openjdk-devel-1.7.0.101-2.6.6.1.el7_2.x86_64

> for jre in /usr/lib/jvm/*; do echo; echo Checking JRE "${jre}"; rpm -qf
> "${jre}"; "${jre}/bin/java" -version; done
[root@ovirtmgr ~]# for jre in /usr/lib/jvm/*; do echo; echo Checking JRE "${jre}"; rpm -qf "${jre}"; "${jre}/bin/java" -version; done

Checking JRE /usr/lib/jvm/java
file /usr/lib/jvm/java is not owned by any package
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/java-1.7.0
file /usr/lib/jvm/java-1.7.0 is not owned by any package
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/java-1.7.0-openjdk
file /usr/lib/jvm/java-1.7.0-openjdk is not owned by any package
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.101-2.6.6.1.el7_2.x86_64
java-1.7.0-openjdk-headless-1.7.0.101-2.6.6.1.el7_2.x86_64
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.95-2.6.4.0.el7_2.x86_64
file /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.95-2.6.4.0.el7_2.x86_64 is not owned by any package
-bash: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.95-2.6.4.0.el7_2.x86_64/bin/java: No such file or directory

Checking JRE /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.99-2.6.5.0.el7_2.x86_64
file /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.99-2.6.5.0.el7_2.x86_64 is not owned by any package
-bash: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.99-2.6.5.0.el7_2.x86_64/bin/java: No such file or directory

Checking JRE /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64
java-1.8.0-openjdk-headless-1.8.0.91-1.b14.el7_2.x86_64
-bash: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64/bin/java: No such file or directory

Checking JRE /usr/lib/jvm/java-openjdk
file /usr/lib/jvm/java-openjdk is not owned by any package
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/jre
file /usr/lib/jvm/jre is not owned by any package
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

Checking JRE /usr/lib/jvm/jre-1.7.0
file /usr/lib/jvm/jre-1.7.0 is not owned by any package
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/jre-1.7.0-openjdk
file /usr/lib/jvm/jre-1.7.0-openjdk is not owned by any package
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/jre-1.7.0-openjdk-1.7.0.101-2.6.6.1.el7_2.x86_64
java-1.7.0-openjdk-headless-1.7.0.101-2.6.6.1.el7_2.x86_64
java version "1.7.0_101"
OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

Checking JRE /usr/lib/jvm/jre-1.8.0
file /usr/lib/jvm/jre-1.8.0 is not owned by any package
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

Checking JRE /usr/lib/jvm/jre-1.8.0-openjdk
file /usr/lib/jvm/jre-1.8.0-openjdk is not owned by any package
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

Checking JRE /usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64
java-1.8.0-openjdk-headless-1.8.0.91-1.b14.el7_2.x86_64
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

Checking JRE /usr/lib/jvm/jre-openjdk
file /usr/lib/jvm/jre-openjdk is not owned by any package
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
Comment 3 Yedidyah Bar David 2016-06-27 13:35:08 UTC 
OK, managed to reproduce:

1. Install ovirt 3.6 and openjdk 1.7.
Note that 3.6 can use either 1.7 or 1.8, and yum might default to 1.8 if not specified.

2. Add 4.0 repos, update setup packages
This does not touch java

3. engine-setup
Will prompt:
          Setup has found updates for some packages:
...
          PACKAGE: [install] 1:java-1.8.0-openjdk-1.8.0.91-1.b14.el7_2.x86_64
...
but later will fail, as java 1.8 is still not installed at that point.

Workaround already mentioned above by Andreas - install openjdk 1.8 prior to engine-setup.

Not sure how we are going to handle this bug. Two main options:

1. Add 1.8 as requirement of ovirt-engine-setup-plugin-ovirt-engine, or even ovirt-engine-setup-plugin-ovirt-engine-common (because the setup code checking the version is there)
Pro: Simplest
Con: Not really needed, if user does not intend to install/setup the engine but wants to setup only some other components, e.g. websocket-proxy.

2. Drop the check, if we see that the correct java version is going to be installed as a requirement of the updated engine.
Pro: Needs less disk space/bandwidth if engine is not set up
Con: More complex, requires making sure the engine-setup code that runs before installing packages does not need the correct version of java and that this correct version is set after installing the package.

Sandro - what do you think? Please also ack/flag. Thanks.
Comment 4 Sandro Bonazzola 2016-07-19 08:51:21 UTC 
Can't we check for java after package transaction?
Comment 6 Yedidyah Bar David 2016-07-20 06:20:48 UTC 
(In reply to Sandro Bonazzola from comment #4)
> Can't we check for java after package transaction?

Of course, but the whole point of testing in validation stage is that we (almost) do not touch the system until that stage, so if we fail there, no need to rollback. We can install Java at STAGE_INTERNAL_PACKAGES instead, but that's ugly too imo.
Comment 7 Sandro Bonazzola 2016-07-20 06:34:47 UTC 
(In reply to Yedidyah Bar David from comment #6)
> (In reply to Sandro Bonazzola from comment #4)
> > Can't we check for java after package transaction?
> 
> Of course, but the whole point of testing in validation stage is that we
> (almost) do not touch the system until that stage, so if we fail there, no
> need to rollback. We can install Java at STAGE_INTERNAL_PACKAGES instead,
> but that's ugly too imo.

Agreed. I'm ok with adding the dependency to ovirt-engine-setup-plugin-ovirt-engine-common; if you don't want to setup the engine you shouldn't install this package anyway since it should not be needed for other services like websocket-proxy.
Comment 9 Sandro Bonazzola 2016-07-25 12:46:45 UTC 
Note that bug #1320556 has been retargeted to 4.1.
Can we drop it as blocker for this one and move to QE?
Comment 10 Sandro Bonazzola 2016-07-25 12:47:19 UTC 
(In reply to Sandro Bonazzola from comment #9)
> Note that bug #1320556 has been retargeted to 4.1.
> Can we drop it as blocker for this one and move to QE?

sorry, wrong bug.
Comment 11 Jiri Belka 2016-07-29 08:29:54 UTC 
ok

# rpm -q ovirt-engine ; rpm -qR ovirt-engine | grep java
ovirt-engine-4.0.2.1-0.1.el7ev.noarch
java-openjdk >= 1:1.8.0