Bug 1009863 - Java update breaks ovirt-engine start
Summary: Java update breaks ovirt-engine start
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-installer
Version: 3.3
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.3.1
Assignee: Alon Bar-Lev
QA Contact:
URL:
Whiteboard: integration
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-19 10:51 UTC by Alexander Ludas
Modified: 2013-11-25 12:05 UTC (History)
8 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-11-25 12:05:47 UTC
oVirt Team: ---
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 19816 0 None None None Never
oVirt gerrit 19844 0 None None None Never

Description Alexander Ludas 2013-09-19 10:51:23 UTC
Description of problem:
After java update from java-1.7.0-openjdk-1.7.0.60-2.4.2.0.fc19.x86_64 to java-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64 ovirt-engine wasn't able to start.

ovirt-engine: ERROR run:485 Error: Directory '/usr/lib/jvm/jre-1.7.0-openjdk-1.7.0.60-2.4.2.0.fc19.x86_64' is required but missing
Version-Release number of selected component (if applicable):

Engine reads JAVA_HOME variable from /etc/ovirt-engine/engine.conf.d/10-setup-java.conf which is set to a different version and fails to start.

How reproducible:

Steps to Reproduce:
1. Install F19 with engine
2. Down- or upgrade java-1.7.0-openjdk java-1.7.0-openjdk-devel
3. Restart engine

Actual results:
Engine fails to start

Expected results:
Engine should start

Additional info:
Changing JAVA_HOME in /etc/ovirt-engine/engine.conf.d/10-setup-java.conf to '/etc/alternatives/jre' fixes the problem.

Comment 1 Juan Hernández 2013-09-20 13:44:33 UTC
Fedora 19 has introduced a new layout in the symlinks created in the /usr/lib/jvm directory. In previous versions of Fedora (and in RHEL and CentOS as well) it was populated with links like these:

  java-1.7.0-openjdk-1.7.0.60.x86_64
  jre -> /etc/alternatives/jre
  jre-1.7.0 -> /etc/alternatives/jre_1.7.0
  jre-1.7.0-openjdk.x86_64 -> java-1.7.0-openjdk-1.7.0.60.x86_64/jre
  jre-openjdk -> /etc/alternatives/jre_openjdk

The setup tool scans this directory, looks for links to real things (not to other links) and selects the one that contains a valid 1.7.0 OpenJDK. This resulted in a name which doesn't change when the package is updated: jre-1.7.0-openjdk.x86_64.

In Fedora 19 the directory is populated as follows (see [1] for details):

  java -> /etc/alternatives/java_sdk
  java-1.7.0 -> /etc/alternatives/java_sdk_1.7.0
  java-1.7.0-openjdk -> /etc/alternatives/java_sdk_1.7.0_openjdk
  java-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64
  java-1.7.0-openjdk.x86_64 -> /usr/lib/jvm/java-1.7.0-openjdk
  java-openjdk -> /etc/alternatives/java_sdk_openjdk
  jre -> /etc/alternatives/jre
  jre-1.7.0 -> /etc/alternatives/jre_1.7.0
  jre-1.7.0_openjdk -> /etc/alternatives/jre_1.7.0_openjdk
  jre-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64 -> java-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64/jre
  jre-openjdk -> /etc/alternatives/jre_openjdk

As you can see the symlink that points to the real installation directory does now contain the release number: jre-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64. This breaks our search algorithm.

Fortunately Fedora 19 also has included "alternatives" support for specific versions of the JRE/JDK:

  # alternatives --list | grep '^jre'
  jre_1.7.0	manual	/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64/jre
  jre_openjdk	manual	/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64/jre
  jre_1.7.0_openjdk	auto	/usr/lib/jvm/jre-1.7.0-openjdk-1.7.0.60-2.4.2.1.fc19.x86_64

So I would suggest to modify the setup tool so that (only for Fedora 19 and newer) it doesn't use the previous algorithm but hardcoders JAVA_HOME as /usr/lib/jvm/jre-1.7.0_openjdk.


[1] http://pkgs.fedoraproject.org/cgit/java-1.7.0-openjdk.git/commit/?id=0cfdf60a4a315c2e0618478c69f887638eaaf93b

Comment 2 Alon Bar-Lev 2013-09-20 17:48:29 UTC
Hi Juan,

What about using the alternatives as you implied above?

JAVA_HOME="$(alternatives --list | grep ^jre_1.7.0_openjdk | cut -f3)"

This will be somewhat like what I do in Gentoo:

JAVA_HOME="$(java-config --select-vm=icedtea-7 --jre-home)"

And avoid breakage when absolute name is modified again.

Thanks,
Alon

[1] http://gerrit.ovirt.org/#/c/7549/ [My, Aug 30, 2012]

Comment 3 Alon Bar-Lev 2013-09-20 18:18:02 UTC
Juan,

I also re-recommend not to store java home in configuration file per default and search for it during runtime unless explicitly available in configuration.

Alon

Comment 4 Juan Hernández 2013-09-23 08:59:25 UTC
Using the output of "alternatives --list" doesn't seem correct to me, as it the alternatives system is designed exactly to avoid using that information directly. Now that the alternatives system solves the version issue and provides an stable version specific JRE location I think we should use it (only for Fedora 19 or newer).

However you are the owner of the setup/startup scripts now, so do as you find appropriate.

Comment 5 Alon Bar-Lev 2013-09-23 09:19:07 UTC
Thanks.

This change of naming convention was expected, and I do not think it will be the last one.

I am going to move JAVA_HOME detection to runtime, unless explicitly specified in configuration.

There will be an option for distro specific script to return the location of java for product.

Comment 6 Itamar Heim 2013-09-23 10:01:24 UTC
didn't we at some point prefer openjdk to other JREs due to some bugs, etc.?

Comment 7 Alon Bar-Lev 2013-09-23 10:04:31 UTC
(In reply to Itamar Heim from comment #6)
> didn't we at some point prefer openjdk to other JREs due to some bugs, etc.?

yes, and it is not going to change.

the question is which openjdk should we use.

if there is an update we should use the latest.

while in current form we have the JAVA_HOME hardcoded during setup.

it used to work, but fedora changed the naming conventions of the directories.

this means that:

1. we should avoid hardcoding patches during setup.

2. we should be prepared for distro specific modifications.

Comment 8 Alon Bar-Lev 2013-10-03 00:24:01 UTC
commit e2f9b2dda8d81f5bda99d7533971cd9c3517d68a
Author: Alon Bar-Lev <alonbl>
Date:   Thu Oct 3 00:55:26 2013 +0300

    packaging: detect java home at runtime
    
    CURRENT IMPLEMENTATION
    
    detect java home during installation.
    
    advantage is optimize application execution post-setup.
    
    disadvantage is that java home may be altered by system updates.
    
    NEW IMPLEMENTATION
    
    detect java home during runtime using pre-defined algorithm.
    
    algorithm:
    
    execute /usr/share/ovirt-engine/bin/java-home.
    
    if java-home.local is available, execute that script. this enables
    distro specific customization.
    
    if OVIRT_ENGINE_JAVA_HOME environment is set, validate java validity
    and use/abort.
    
    search predefined selected alternatives location by order, first wins.
    
    search /usr/lib/jvm for compatible jvm.
    
    NOTES
    
    the java-home script resides with the lib package as it cannot reside
    within any version locked package since it is used by setup, and it
    cannot reside within setup since it used by post setup.
    
    Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1009863
    Change-Id: I833362d0498d534b06e0d7660f1e8a61e9b1713b
    Signed-off-by: Alon Bar-Lev <alonbl>

Comment 9 Pavel Stehlik 2013-10-31 19:28:06 UTC
Hi Alexander, could you please help us with testing this bug?
 Thank you, P.

Comment 10 Alexander Ludas 2013-10-31 21:57:42 UTC
No problem!

Did some basic testing based on 3.3.1 beta2 and everything works as expected.

Test procedure:
- install 3.3.1b2, run engine-setup
- downgrade openjdk(-devel) from 1.7.0.60-2.4.3.0.fc19.x86_64 to 1.7.0.25-2.3.10.3.fc19.x86_64 && restart ovirt-engine 
- engine-cleanup && engine-setup 
- upgrade java && restart ovirt-engine 

No problems at all.

Comment 11 Sandro Bonazzola 2013-11-07 08:52:17 UTC
Targeting to 3.3.2, it's not in 3.3.1 tree.

Comment 12 Sandro Bonazzola 2013-11-07 08:54:34 UTC
sorry, it's in 3.3.1, targeting to 3.3.1.

Comment 13 Sandro Bonazzola 2013-11-25 12:05:47 UTC
oVirt 3.3.1 has been released


Note You need to log in before you can comment on or make changes to this bug.