Bug 1353999

Summary: CNFE when running thermostat web-storage-service with jetty 9.4.0.M0
Product: [Fedora] Fedora Reporter: Severin Gehwolf <sgehwolf>
Component: thermostatAssignee: Severin Gehwolf <sgehwolf>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 25CC: ebaron, eclipse-sig, java-sig-commits, jerboaa, krzysztof.daniel, mizdebsk, msimacek, omajid
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: thermostat-1.6.4-2.fc25 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-09 02:54:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Proposed fix which works for us. none

Description Severin Gehwolf 2016-07-08 16:32:45 UTC
Created attachment 1177703 [details]
Proposed fix which works for us.

Description of problem:
This is an upstream bug in upcoming jetty 9.4.0. Caused by this refactoring:
https://github.com/eclipse/jetty.project/commit/a311c8bde171551a57ff28c04851a3b53acfc1e0

Version-Release number of selected component (if applicable):
$ rpm -q jetty-server
jetty-server-9.4.0-0.2.M0.fc25.noarch
$ rpm -q thermostat
thermostat-1.6.0-1.fc25.x86_64

How reproducible:
100%

Steps to Reproduce:
1. dnf install thermostat-webapp (A mock setup will do too)
2. echo -e 'yes\nm\nm\nm\nc\nc\nc\na\na\na' | thermostat setup -c
3. thermostat web-storage-service

Actual results:
starting storage server...
server listening on ip: mongodb://127.0.0.1:27518
log file is here: /root/.thermostat-1.6/logs/db.log
pid: 8285
2016-07-08 15:01:52.035:INFO::Thread-3: Logging initialized @1374ms
2016-07-08 15:01:52.083:INFO:oejs.Server:Thread-3: jetty-9.4.0.M0
2016-07-08 15:01:52.083:WARN:oejs.Server:Thread-3: THIS IS NOT A STABLE RELEASE! DO NOT USE IN PRODUCTION!
2016-07-08 15:01:52.083:WARN:oejs.Server:Thread-3: Download a stable release from http://download.eclipse.org/jetty/
2016-07-08 15:01:52.092:WARN:oejw.WebAppContext:Thread-3: Failed startup of context o.e.j.w.WebAppContext@6cdce5ed{/thermostat/storage,null,null}{/usr/share/thermostat/webapp}
java.lang.ClassNotFoundException: org.eclipse.jetty.webapp.WebInfConfiguration
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.eclipse.jetty.util.Loader.loadClass(Loader.java:65)
        at org.eclipse.jetty.webapp.WebAppContext.loadConfigurations(WebAppContext.java:931)
        at org.eclipse.jetty.webapp.WebAppContext.preConfigure(WebAppContext.java:451)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:437)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:93)
        at org.eclipse.jetty.server.Server.doStart(Server.java:404)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at com.redhat.thermostat.web.endpoint.internal.JettyContainerLauncher.doStartContainerAndDeployWar(JettyContainerLauncher.java:214)
        at com.redhat.thermostat.web.endpoint.internal.JettyContainerLauncher.startContainerAndDeployWar(JettyContainerLauncher.java:125)
        at com.redhat.thermostat.web.endpoint.internal.JettyContainerLauncher.access$000(JettyContainerLauncher.java:77)
        at com.redhat.thermostat.web.endpoint.internal.JettyContainerLauncher$1.run(JettyContainerLauncher.java:99)
        at java.lang.Thread.run(Thread.java:745)
server shutdown complete: /root/.thermostat-1.6/data/db
log file is here: /root/.thermostat-1.6/logs/db.log
Failed to start embedded jetty instance

Expected results:
No ClassNotFoundException and successful startup.

Additional info:
This is likely only reproducible when jetty is being started within an OSGi environment. Such is the case for "thermostat web-storage-service".

Note that the code in Loader around line 65 looks like this:

    @SuppressWarnings("rawtypes")
    public static Class loadClass(String name)
        throws ClassNotFoundException
    {
        ClassLoader loader=Thread.currentThread().getContextClassLoader();
        return (loader==null ) ? Class.forName(name) : loader.loadClass(name);
    }

loader in this context points to the app class loader which booted up the OSGi framework. However, the OSGi bundle jetty-webapp was loaded as a bundle and thus, classes inside it will get loaded with the bundles class loader. Then TCCL won't see the relevant config classes in the webapp bundle whereas the bundle class loader would see it.

Comment 1 Severin Gehwolf 2016-07-08 16:54:00 UTC
Upstream issue:
https://github.com/eclipse/jetty.project/issues/705

Comment 2 Severin Gehwolf 2016-07-14 12:12:29 UTC
We've worked around this in thermostat-1.6.0-2.fc25

Comment 3 Jan Kurik 2016-07-26 04:46:21 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 25 development cycle.
Changing version to '25'.

Comment 4 Severin Gehwolf 2016-08-03 13:53:57 UTC
Upstream discussion convinced me that there is likely not much to be done on the jetty side. Re-assigning to thermostat component.

Comment 5 Fedora Update System 2016-09-19 16:33:44 UTC
thermostat-1.6.4-2.fc25 jgraphx-3.6.0.0-1.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-8e90151ad6

Comment 6 Fedora Update System 2016-09-20 19:57:36 UTC
jgraphx-3.6.0.0-1.fc25, thermostat-1.6.4-2.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-8e90151ad6

Comment 7 Fedora Update System 2016-10-09 02:54:24 UTC
jgraphx-3.6.0.0-1.fc25, thermostat-1.6.4-2.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.