Bug 1549218

Summary: [Deployment] Docker healthcheck fails for OpenDaylight due to missing features loaded in ODL
Product: Red Hat OpenStack Reporter: Tim Rozet <trozet>
Component: openstack-tripleo-heat-templatesAssignee: Tim Rozet <trozet>
Status: CLOSED ERRATA QA Contact: Tomas Jamrisko <tjamrisk>
Severity: high Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: aadam, atelang, dfarrell, itbrown, jchhatba, jluhrsen, jschluet, mburns, mkolesni, nyechiel, rhel-osp-director-maint, rszmigie, tjamrisk, trozet, vorburger
Target Milestone: betaKeywords: Triaged
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: odl_deployment
Fixed In Version: openstack-tripleo-heat-templates-8.0.2-0.20180414062830.5f869f2.el7ost openstack-tripleo-common-8.6.1-0.20180410165748.4d8ca16.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
N/A
Last Closed: 2018-06-27 13:45:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Feature list none

Description Tim Rozet 2018-02-26 17:51:49 UTC
Description of problem:
Docker healthcheck works by checking for index.html in ODL.  However, netvirt-ui feature which offers this URL was removed from Oxygen build.  In order to fix the problem, we can load the dlux gui feature instead (odl-dlux-core).

Comment 1 jamo luhrsen 2018-02-26 23:47:55 UTC
(In reply to Tim Rozet from comment #0)
> Description of problem:
> Docker healthcheck works by checking for index.html in ODL.  However,
> netvirt-ui feature which offers this URL was removed from Oxygen build.  In
> order to fix the problem, we can load the dlux gui feature instead
> (odl-dlux-core).

Can we change how the docker healthcheck works?

If not, is there another way we can get the index.html to show up without
installing dlux?

that project is pretty stale and maybe slated for removal. It'd be nice
not to have to load it.

Comment 8 Tim Rozet 2018-03-08 17:17:19 UTC
Patch merged into upstream queens

Comment 9 jamo luhrsen 2018-03-19 21:54:48 UTC
There is a REST endpoint we can use that seems like a better option than
using the presence of that index.html file.

we can hit this:

curl -u "admin:admin" http://${ODL_IP}:8081/jolokia/read/org.opendaylight.infrautils:type=ready,name=SystemState

The response is something like this:

{"request":{"mbean":"org.opendaylight.infrautils:name=SystemState,type=ready","type":"read"},"value":{"SystemState":"ACTIVE"},"timestamp":1521496065,"status":200}

notice the SystemState:ACTIVE.

This would require the odl-infrautils-ready feature to be installed.

Comment 12 Tim Rozet 2018-03-22 13:36:51 UTC
Lowering severity on this as unhealthy containers do not fail deployments. See https://bugs.launchpad.net/tripleo/+bug/1752925

Comment 16 Itzik Brown 2018-04-09 07:28:15 UTC
Created attachment 1419132 [details]
Feature list

Comment 17 Itzik Brown 2018-04-09 07:30:54 UTC
Checked with openstack-tripleo-heat-templates-8.0.2-0.20180327213843.f25e2d8.el7ost.noarch

I don't see the feature odl-dlux-core (I attached the features list).
Also the container in unhealthy.

Comment 18 Janki 2018-04-09 08:17:18 UTC
I am seeing unhealthy status too. I suspect it has something to do with jolokia or some config somewhere.

$ curl http://10.10.125.12:8081/index.html
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /index.html. Reason:
<pre>    Unauthorized</pre></p>
</body>
</html>

"Unauthorised" reason is weird too.
 
 
$ curl -u admin:admin http://10.10.125.12:8081/index.html
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /index.html. Reason:
<pre>    Not Found</pre></p>
</body>
</html>

Comment 19 Janki 2018-04-09 08:20:04 UTC
(In reply to jamo luhrsen from comment #9)
> There is a REST endpoint we can use that seems like a better option than
> using the presence of that index.html file.
> 
> we can hit this:
> 
> curl -u "admin:admin"
> http://${ODL_IP}:8081/jolokia/read/org.opendaylight.infrautils:type=ready,
> name=SystemState
> 
> The response is something like this:
> 
> {"request":{"mbean":"org.opendaylight.infrautils:name=SystemState,
> type=ready","type":"read"},"value":{"SystemState":"ACTIVE"},"timestamp":
> 1521496065,"status":200}
> 
> notice the SystemState:ACTIVE.
> 
> This would require the odl-infrautils-ready feature to be installed.

We moved to using index.html because it doesnot need (atleast when it was implemented) credentails. TripleO can provide custom credentials and no proper way was found to get credentials from any file in ODL.

Comment 22 Michael Vorburger 2018-04-10 09:18:43 UTC
Agreed that re-packaging DLUX again is not the way we want to go IMHO... Let us instead first clarify what we really want to check for, and then change the health check to probe something better than that /index.html?

FYI that always was fairly useless, anyway (the fact that HTTP GET /index.html returns HTTP never really proved anything; just Karaf is running, but netvirt et al can be completely broken).

Comment 23 Mike Kolesnik 2018-04-10 11:58:09 UTC
Let's decide what should be the way to do this check.

For now I propose to curl the /controller/nb/v2/neutron path and see that it returns a non-error code (currently from what I see you get a 302 Found code).

We can consider better alternatives for future versions but I suggest we track it in a different BZ.

Comment 25 Michael Vorburger 2018-04-11 13:29:31 UTC
Agreed, lets use /controller/nb/v2/neutron for the short term, and improve it later in bug 1566080.

Comment 26 Mike Kolesnik 2018-04-15 06:20:05 UTC
All patches are merged upstream

Comment 27 Itzik Brown 2018-04-15 06:35:22 UTC
*** Bug 1549298 has been marked as a duplicate of this bug. ***

Comment 28 Jon Schlueter 2018-04-16 12:36:49 UTC
dropping master external trackers as they have all landed on stable/queens

Comment 32 Itzik Brown 2018-04-26 09:20:31 UTC
Checked with:
openstack-tripleo-heat-templates-8.0.2-0.20180416194362.29a5ad5.el7ost.noarch

Comment 33 Mike Kolesnik 2018-04-29 06:50:40 UTC
*** Bug 1570927 has been marked as a duplicate of this bug. ***

Comment 35 errata-xmlrpc 2018-06-27 13:45:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086