Bug 1625599

So far we've observed this only on glance_api docker logs (most of the messages) and some in cinder-api logs in /var/log/containers/httpd/cinder-api/*wsgi*error*log

There is some related work done upstream: https://bugs.launchpad.net/openstack-requirements/+bug/1765748

So basically, https://review.openstack.org/#/c/564883/ fixes the issue, right? It will be part of one of the next OSP14 builds, so this issue should disappear.

(In reply to Cyril Roelandt from comment #4)
> So basically, https://review.openstack.org/#/c/564883/ fixes the issue,
> right? It will be part of one of the next OSP14 builds, so this issue should
> disappear.

I believe so, at least for the glance which seems to be the main producer of the msgs.

(In reply to Cyril Roelandt from comment #4)
> So basically, https://review.openstack.org/#/c/564883/ fixes the issue,
> right? It will be part of one of the next OSP14 builds, so this issue should
> disappear.

I thought that code is already there:
()[glance@controller-0 /]$ python
Python 2.7.5 (default, May 31 2018, 09:41:32) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     from webob.acceptparse import AcceptLanguageValidHeader  # noqa
...     USING_WEBOB_1_8 = True
... except ImportError:
...     USING_WEBOB_1_8 = False
... 
>>> print USING_WEBOB_1_8
True
>>> 

So we are taking the def _best_match_language_1_8() branch of https://review.openstack.org/#/c/564883/1/glance/common/wsgi.py, but we are still logging a truckload of messages. Or am I looking at thing incorrectly?

(In reply to Michele Baldessari from comment #6)
> (In reply to Cyril Roelandt from comment #4)
> > So basically, https://review.openstack.org/#/c/564883/ fixes the issue,
> > right? It will be part of one of the next OSP14 builds, so this issue should
> > disappear.
> 
> I thought that code is already there:
> ()[glance@controller-0 /]$ python
> Python 2.7.5 (default, May 31 2018, 09:41:32) 
> [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> try:
> ...     from webob.acceptparse import AcceptLanguageValidHeader  # noqa
> ...     USING_WEBOB_1_8 = True
> ... except ImportError:
> ...     USING_WEBOB_1_8 = False
> ... 
> >>> print USING_WEBOB_1_8
> True
> >>> 
> 
> So we are taking the def _best_match_language_1_8() branch of
> https://review.openstack.org/#/c/564883/1/glance/common/wsgi.py, but we are
> still logging a truckload of messages. Or am I looking at thing incorrectly?

Michele is completely right about that, The code is already in downstream. It seem that there is one more usage of "old" best_match() method in:
    def best_match_content_type(self):
        """Determine the requested response content-type."""
        supported = ('application/json',)
        bm = self.accept.best_match(supported)
        return bm or 'application/json'
?

I just sent a patch upstream, let's see how this goes.

Upstream patch merged on master, proposed backport to Rocky.

Hey Mike,

I asked Steve on IRC, and it should be in z1 (2019-02-22.2).

I suspect no luck,
Just finished installing a "latest" which produced the z1 build date/puddle:

$ cat /etc/yum.repos.d/latest-installed 
14   -p 2019-02-22.2

However the resulting version is still below fixed-in
openstack-glance-17.0.1-0.20180924181829.afc0ceb.el7ost.noarch

Mike has lunched another "latest" deployment today (Fed 27),
resulting in same puddle as #24 above -> 14 -p 2019-02-22.2

openstack-glance-17.0.1-0.20180924181829.afc0ceb.el7ost.noarch < fixed-in

Still can't verify, waiting for newer puddle to recheck.

@Mike: can I ssh the machine?

@Mike: I managed to get to the Docker container this morning, but now I seem to be running into a network issue:

[root@titan34 ~]# ssh stack@undercloud-0
$ source overcloudrc
(overcloud) [stack@undercloud-0 ~]$ ssh -t heat-admin.24.13
ssh: connect to host 192.168.24.13 port 22: No route to host
$

Is there something wrong with the machine?

Thanks.

So, looking at the places where best_match is used:

> ()[glance@controller-0 /usr/lib/python2.7/site-packages]$ grep -rl "\.best_match("
> pkg_resources.py
> glance/common/wsgi.py
> wsme/protocol.py
> wsme/rest/protocol.py
> wsme/root.py
> jsonschema/tests/test_exceptions.py
> oslo_middleware/healthcheck/__init__.py
> webob/acceptparse.py


pkg_resources.py                            -> Unrelated
glance/common/wsgi.py                       -> fixed by a Glance patch
wsme/protocol.py                            -> Patch under review upstream
wsme/rest/protocol.py                       -> Same as above
wsme/root.py                                -> Same as above
jsonschema/tests/test_exceptions.py         -> Tests, should not be called
oslo_middleware/healthcheck/__init__.py     -> Patch under review upstream
webob/acceptparse.py                        -> Webob itself, we don't mind

We need to merge a bunch of patches upstream, backport them and get new puddles that include the fixes.

python-wsme python-wsme-0.9.4-0.20190508170502.bff9624.el7ost should contain the python-wsme fix.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1671