Bug 1465713 - Traceback info in curator pod's log
Summary: Traceback info in curator pod's log
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.7.0
Assignee: Rich Megginson
QA Contact: Junqi Zhao
URL: aos-scalability-36
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-28 01:35 UTC by Junqi Zhao
Modified: 2017-11-28 21:58 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-11-28 21:58:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
curator pods' log (3.86 KB, text/plain)
2017-06-28 01:35 UTC, Junqi Zhao
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:3188 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-29 02:34:54 UTC

Description Junqi Zhao 2017-06-28 01:35:29 UTC
Created attachment 1292538 [details]
curator pods' log

Description of problem:
Traceback info in curator pod's log
# oc logs logging-curator-1-12sq7
curator running [1] jobs
Traceback (most recent call last):
  File "/usr/bin/curator", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3084, in <module>
    @_call_aside
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3070, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3097, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 653, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 666, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 839, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'urllib3<2.0,>=1.8' distribution was not found and is required by elasticsearch

Version-Release number of selected component (if applicable):
Images from brew registry
logging-elasticsearch      v3.6                19ad6f8e4738        29 minutes ago      404.2 MB
logging-auth-proxy         v3.6                d94bddb3dcba        8 hours ago         214.8 MB
logging-kibana             v3.6                4eabc3acd717        21 hours ago        342.4 MB
logging-fluentd            v3.6                08e8a59602fe        21 hours ago        232.5 MB
logging-curator            v3.6                a0148dd96b8d        2 weeks ago         221.5 MB

How reproducible:
Always

Steps to Reproduce:
1. Deploy logging via ansible
2. Check curator pod's log
3.

Actual results:
Traceback info in curator pod's log

Expected results:
Should not be exception info in pod's log

Additional info:

Comment 1 Rich Megginson 2017-06-28 02:19:08 UTC
I just made a new build to check which packages are available:
2017-06-28 01:55:45,841 - atomic_reactor.plugins.orchestrate_build - INFO - x86_64 - 2017-06-28 01:55:45,826 - atomic_reactor.util - DEBUG - --> Processing Dependency: python-urllib3 for package: elastic-curator-3.5.0-2.el7.noarch
2017-06-28 01:55:45,898 - atomic_reactor.plugins.orchestrate_build - INFO - x86_64 - 2017-06-28 01:55:45,846 - atomic_reactor.util - DEBUG - ---> Package python2-urllib3.noarch 0:1.21-1.el7 will be installed
2017-06-28 01:55:46,350 - atomic_reactor.plugins.orchestrate_build - INFO - x86_64 - 2017-06-28 01:55:46,234 - atomic_reactor.util - DEBUG - python2-urllib3    noarch 1.21-1.el7          rhel-7-server-ose-3.6-rpms 172 k

This looks correct: python2-urllib3    noarch 1.21-1.el7

I'm not sure what could have happened.

Please try this image:
koji_builds:
  https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=569038
repositories:
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-curator:rhaos-3.6-rhel-7-docker-candidate-76378-20170628015440
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-curator:3.6.99
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-curator:3.6.99-3
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-curator:latest
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-curator:v3.6

Comment 2 Junqi Zhao 2017-06-28 04:23:46 UTC
curator pod's log still has Traceback info, same error with Comment 0

# python --version
Python 2.7.5

# docker images | grep logging
brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-curator         3.6.99-3            757cf4948069        2 hours ago         221.5 MB
brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-curator         v3.6                757cf4948069        2 hours ago         221.5 MB

Comment 3 Jason Montleon 2017-06-30 17:47:25 UTC
I think the package is reporting the wrong version.

>>> import urllib3
>>> print urllib3.__version__
dev


>>> import pkg_resources
>>> pkg_resources.get_distribution("urllib3").version
'dev'

instead of 1.21. Looking at the upstream it looks like it's the correct source commit, but the wrong branch (master vs. release). I should have ann updated package shortly.

Comment 7 Junqi Zhao 2017-07-03 23:49:33 UTC
Tested, no Traceback info in curator pod's log now.

# oc logs ${curator_pod}
curator running [1] jobs
No indices matched provided args: {'regex': None, 'index': (), 'suffix': None, 'newer_than': None, 'closed_only': False, 'prefix': None, 'time_unit': 'days', 'timestring': '%Y.%m.%d', 'exclude': ('.searchguard*', '.kibana*'), 'older_than': 30, 'all_indices': False}
curator run finish

# openshift version
openshift v3.6.132
kubernetes v1.6.1+5115d708d7
etcd 3.2.1

Curator images from brew registry
logging-curator         v3.6                b609245a72f9        17 hours ago        221.5 MB
logging-curator         v3.6.132-1          b609245a72f9        17 hours ago        221.5 MB

Comment 11 errata-xmlrpc 2017-11-28 21:58:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188


Note You need to log in before you can comment on or make changes to this bug.