1374871 – Swift with Ceilometer integration doesn't work due to broken dependency ( python-pycadf-1.1.0-1.el7ost needs pytz>=2013.6 for python-ceilometerclient-1.5.2-1.el7ost ,but pytz is 2012d ).

Bug 1374871 - Swift with Ceilometer integration doesn't work due to broken dependency ( python-pycadf-1.1.0-1.el7ost needs pytz>=2013.6 for python-ceilometerclient-1.5.2-1.el7ost ,but pytz is 2012d ).

Summary: Swift with Ceilometer integration doesn't work due to broken dependency ( pyt...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	pytz
Sub Component:
Version:	7.2
Hardware:	Unspecified
OS:	Unspecified
Priority:	urgent
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Charalampos Stratakis
QA Contact:	Vaclav Danek
Docs Contact:
URL:
Whiteboard:	Triaged
Depends On:	1284058
Blocks:	1298243 1380359 1393868 1400961
TreeView+	depends on / blocked

Reported:	2016-09-09 22:21 UTC by Masaki Furuta ( RH )
Modified:	2019-12-16 06:42 UTC (History)
CC List:	16 users (show)
Fixed In Version:	pytz-2016.10-1.el7
Doc Type:	No Doc Update
Doc Text:	undefined
Clone Of:	1284058
Environment:
Last Closed:	2017-08-01 20:30:53 UTC
Target Upstream Version:

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2017:1915	0	normal	SHIPPED_LIVE	pytz bug fix update	2017-08-01 18:26:25 UTC

Description Masaki Furuta ( RH ) 2016-09-09 22:21:05 UTC

+++ This bug was initially created as a clone of Bug #1284058 +++

Description of problem:

On a director deployed swift we are using deprecated ceilometer
middleware that has been dropped in liberty.

We have (in /etc/swift/proxy-server.conf)
[filter:ceilometer]
use = egg:ceilometer#swift

we should be using the middleware provided by the python-ceilometermiddleware package (ceilometermiddleware.swift:filter_factory)

This should be changed in OSP-8 and ideally before (to simplify upgrades) 


upstream we dropped the middleware from the pipeline because the package wasn't available at the time
https://review.openstack.org/#/c/197933/1

now that the package is available for python-ceilometermiddleware we should use it instead.

Version-Release number of selected component (if applicable):
openstack-puppet-modules-2015.1.8-29.el7ost.noarch
openstack-ceilometer-common-2015.1.2-1.el7ost.noarch
openstack-tripleo-heat-templates-0.8.6-78.el7ost.noarch

--- Additional comment from Red Hat Bugzilla Rules Engine on 2015-11-21 01:44:19 JST ---

Since this issue was entered in bugzilla without a release flag set, rhos-8.0? has been automatically added to ensure that it is properly evaluated for this release.

--- Additional comment from John Skeoch on 2016-02-01 11:45:29 JST ---

User calfonso's account has been closed

--- Additional comment from Mike Burns on 2016-04-08 05:57:01 JST ---

This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

--- Additional comment from John Skeoch on 2016-04-18 17:29:06 JST ---

User yeylon's account has been closed

--- Additional comment from Sachin on 2016-09-09 18:54:38 JST ---

This is the reason why following meters are not displayed/available?

- storage.objects.outgoing.bytes
- storage.objects.incoming.bytes


--
Sachin

--- Additional comment from Sachin on 2016-09-09 19:01:52 JST ---

@Hugh, I don't see much activity, are you still working on this bug?

--- Additional comment from Masaki Furuta on 2016-09-10 07:10:18 JST ---

 === In Red Hat Customer Portal Case 01696992 ===
--- Comment by Furuta, Masaki on 9/10/2016 7:01 AM ---

_internal_

For OSP8, 

As we discussed on IRC, we don't have entry point for swift in ceilometer but in ceilometerclient.
But ceilometermiddleware is currently broken due to pytz requirements and we cannot run it in supported method.


osp7: we can use 'use = egg:ceilometer#swift' and 'paste.filter_factory = ceilometer.objectstore.swift_middleware:filter_factory'. in [filter:ceilometer] section, and it works. ( currently 'use = egg:ceilometermiddleware#swift' and 'paste.filter_factory = ceilometermiddleware.swift:filter_factory' does not work due to "swift-proxy-server: pkg_resources.VersionConflict: (pytz 2012d (/usr/lib/python2.7/site-packages), Requirement.parse('pytz>=2013.6'))" error unfortunately)

  # grep -FrIA1 '[paste.filter_factory]' /usr/lib/python2.7/site-packages/ceilometer*
  /usr/lib/python2.7/site-packages/ceilometer-2015.1.3-py2.7.egg-info/entry_points.txt:[paste.filter_factory]
  /usr/lib/python2.7/site-packages/ceilometer-2015.1.3-py2.7.egg-info/entry_points.txt-swift = ceilometer.objectstore.swift_middleware:filter_factory
  --
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.1.0-py2.7.egg-info/entry_points.txt:[paste.filter_factory]
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.1.0-py2.7.egg-info/entry_points.txt-swift = ceilometermiddleware.swift:filter_factory

  # cat /usr/lib/python2.7/site-packages/pycadf-0.8.0-py2.7.egg-info/requires.txt
  Babel>=1.3
  oslo.config>=1.6.0  # Apache-2.0
  oslo.context>=0.1.0                     # Apache-2.0
  oslo.i18n>=1.3.0  # Apache-2.0
  oslo.serialization>=1.2.0               # Apache-2.0
  pytz>=2013.6
  six>=1.7.0


osp8: we can use 'use = egg:ceilometermiddleware#swift' and 'paste.filter_factory = ceilometermiddleware.swift:filter_factory', which does not work due to "swift-proxy-server: pkg_resources.VersionConflict: (pytz 2012d (/usr/lib/python2.7/site-packages), Requirement.parse('pytz>=2013.6'))" error unfortunately.

  # grep -FrIA1 '[paste.filter_factory]' /usr/lib/python2.7/site-packages/ceilometer*
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.3.0-py2.7.egg-info/entry_points.txt:[paste.filter_factory]
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.3.0-py2.7.egg-info/entry_points.txt-swift = ceilometermiddleware.swift:filter_factory

  # cat /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt
  oslo.config>=2.1.0
  oslo.serialization>=1.4.0
  pytz>=2013.6


**Private note -- please do NOT disclose this to customer **

As we discussed on IRC, I just tried to hack version requirement and test out of my curiosity (not to answer to customer), it seemed to work as follows.
But anyway we cannot provide this to customer and we have to wait for fix (osp10 ??)...

~~~
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# diff -u /etc/ceilometer/ceilometer.conf-2016-09-01  /etc/ceilometer/ceilometer.conf;diff -u /etc/swift/proxy-server.conf-2016-09-09  /etc/swift/proxy-server.conf
--- /etc/ceilometer/ceilometer.conf-2016-09-01	2016-09-10 06:52:19.535466816 +0900
+++ /etc/ceilometer/ceilometer.conf	2016-09-10 06:52:33.634539475 +0900
@@ -996,7 +996,8 @@
 # Save event details. (boolean value)
 # Deprecated group/name - [collector]/store_events
 #store_events = false
-store_events = False
+store_events = True
+disable_non_metric_meters = False
 
 # WARNING: Ceilometer historically offered the ability to store events
 # as meters. This usage is NOT advised as it can flood the metering
--- /etc/swift/proxy-server.conf-2016-09-09	2016-09-09 19:18:43.087329951 +0900
+++ /etc/swift/proxy-server.conf	2016-09-10 06:49:29.067588316 +0900
@@ -9,20 +9,21 @@
 user = swift
 log_name = proxy-server
 log_facility = LOG_LOCAL1
-log_level = INFO
+log_level = DEBUG
 log_headers = False
 log_address = /dev/log
 
 
 
 [pipeline:main]
-pipeline = catch_errors bulk healthcheck cache crossdomain ratelimit authtoken keystone staticweb tempurl slo formpost account_quotas container_quotas proxy-server
+pipeline = catch_errors bulk healthcheck cache crossdomain ratelimit authtoken keystone staticweb tempurl slo formpost account_quotas container_quotas ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer  ceilometer proxy-server
+
 
 [app:proxy-server]
 use = egg:swift#proxy
 set log_name = proxy-server
 set log_facility = LOG_LOCAL1
-set log_level = INFO
+set log_level = DEBUG
 set log_address = /dev/log
 log_handoffs = true
 allow_account_management = true
@@ -104,3 +105,18 @@
 
 [filter:container_quotas]
 use = egg:swift#container_quotas
+
+
+[filter:ceilometer]
+use = egg:ceilometermiddleware#swift
+paste.filter_factory = ceilometermiddleware.swift:filter_factory
+metadata_headers = X-Client,X-Service,X-Host
+control_exchange = swift
+driver = messagingv2
+topic = notifications
+log_level = WARN
+url = rabbit://guest:guest.123.101:5672/
+
+[filter:proxy-logging]
+use = egg:swift#proxy_logging

[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# diff -u /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt-2016-09-09   /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt
--- /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt-2016-09-09	2016-09-09 21:02:41.401097809 +0900
+++ /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt	2016-09-10 06:49:29.118588579 +0900
@@ -1,4 +1,4 @@
 oslo.config>=2.1.0
 oslo.serialization>=1.4.0
-pytz>=2013.6
+pytz>=2012d


+ openstack-config --get /etc/swift/proxy-server.conf filter:ceilometer use
egg:ceilometermiddleware#swift
+ ceilometer sample-list -m storage.objects.incoming.bytes
+----------------------------------+--------------------------------+-------+-------------+------+----------------------------+
| Resource ID                      | Name                           | Type  | Volume      | Unit | Timestamp                  |
+----------------------------------+--------------------------------+-------+-------------+------+----------------------------+
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.490000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.471000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.454000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.437000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.421000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.400000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.383000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.369000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.350000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.325000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.306000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.110000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.086000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.066000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.048000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.020000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.994000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.977000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.957000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.941000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.923000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.904000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.811000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.800000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.793000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.787000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.783000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.777000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.773000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.768000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.764000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.753000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.747000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.583000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.570000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.553000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.538000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.385000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.366000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.353000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.338000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.386000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.385000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.383000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.381000 |
+----------------------------------+--------------------------------+-------+-------------+------+----------------------------+
+ ceilometer sample-list -m storage.objects.outgoing.bytes
+----------------------------------+--------------------------------+-------+--------+------+----------------------------+
| Resource ID                      | Name                           | Type  | Volume | Unit | Timestamp                  |
+----------------------------------+--------------------------------+-------+--------+------+----------------------------+
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.261000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.259000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.257000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.255000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.253000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.251000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.248000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.241000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.229000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.225000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.223000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.038000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.025000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.011000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.007000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.457000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.456000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.454000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.452000 |
+----------------------------------+--------------------------------+-------+--------+------+----------------------------+

[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# 
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -qf /etc/ceilometer/ceilometer.conf
openstack-ceilometer-common-5.0.2-2.el7ost.noarch
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -qf /etc/swift/proxy-server.conf
openstack-swift-proxy-2.5.0-2.el7ost.noarch
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -qf /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt
python-pycadf-1.1.0-1.el7ost.noarch
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -q python-ceilometerclient
python-ceilometerclient-1.5.2-1.el7ost.noarch
~~~

Comment 9 errata-xmlrpc 2017-08-01 20:30:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1915

Note You need to log in before you can comment on or make changes to this bug.