Bug 1374871

Summary:	Swift with Ceilometer integration doesn't work due to broken dependency ( python-pycadf-1.1.0-1.el7ost needs pytz>=2013.6 for python-ceilometerclient-1.5.2-1.el7ost ,but pytz is 2012d ).
Product:	Red Hat Enterprise Linux 7	Reporter:	Masaki Furuta ( RH ) <mfuruta>
Component:	pytz	Assignee:	Charalampos Stratakis <cstratak>
Status:	CLOSED ERRATA	QA Contact:	Vaclav Danek <vdanek>
Severity:	unspecified	Docs Contact:
Priority:	urgent
Version:	7.2	CC:	aschultz, bgollahe, cstratak, derekh, hbrock, jcoufal, jscotka, mburns, mfuruta, nkrishna, pgrist, pviktori, rhel-osp-director-maint, sacpatil, srevivo, vdanek
Target Milestone:	rc	Keywords:	Rebase
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	Triaged
Fixed In Version:	pytz-2016.10-1.el7	Doc Type:	No Doc Update
Doc Text:	undefined	Story Points:	---
Clone Of:	1284058	Environment:
Last Closed:	2017-08-01 20:30:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1284058
Bug Blocks:	1298243, 1380359, 1393868, 1400961

Description Masaki Furuta ( RH ) 2016-09-09 22:21:05 UTC

+++ This bug was initially created as a clone of Bug #1284058 +++

Description of problem:

On a director deployed swift we are using deprecated ceilometer
middleware that has been dropped in liberty.

We have (in /etc/swift/proxy-server.conf)
[filter:ceilometer]
use = egg:ceilometer#swift

we should be using the middleware provided by the python-ceilometermiddleware package (ceilometermiddleware.swift:filter_factory)

This should be changed in OSP-8 and ideally before (to simplify upgrades) 


upstream we dropped the middleware from the pipeline because the package wasn't available at the time
https://review.openstack.org/#/c/197933/1

now that the package is available for python-ceilometermiddleware we should use it instead.

Version-Release number of selected component (if applicable):
openstack-puppet-modules-2015.1.8-29.el7ost.noarch
openstack-ceilometer-common-2015.1.2-1.el7ost.noarch
openstack-tripleo-heat-templates-0.8.6-78.el7ost.noarch

--- Additional comment from Red Hat Bugzilla Rules Engine on 2015-11-21 01:44:19 JST ---

Since this issue was entered in bugzilla without a release flag set, rhos-8.0? has been automatically added to ensure that it is properly evaluated for this release.

--- Additional comment from John Skeoch on 2016-02-01 11:45:29 JST ---

User calfonso's account has been closed

--- Additional comment from Mike Burns on 2016-04-08 05:57:01 JST ---

This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

--- Additional comment from John Skeoch on 2016-04-18 17:29:06 JST ---

User yeylon's account has been closed

--- Additional comment from Sachin on 2016-09-09 18:54:38 JST ---

This is the reason why following meters are not displayed/available?

- storage.objects.outgoing.bytes
- storage.objects.incoming.bytes


--
Sachin

--- Additional comment from Sachin on 2016-09-09 19:01:52 JST ---

@Hugh, I don't see much activity, are you still working on this bug?

--- Additional comment from Masaki Furuta on 2016-09-10 07:10:18 JST ---

 === In Red Hat Customer Portal Case 01696992 ===
--- Comment by Furuta, Masaki on 9/10/2016 7:01 AM ---

_internal_

For OSP8, 

As we discussed on IRC, we don't have entry point for swift in ceilometer but in ceilometerclient.
But ceilometermiddleware is currently broken due to pytz requirements and we cannot run it in supported method.


osp7: we can use 'use = egg:ceilometer#swift' and 'paste.filter_factory = ceilometer.objectstore.swift_middleware:filter_factory'. in [filter:ceilometer] section, and it works. ( currently 'use = egg:ceilometermiddleware#swift' and 'paste.filter_factory = ceilometermiddleware.swift:filter_factory' does not work due to "swift-proxy-server: pkg_resources.VersionConflict: (pytz 2012d (/usr/lib/python2.7/site-packages), Requirement.parse('pytz>=2013.6'))" error unfortunately)

  # grep -FrIA1 '[paste.filter_factory]' /usr/lib/python2.7/site-packages/ceilometer*
  /usr/lib/python2.7/site-packages/ceilometer-2015.1.3-py2.7.egg-info/entry_points.txt:[paste.filter_factory]
  /usr/lib/python2.7/site-packages/ceilometer-2015.1.3-py2.7.egg-info/entry_points.txt-swift = ceilometer.objectstore.swift_middleware:filter_factory
  --
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.1.0-py2.7.egg-info/entry_points.txt:[paste.filter_factory]
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.1.0-py2.7.egg-info/entry_points.txt-swift = ceilometermiddleware.swift:filter_factory

  # cat /usr/lib/python2.7/site-packages/pycadf-0.8.0-py2.7.egg-info/requires.txt
  Babel>=1.3
  oslo.config>=1.6.0  # Apache-2.0
  oslo.context>=0.1.0                     # Apache-2.0
  oslo.i18n>=1.3.0  # Apache-2.0
  oslo.serialization>=1.2.0               # Apache-2.0
  pytz>=2013.6
  six>=1.7.0


osp8: we can use 'use = egg:ceilometermiddleware#swift' and 'paste.filter_factory = ceilometermiddleware.swift:filter_factory', which does not work due to "swift-proxy-server: pkg_resources.VersionConflict: (pytz 2012d (/usr/lib/python2.7/site-packages), Requirement.parse('pytz>=2013.6'))" error unfortunately.

  # grep -FrIA1 '[paste.filter_factory]' /usr/lib/python2.7/site-packages/ceilometer*
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.3.0-py2.7.egg-info/entry_points.txt:[paste.filter_factory]
  /usr/lib/python2.7/site-packages/ceilometermiddleware-0.3.0-py2.7.egg-info/entry_points.txt-swift = ceilometermiddleware.swift:filter_factory

  # cat /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt
  oslo.config>=2.1.0
  oslo.serialization>=1.4.0
  pytz>=2013.6


**Private note -- please do NOT disclose this to customer **

As we discussed on IRC, I just tried to hack version requirement and test out of my curiosity (not to answer to customer), it seemed to work as follows.
But anyway we cannot provide this to customer and we have to wait for fix (osp10 ??)...

~~~
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# diff -u /etc/ceilometer/ceilometer.conf-2016-09-01  /etc/ceilometer/ceilometer.conf;diff -u /etc/swift/proxy-server.conf-2016-09-09  /etc/swift/proxy-server.conf
--- /etc/ceilometer/ceilometer.conf-2016-09-01	2016-09-10 06:52:19.535466816 +0900
+++ /etc/ceilometer/ceilometer.conf	2016-09-10 06:52:33.634539475 +0900
@@ -996,7 +996,8 @@
 # Save event details. (boolean value)
 # Deprecated group/name - [collector]/store_events
 #store_events = false
-store_events = False
+store_events = True
+disable_non_metric_meters = False
 
 # WARNING: Ceilometer historically offered the ability to store events
 # as meters. This usage is NOT advised as it can flood the metering
--- /etc/swift/proxy-server.conf-2016-09-09	2016-09-09 19:18:43.087329951 +0900
+++ /etc/swift/proxy-server.conf	2016-09-10 06:49:29.067588316 +0900
@@ -9,20 +9,21 @@
 user = swift
 log_name = proxy-server
 log_facility = LOG_LOCAL1
-log_level = INFO
+log_level = DEBUG
 log_headers = False
 log_address = /dev/log
 
 
 
 [pipeline:main]
-pipeline = catch_errors bulk healthcheck cache crossdomain ratelimit authtoken keystone staticweb tempurl slo formpost account_quotas container_quotas proxy-server
+pipeline = catch_errors bulk healthcheck cache crossdomain ratelimit authtoken keystone staticweb tempurl slo formpost account_quotas container_quotas ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer ceilometer  ceilometer proxy-server
+
 
 [app:proxy-server]
 use = egg:swift#proxy
 set log_name = proxy-server
 set log_facility = LOG_LOCAL1
-set log_level = INFO
+set log_level = DEBUG
 set log_address = /dev/log
 log_handoffs = true
 allow_account_management = true
@@ -104,3 +105,18 @@
 
 [filter:container_quotas]
 use = egg:swift#container_quotas
+
+
+[filter:ceilometer]
+use = egg:ceilometermiddleware#swift
+paste.filter_factory = ceilometermiddleware.swift:filter_factory
+metadata_headers = X-Client,X-Service,X-Host
+control_exchange = swift
+driver = messagingv2
+topic = notifications
+log_level = WARN
+url = rabbit://guest:guest.123.101:5672/
+
+[filter:proxy-logging]
+use = egg:swift#proxy_logging

[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# diff -u /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt-2016-09-09   /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt
--- /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt-2016-09-09	2016-09-09 21:02:41.401097809 +0900
+++ /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt	2016-09-10 06:49:29.118588579 +0900
@@ -1,4 +1,4 @@
 oslo.config>=2.1.0
 oslo.serialization>=1.4.0
-pytz>=2013.6
+pytz>=2012d


+ openstack-config --get /etc/swift/proxy-server.conf filter:ceilometer use
egg:ceilometermiddleware#swift
+ ceilometer sample-list -m storage.objects.incoming.bytes
+----------------------------------+--------------------------------+-------+-------------+------+----------------------------+
| Resource ID                      | Name                           | Type  | Volume      | Unit | Timestamp                  |
+----------------------------------+--------------------------------+-------+-------------+------+----------------------------+
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.490000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.471000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.454000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.437000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.421000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.400000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.383000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.369000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.350000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.325000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T21:51:07.306000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.110000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.086000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.066000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.048000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:07.020000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.994000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.977000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.957000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.941000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.923000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:06.904000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.811000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.800000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.793000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.787000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.783000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.777000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.773000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.768000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.764000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.753000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T21:51:05.747000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.583000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.570000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.553000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 24576000.0  | B    | 2016-09-09T13:47:25.538000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.385000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.366000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.353000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:25.338000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.386000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.385000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.383000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.incoming.bytes | delta | 204800000.0 | B    | 2016-09-09T13:47:24.381000 |
+----------------------------------+--------------------------------+-------+-------------+------+----------------------------+
+ ceilometer sample-list -m storage.objects.outgoing.bytes
+----------------------------------+--------------------------------+-------+--------+------+----------------------------+
| Resource ID                      | Name                           | Type  | Volume | Unit | Timestamp                  |
+----------------------------------+--------------------------------+-------+--------+------+----------------------------+
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.261000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.259000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.257000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.255000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.253000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.251000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.248000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.241000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.229000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.225000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T21:51:03.223000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.038000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.025000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.011000 |
| ea8c5061423540b2b7e99c77d4041202 | storage.objects.outgoing.bytes | delta | 2.0    | B    | 2016-09-09T13:47:47.007000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.457000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.456000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.454000 |
| 665cc37fd9c947b39bfd7f7520c496bd | storage.objects.outgoing.bytes | delta | 632.0  | B    | 2016-09-09T13:47:22.452000 |
+----------------------------------+--------------------------------+-------+--------+------+----------------------------+

[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# 
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -qf /etc/ceilometer/ceilometer.conf
openstack-ceilometer-common-5.0.2-2.el7ost.noarch
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -qf /etc/swift/proxy-server.conf
openstack-swift-proxy-2.5.0-2.el7ost.noarch
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -qf /usr/lib/python2.7/site-packages/pycadf-1.1.0-py2.7.egg-info/requires.txt
python-pycadf-1.1.0-1.el7ost.noarch
[root@el72-osp8-all-virbr1 ~(keystone_project1-admin)]# rpm -q python-ceilometerclient
python-ceilometerclient-1.5.2-1.el7ost.noarch
~~~

Comment 9 errata-xmlrpc 2017-08-01 20:30:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1915