Bug 1524393

Summary: [RFE] extract volume_image_metadata from volume events
Product: Red Hat OpenStack Reporter: Luca Miccini <lmiccini>
Component: openstack-ceilometerAssignee: Eoghan Glynn <eglynn>
Status: CLOSED ERRATA QA Contact: Nataf Sharabi <nsharabi>
Severity: high Docs Contact:
Priority: medium    
Version: 10.0 (Newton)CC: achernet, augol, dcadzow, dwojewod, lnatapov, rlondhe, slinaber, srevivo
Target Milestone: Upstream M1Keywords: FutureFeature, TechPreview, Triaged
Target Release: 15.0 (Stein)   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-ceilometer-10.0.1-0.20180530162349.1c02e4b.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-09-21 11:15:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1625912    
Bug Blocks:    

Description Luca Miccini 2017-12-11 12:26:11 UTC
Description of problem:

The default set of event definitions shipped with ceilometer does not collect any information about metadata associated to volumes.

It would be useful to extract at least image_id from the cinder notification and store it for auditing/reporting/charge back purposes.

| volume_image_metadata          | {u'container_format': u'bare', u'min_ram': u'0', u'disk_format': u'qcow2', u'image_name': u'cirros', u'image_id': u'dc542831-eb29-472d-9a0b-41ca39a51840', u'checksum': u'f8ab98ff5e73ebab884d80c9dc9c7290', u'min_disk': u'0', u'size': u'13267968'}

The glance_metadata is contained inside the cinder notification payload, like:

glance_metadata=[{"deleted": "false", "created_at": "2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "bare", "key": "container_format", "snapshot_id": "null", "deleted_at": "null", "id": 41},{"deleted": "false", "created_at": "2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "0", "key": "min_ram", "snapshot_id": "null", "deleted_at": "null", "id": 42},{"deleted": "false", "created_at": "2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "qcow2", "key": "disk_format", "snapshot_id": "null", "deleted_at": "null", "id": 43},{"deleted": "false", "created_at": "2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "cirros", "key": "image_name", "snapshot_id": "null", "deleted_at": "null", "id": 44},{"deleted": "false", "created_at": "!
> 2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "dc542831-eb29-472d-9a0b-41ca39a51840", "key": "image_id", "snapshot_id": "null", "deleted_at": "null", "id": 45},{"deleted": "false", "created_at": "2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "f8ab98ff5e73ebab884d80c9dc9c7290", "key": "checksum", "snapshot_id": "null", "deleted_at": "null", "id": 46},{"deleted": "false", "created_at": "2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "0", "key": "min_disk", "snapshot_id": "null", "deleted_at": "null", "id": 47},{"deleted": "false", "created_at": "2017-12-08T12:14:46.000000", "volume_id": "7feb7fce-6bff-4683-8a03-8ab634784193", "updated_at": "null", "value": "13267968", "key": "size", "snapshot_id": "null", "deleted_at": "null", "id": 48}]

Version-Release number of selected component (if applicable):

[root@overcloud-controller-0 ~]# rpm -qa |grep ceilometer
python-ceilometer-7.1.1-4.el7ost.noarch
openstack-ceilometer-compute-7.1.1-4.el7ost.noarch
puppet-ceilometer-9.5.0-2.el7ost.noarch
python-ceilometerclient-2.6.2-1.el7ost.noarch
openstack-ceilometer-notification-7.1.1-4.el7ost.noarch
openstack-ceilometer-polling-7.1.1-4.el7ost.noarch
openstack-ceilometer-central-7.1.1-4.el7ost.noarch
openstack-ceilometer-api-7.1.1-4.el7ost.noarch
openstack-ceilometer-collector-7.1.1-4.el7ost.noarch
python-ceilometermiddleware-0.5.2-1.el7ost.noarch
openstack-ceilometer-common-7.1.1-4.el7ost.noarch


How reproducible:

always

Steps to Reproduce:
1. deploy OSP10 overcloud
2. create a volume
3. check the content of the ceilometer event

Actual results:

no metadata is extracted / saved from the cinder notification

Expected results:

glance image_id to be included in the event

Additional info:

Comment 3 Luca Miccini 2017-12-11 14:02:22 UTC
Meanwhile users can adjust the event_definitions.yaml file like:

- event_type: ['volume.exists', 'volume.create.*', 'volume.delete.*', 'volume.resize.*', 'volume.attach.*', 'volume.detach.*', 'volume.update.*', 'snapshot.exists', 'snapshot.create.*', 'snapshot.delete.*', 'snapshot.update.*']
  traits: &cinder_traits
    user_id:
      fields: payload.user_id
    project_id:
      fields: payload.tenant_id
    availability_zone:
      fields: payload.availability_zone
    display_name:
      fields: payload.display_name
    source_image_id:
      fields: payload.glance_metadata[?key = "image_id"].value   
    replication_status:
      fields: payload.replication_status
    status:
      fields: payload.status
    created_at:
      fields: payload.created_at
- event_type: ['volume.exists', 'volume.create.*', 'volume.delete.*', 'volume.resize.*', 'volume.attach.*', 'volume.detach.*', 'volume.update.*']
  traits:
    <<: *cinder_traits
    resource_id:
      fields: payload.volume_id
    host:
      fields: payload.host
    size:
      fields: payload.size
    type:
      fields: payload.volume_type
    source_image_id:
      fields: payload.glance_metadata[?key = "image_id"].value
    replication_status:
      fields: payload.replication_status

Comment 6 Mehdi ABAAKOUK 2018-04-02 15:58:01 UTC
This have been implemented upstream and will be part of OSP14 as planned.

Comment 20 Leonid Natapov 2019-01-15 12:45:15 UTC
Mehdi,please,provide testing instructions for this RFE. Thanks.

Comment 21 Mehdi ABAAKOUK 2019-01-16 07:06:01 UTC
Already done in: https://bugzilla.redhat.com/show_bug.cgi?id=1524393#c12

Comment 27 errata-xmlrpc 2019-09-21 11:15:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811