Bug 1508694
Summary: | How to get per instance bandwidth | ||
---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Robin Cernin <rcernin> |
Component: | openstack-tripleo-heat-templates | Assignee: | Pradeep Kilambi <pkilambi> |
Status: | CLOSED CURRENTRELEASE | QA Contact: | Gurenko Alex <agurenko> |
Severity: | high | Docs Contact: | |
Priority: | high | ||
Version: | 8.0 (Liberty) | CC: | alolivei, jruzicka, mabaakou, mburns, mlopes, pablo.iranzo, pkilambi, rcernin, rhel-osp-director-maint, rhos-docs, samccann, srevivo |
Target Milestone: | zstream | Keywords: | Triaged, ZStream |
Target Release: | 10.0 (Newton) | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | openstack-tripleo-heat-templates-5.3.3-3.el7ost | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-03-08 07:44:22 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1473146 |
Description
Robin Cernin
2017-11-02 00:49:10 UTC
The instance network bandwidth works only if libvirt is able to measure it. To check if libvirt side works well: # virsh domiflist 20932a75-3d04-44b1-b5c6-fcbeb7bfbf69 Interface Type Source Model MAC ------------------------------------------------------- tapda0261ee-9f bridge brq07487477-8c virtio fa:16:3e:f0:c3:89 # virsh domifstat 20932a75-3d04-44b1-b5c6-fcbeb7bfbf69 tapda0261ee-9f tapda0261ee-9f rx_bytes 1658017639 tapda0261ee-9f rx_packets 19072434 tapda0261ee-9f rx_errs 0 tapda0261ee-9f rx_drop 0 tapda0261ee-9f tx_bytes 38658724198 tapda0261ee-9f tx_packets 20335025 tapda0261ee-9f tx_errs 0 tapda0261ee-9f tx_drop 0 If "Interface" is empty in "virsh domiflist" or if "virsh domifstat" doesn't output anything, that means the network interface type does not support statistics, so Ceilometer can't retrieved bandwidth. I'm sure it works with libvirt >= 2.0.0, maybe in recent 1.3.X, but it was not implemented on < 1.3.X. In the lab: libvirt-3.2.0-14.el7_4.2.x86_64 [root@compute-0 ~]# virsh domiflist instance-0000000c Interface Type Source Model MAC ------------------------------------------------------- tapaef4ac7c-50 bridge qbraef4ac7c-50 virtio fa:16:3e:34:71:95 [root@compute-0 ~]# virsh domifstat instance-0000000c tapaef4ac7c-50 tapaef4ac7c-50 rx_bytes 47819 tapaef4ac7c-50 rx_packets 387 tapaef4ac7c-50 rx_errs 0 tapaef4ac7c-50 rx_drop 0 tapaef4ac7c-50 tx_bytes 40082 tapaef4ac7c-50 tx_packets 347 tapaef4ac7c-50 tx_errs 0 tapaef4ac7c-50 tx_drop 0 No network in ceilometer meter-list --- +----------------------------+-------+-----------+--------------------------------------+----------------------------------+----------------------------------+ | Name | Type | Unit | Resource ID | User ID | Project ID | +----------------------------+-------+-----------+--------------------------------------+----------------------------------+----------------------------------+ | bandwidth | delta | B | 40b6d8b9-a687-423f-8add-fb904db6e7a9 | None | 2c86836ca10e47b4b2a9be60316772c2 | | disk.ephemeral.size | gauge | GB | 794677ff-50fb-4c0f-b3cf-da38752ecbe7 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | disk.ephemeral.size | gauge | GB | 970e25fb-2f8e-47ef-a433-b5f0ea22073e | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | disk.ephemeral.size | gauge | GB | e01f6a01-d3d8-4eb8-8880-e670f6882164 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | disk.ephemeral.size | gauge | GB | f6e421a0-3d12-40c1-9ee1-9f6c15893c63 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | disk.root.size | gauge | GB | 794677ff-50fb-4c0f-b3cf-da38752ecbe7 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | disk.root.size | gauge | GB | 970e25fb-2f8e-47ef-a433-b5f0ea22073e | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | disk.root.size | gauge | GB | e01f6a01-d3d8-4eb8-8880-e670f6882164 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | disk.root.size | gauge | GB | f6e421a0-3d12-40c1-9ee1-9f6c15893c63 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | image | gauge | image | cacbb8be-921b-4051-9d82-db43dd3b2ae6 | None | 2c86836ca10e47b4b2a9be60316772c2 | | image.size | gauge | B | cacbb8be-921b-4051-9d82-db43dd3b2ae6 | None | 2c86836ca10e47b4b2a9be60316772c2 | | memory | gauge | MB | 794677ff-50fb-4c0f-b3cf-da38752ecbe7 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | memory | gauge | MB | 970e25fb-2f8e-47ef-a433-b5f0ea22073e | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | memory | gauge | MB | e01f6a01-d3d8-4eb8-8880-e670f6882164 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | memory | gauge | MB | f6e421a0-3d12-40c1-9ee1-9f6c15893c63 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | storage.objects | gauge | object | 2c86836ca10e47b4b2a9be60316772c2 | None | 2c86836ca10e47b4b2a9be60316772c2 | | storage.objects | gauge | object | 6583fa75c05d42ee849adba853483a1f | None | 6583fa75c05d42ee849adba853483a1f | | storage.objects.containers | gauge | container | 2c86836ca10e47b4b2a9be60316772c2 | None | 2c86836ca10e47b4b2a9be60316772c2 | | storage.objects.containers | gauge | container | 6583fa75c05d42ee849adba853483a1f | None | 6583fa75c05d42ee849adba853483a1f | | storage.objects.size | gauge | B | 2c86836ca10e47b4b2a9be60316772c2 | None | 2c86836ca10e47b4b2a9be60316772c2 | | storage.objects.size | gauge | B | 6583fa75c05d42ee849adba853483a1f | None | 6583fa75c05d42ee849adba853483a1f | | vcpus | gauge | vcpu | 794677ff-50fb-4c0f-b3cf-da38752ecbe7 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | vcpus | gauge | vcpu | 970e25fb-2f8e-47ef-a433-b5f0ea22073e | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | vcpus | gauge | vcpu | e01f6a01-d3d8-4eb8-8880-e670f6882164 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | vcpus | gauge | vcpu | f6e421a0-3d12-40c1-9ee1-9f6c15893c63 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | +----------------------------+-------+-----------+--------------------------------------+----------------------------------+----------------------------------+ I have checked what wrong on the labs. I have found that "host" is unset in ceilometer.conf, that means Ceilometer use the hostname (without domain) as fallback to determine the name of the compute node registered in Nova. But Nova have fqdn configured as "host". So Ceilometer and Nova compute host are out of sync, making all libvirt based metrics collection broken. I have set "host = compute-0.localdomain" to ceilometer.conf and restart openstack-ceilometer-compute.service. Metrics are there now: $ ceilometer meter-list | grep network | network.incoming.bytes | cumulative | B | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | network.incoming.bytes.rate | gauge | B/s | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | network.incoming.packets | cumulative | packet | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | network.incoming.packets.rate | gauge | packet/s | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | network.outgoing.bytes | cumulative | B | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | network.outgoing.bytes.rate | gauge | B/s | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | network.outgoing.packets | cumulative | packet | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | | network.outgoing.packets.rate | gauge | packet/s | instance-0000000c-970e25fb-2f8e-47ef-a433-b5f0ea22073e-tapaef4ac7c-50 | 6d361abcd4a1471ea69519f578961f9e | 2c86836ca10e47b4b2a9be60316772c2 | We fixed "out of sync host" in recent OSP version, we may need to backport this to OSP8. I will see with the team what we can do to fix it. The fix have been related since a while noww and already part of the latest z release. So I'm closing this. |