Bug 833398 - [RFE][nova][preformance] qemu-img info command is executed twice within the same second every 63 seconds for each running instance
[RFE][nova][preformance] qemu-img info command is executed twice within the s...
Status: CLOSED UPSTREAM
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova (Show other bugs)
1.0 (Essex)
Unspecified Unspecified
low Severity low
: Upstream M3
: 5.0 (RHEL 7)
Assigned To: Xavier Queralt
Dafna Ron
storage
: CodeChange, FutureFeature, MoveUpstream, Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-19 08:13 EDT by David Naori
Modified: 2016-04-26 13:53 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-03-10 09:59:01 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
compute.log (281.83 KB, application/x-gzip)
2012-06-19 08:19 EDT, David Naori
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1154162 None None None Never

  None (edit)
Description David Naori 2012-06-19 08:13:14 EDT
Description of problem:
qemu-img info command is executed twice within the same second every 00:01:03 minutes 

Version-Release number of selected component (if applicable):
openstack-nova-2012.1-12.el6.noarch


Same command executed twice within the same second:
2012-06-19 15:05:31 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img info /var/lib/nova/instances/instance-00000010/disk from (pid=30842) execute /usr/lib/python2.6/site-packages/nova/utils.py:220
2012-06-19 15:05:31 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img info /var/lib/nova/instances/instance-00000010/disk from (pid=30842) execute /usr/lib/python2.6/site-packages/nova/utils.py:220


# cat /var/log/nova/compute.log |awk '/qemu-img info/ {print $2}' | tail -10
15:06:36
15:06:36
15:07:40
15:07:40
15:08:45
15:08:45
15:09:48
15:09:48
15:10:53
15:10:53
Comment 1 Pádraig Brady 2012-06-19 08:18:42 EDT
Period is 63 seconds, while cache manager shoud be every
image_cache_manager_interval * periodic_interval = 40 * 60 = 40 mins
Comment 2 David Naori 2012-06-19 08:19:55 EDT
Created attachment 592930 [details]
compute.log
Comment 3 Pádraig Brady 2012-07-10 11:48:11 EDT
Note to disable the cache manager set image_cache_manager_interval = 0
That's overkill though, especially considering bug 837627
Comment 4 Pádraig Brady 2012-07-18 10:41:36 EDT
So this is coming from:

ComputeManager.update_available_resource() # runs every 60s
  libvirt.connection.update_available_resource()
    get_disk_available_least()
      for instance in instances:
        get_instance_disk_info(instance)
          if qcow2:
              virt_size = execute('qemu-img', 'info')...
              backing_file = libvirt_utils.get_disk_backing_file()
                backing file = execute('qemu-img', 'info')...

The backing file is not needed in the above.
Also the virtual size should not be changing and could be cached.
Comment 5 David Naori 2012-07-18 19:58:25 EDT
Note: nova executes 'qemu-img info' twice every 60 seconds for every disk file on each instance.
in 4 out of 5 of the default flavors instances has Ephemeral disk too.

So actually in most cases it will run 4 times for each instance every 60 seconds which makes it a bigger issue.

2012-07-19 02:49:12 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img info /mnt/nova-lib//instances/instance-0000016b/disk from (pid=7273) execute /usr/lib/python2.6/site-packages/nova/utils.py:220
2012-07-19 02:49:12 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img info /mnt/nova-lib//instances/instance-0000016b/disk from (pid=7273) execute /usr/lib/python2.6/site-packages/nova/utils.py:220
2012-07-19 02:49:13 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img info /mnt/nova-lib//instances/instance-0000016b/disk.local from (pid=7273) execute /usr/lib/python2.6/site-packages/nova/utils.py:220
2012-07-19 02:49:13 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img info /mnt/nova-lib//instances/instance-0000016b/disk.local from (pid=7273) execute /usr/lib/python2.6/site-packages/nova/utils.py:220
Comment 6 Nikola Dipanov 2012-12-14 07:52:41 EST
Targetting it to Grizzly (RHOS-3.0) as there seems to be very little incentive to backport such an enhancement.
Comment 8 Nikola Dipanov 2013-09-25 06:52:17 EDT
There is an upstream bug for this so I am linking it. It seems it is not being worked on upstream ATM - but seems like a nice improvement to have.

A lot of the code has been changed since this bug was reported - but the gist of the problem is still there. I will comment more on the upstream bug.

Moving to 5.0.
Comment 9 Xavier Queralt 2014-03-10 09:59:01 EDT
For any further follow up on this issue, please refer to the upstream bug linked to this bug.

Note You need to log in before you can comment on or make changes to this bug.