RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 618142 - [vdsm] [libvirt] water mark notifications from qemu are not handled
Summary: [vdsm] [libvirt] water mark notifications from qemu are not handled
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm
Version: 6.1
Hardware: All
OS: Linux
low
high
Target Milestone: rc
: ---
Assignee: Dan Kenigsberg
QA Contact: yeylon@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 581275
TreeView+ depends on / blocked
 
Reported: 2010-07-26 09:14 UTC by Haim
Modified: 2016-04-18 06:33 UTC (History)
9 users (show)

Fixed In Version: vdsm-4.9-11.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-08-19 15:10:39 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
vdsm full log (2.99 MB, application/octet-stream)
2010-07-26 09:14 UTC, Haim 		no flags Details
View All

Description Haim 2010-07-26 09:14:44 UTC 
Created attachment 434367 [details]
vdsm full log

Description of problem:

currently, qemu, libvirt and vdsm supports 'enospace' notification, for sparse disk usage, and its working (space is over, new extend msg is created, and lvextend command is created to extend disk size), however, there is another notification by qemu that notifies current disk allocation is about to exceed (low water mark reached) where vdsm should catch, and produce an lvextend command. 

currently, sometimes vm goes to pause and when I search the logs, I don't see any 'reach|water|high'. 


Version-Release number of selected component (if applicable):

vdsm-4.9-10.1.x86_64
libvirt-0.8.1-15.el6.x86_64
qemu-kvm-0.12.1.2-2.96.el6.x86_64


Steps to Reproduce:
1. create a new guest machine 
2. set machine disk to sparse (5G) 
3. install linux redhat operating system on it (default disk allocation is 0.5G where linux installation takes about 800M, so extend message should and will be created)

attached full vdsm log from the moment vdsm is up (with Danken's patch).
  
Actual results:


Expected results:


Additional info:
Comment 1 Haim 2010-08-01 08:53:25 UTC 
fixed on version vdsm-4.9-11.el6.x86_64. 

tested both virtio and ide block. 

vm doesn't enter to pause, and extend the lv on time. 

some logs for indication and review: 

hread-77::INFO::2010-08-01 04:25:42,135::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 859766272
Thread-77::DEBUG::2010-08-01 04:25:42,136::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:25:44,169::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 872152576
Thread-77::DEBUG::2010-08-01 04:25:44,170::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:25:46,176::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 872152576
Thread-77::DEBUG::2010-08-01 04:25:46,177::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:25:48,184::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 883621376
Thread-77::DEBUG::2010-08-01 04:25:48,184::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:25:50,191::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 883621376
Thread-77::DEBUG::2010-08-01 04:25:50,192::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:25:52,198::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 883621376
Thread-77::DEBUG::2010-08-01 04:25:52,199::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:25:54,319::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 914292224
Thread-77::DEBUG::2010-08-01 04:25:54,320::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:25:56,431::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 937754112
Thread-77::DEBUG::2010-08-01 04:25:56,432::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:15,340::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1932787200
Thread-77::DEBUG::2010-08-01 04:35:15,341::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:17,348::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1932787200
Thread-77::DEBUG::2010-08-01 04:35:17,349::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:19,626::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1933639168
Thread-77::DEBUG::2010-08-01 04:35:19,626::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:22,012::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1933704704
Thread-77::DEBUG::2010-08-01 04:35:22,012::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:24,018::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1933835776
Thread-77::DEBUG::2010-08-01 04:35:24,019::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:26,025::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1933901312
Thread-77::DEBUG::2010-08-01 04:35:26,026::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:28,033::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1933901312
Thread-77::DEBUG::2010-08-01 04:35:28,033::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:30,481::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1935932928
Thread-77::DEBUG::2010-08-01 04:35:30,481::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:32,883::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1937636864
Thread-77::DEBUG::2010-08-01 04:35:32,884::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:34,890::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1939209728
Thread-77::DEBUG::2010-08-01 04:35:34,891::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:37,035::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1940389376
Thread-77::DEBUG::2010-08-01 04:35:37,036::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:39,047::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1941110272
Thread-77::DEBUG::2010-08-01 04:35:39,048::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:41,054::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1941110272
Thread-77::DEBUG::2010-08-01 04:35:41,055::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:43,061::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1941110272
Thread-77::DEBUG::2010-08-01 04:35:43,062::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:45,624::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1942289920
Thread-77::DEBUG::2010-08-01 04:35:45,625::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:48,852::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1944389632
Thread-77::DEBUG::2010-08-01 04:35:48,853::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:50,874::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1945435648
Thread-77::DEBUG::2010-08-01 04:35:50,874::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:52,881::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1945435648
Thread-77::DEBUG::2010-08-01 04:35:52,882::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:54,889::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1945697792
Thread-77::DEBUG::2010-08-01 04:35:54,890::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:56,896::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1948777984
Thread-77::DEBUG::2010-08-01 04:35:56,897::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:35:59,290::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1953234432
Thread-77::DEBUG::2010-08-01 04:35:59,291::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:36:01,644::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1961229824
Thread-77::DEBUG::2010-08-01 04:36:01,645::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Thread-77::INFO::2010-08-01 04:36:03,652::vm::1260::vds.vmlog.01e2aed7-86ec-4994-baee-92df140c81b1::_onHighWrite: write above watermark on hda offset 1962147328
Thread-77::DEBUG::2010-08-01 04:36:03,653::storage_mailbox::99::irs::new extend msg created: domain: 7caedecf-0d66-482d-b6b7-4a1498beb79f, volume: b6514095-d41f-426f-8664-80dd1d535804
Note You need to log in before you can comment on or make changes to this bug.