Bug 664882 - vdsm produces too many iops
vdsm produces too many iops
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: vdsm22 (Show other bugs)
5.5.z
Unspecified Unspecified
urgent Severity urgent
: rc
: ---
Assigned To: Ayal Baron
Daniel Paikov
Storage
: ZStream
Depends On: 668247
Blocks: 666227 672238
  Show dependency treegraph
 
Reported: 2010-12-21 19:50 EST by Vladik Romanovsky
Modified: 2013-07-04 03:38 EDT (History)
12 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Hosts that held the Storage Pool Manager (SPM) role performed unnecessary I/O operations when periodically querying their inbox. The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage. With this update, VDSM does not perform unnecessary I/O operations when it periodically polls the inbox. As a result, I/O overhead on the SPM is reduced.
Story Points: ---
Clone Of:
: 666227 (view as bug list)
Environment:
Last Closed: 2012-02-20 23:52:15 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Vladik Romanovsky 2010-12-21 19:50:43 EST
Hi,

SPM hosts produce ~ 2000 iops, some times even more, every 2 seconds.

This is probably because the periodic inbox reads are being done as the following:

dd if=[path_to_inbox] iflag=direct bs=512 count=2000, while the total size of it  is only 1M

This situation heavily impacts customers SAN.

Thanks,

Vladik
Comment 6 Omri Hochman 2011-01-26 04:42:30 EST
verified with vdsm22-4.5-63.13.el5_6,


Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz
  await  svctm  %util
sda               0.01    45.52  0.03 20.20     0.68   525.75    26.02     0.20
   9.81   5.42  10.97
sda1              0.00     0.00  0.00  0.00     0.00     0.00    13.33     0.00
   4.26   4.09   0.00
sda2              0.01    45.52  0.03 20.20     0.68   525.75    26.03     0.20
   9.81   5.42  10.97
dm-0              0.00     0.00  0.04 65.72     0.68   525.75     8.01     1.20
  18.22   1.67  10.97
dm-1              0.00     0.00  0.00  0.00     0.00     0.00     8.00     0.00
   5.14   1.31   0.00
dm-14             0.00     0.00  1.00  0.00     8.03     0.00     8.00     0.00
   0.02   0.02   0.00
dm-34             0.00     0.00  0.99  0.01     7.93     0.94     8.85     0.00
   3.11   0.02   0.00
dm-15             0.00     0.00  0.44  0.00     3.44     0.00     7.75     0.00
Comment 7 Omri Hochman 2011-01-26 05:25:59 EST
should be re-verified for 5.7
Comment 10 Stephen Gordon 2012-02-10 16:13:53 EST
Please provide a technical note for this bugzilla in the "Technical Notes" field, or an indication that it is not to be documented in the errata text.
Comment 11 Dan Kenigsberg 2012-02-12 10:46:47 EST
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
C: On Red Hat Enterprise Linux 5.6 hosts that held the SPM role performed additional I/O operations when periodically querying their inbox.
C: The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage.
F: As of the vdsm22-4.5-63.14.el5_6 package vdsm does not perform additional un-necessary I/O when it periodically polls the inbox.
R: The SPM uses less I/O operations.

Text: Hosts that held the SPM role performed additional I/O operations when periodically querying their inbox. The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage. With this update, VDSM does not perform additional un-necessary I/O when it periodically polls the inbox. As a result, I/O overhead on the SPM is reduced.
Comment 12 Stephen Gordon 2012-02-13 10:31:34 EST
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1,6 +1 @@
-C: On Red Hat Enterprise Linux 5.6 hosts that held the SPM role performed additional I/O operations when periodically querying their inbox.
+Hosts that held the SPM role performed unnecessary I/O operations when periodically querying their inbox. The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage. With this update, VDSM does not perform unnecessary I/O operations when it periodically polls the inbox. As a result, I/O overhead on the SPM is reduced.-C: The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage.
-F: As of the vdsm22-4.5-63.14.el5_6 package vdsm does not perform additional un-necessary I/O when it periodically polls the inbox.
-R: The SPM uses less I/O operations.
-
-Text: Hosts that held the SPM role performed additional I/O operations when periodically querying their inbox. The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage. With this update, VDSM does not perform additional un-necessary I/O when it periodically polls the inbox. As a result, I/O overhead on the SPM is reduced.
Comment 13 Stephen Gordon 2012-02-13 10:34:47 EST
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-Hosts that held the SPM role performed unnecessary I/O operations when periodically querying their inbox. The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage. With this update, VDSM does not perform unnecessary I/O operations when it periodically polls the inbox. As a result, I/O overhead on the SPM is reduced.+Hosts that held the Storage Pool Manager (SPM) role performed unnecessary I/O operations when periodically querying their inbox. The additional I/O operations resulted in performance degradation in environments that were backed by SAN storage. With this update, VDSM does not perform unnecessary I/O operations when it periodically polls the inbox. As a result, I/O overhead on the SPM is reduced.
Comment 14 errata-xmlrpc 2012-02-20 23:52:15 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0169.html

Note You need to log in before you can comment on or make changes to this bug.