Bug 1592847
| Summary: | [v2v] ovirt-Imageio-daemon Memory growth unreasonable during disk transfer | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | guy chen <guchen> | ||||||||
| Component: | ovirt-imageio-daemon | Assignee: | Nir Soffer <nsoffer> | ||||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | guy chen <guchen> | ||||||||
| Severity: | high | Docs Contact: | |||||||||
| Priority: | unspecified | ||||||||||
| Version: | 4.2.4 | CC: | dagur, derez, ebenahar, guchen, nsoffer, tnisan | ||||||||
| Target Milestone: | ovirt-4.2.6 | Keywords: | Performance | ||||||||
| Target Release: | --- | Flags: | derez:
needinfo-
|
||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | ovirt-imageio-{common,daemon}-1.4.3 | Doc Type: | If docs needed, set a value | ||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2018-11-28 12:40:37 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | Scale | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
|
Description
guy chen
2018-06-19 12:35:44 UTC
Created attachment 1454997 [details]
Updated graph of 4 VMS transfer
Guy, this issue should be fixed by the attached patch. Can you check that it solve the issue you see? I tests this also in scale setup now. Before this change we had 2.5G RES (in top)
after 20-30 100G virt-v2v imports. With this change, we have constant 100M RES
after 10 100G virt-v2v imports.
Fixed in:
commit b8e83ef58d3b8c57191cd5913fc348153aace094
Author: Nir Soffer <nsoffer>
Date: Sat Jul 21 01:15:40 2018 +0300
daemon: Optimize transferred calculation
When uploading or downloading big images using tiny chunks, the
operations list becomes huge, taking lot of memory and consuming
significant cpu time.
Optimize the calculation by removing completed operations from the
ticket, and keeping a sorted list of transferred ranges. When an
operation is removed, the operation range is merged into the completed
ranges list, which should be small in typical download or upload flows.
When calculating total transferred bytes, we create list of ranges from
the ongoing operations, which should be small, and merge the completed
and ongoing ranges.
Change-Id: Ib33967d9e858353037542eedbb3c68d350bf3ad4
Bug-Url: https://bugzilla.redhat.com/1592847
Signed-off-by: Nir Soffer <nsoffer>
We're releasing today 4.2.6 RC2 including v1.4.3 which is referencing this bug. can you please check this bug status? (In reply to Sandro Bonazzola from comment #6) 1.4.3 should fix this, but it was not tested yet by QE. We can list this bug as fixes included by this version, but we cannot mark it as verified without testing. We have a downstream build, moving to ON_QA Created attachment 1479169 [details]
RES of imageio build 1.4.3
From load run on 19.8 with ovirt-imageio-daemon-1.4.3 version with V2V migration of 10 VMS 100GB to FC 66% full disk we see the RES is indeed stable on 95G and not increasing as before, thus verifying the bug, attache the RES image. Guy, can you add more details how the behavior changed since previous versions during same flows? (e.g. 10x100g import) - On which versions we tested the same flow? - What was the memory usage seen in each tested version? Created attachment 1479201 [details]
1.4.2 imagio rate
Before comment10 got mixed up, should be MB not GB. Added previous ovirt-imageio-daemon 1.4.2 version RES with the same scenario. So in 1.4.2 the RES increases throughout the test up to 3.6 GB. In 1.4.3 the RES stabilize around 95 MB and does not continue to increase. Thanks! I did not know it was so bad. any chance to get a graph from the 1.4.3 test run? 1.4.3 test run graph is attached - "RES of imageio build 1.4.3" Thanks Guy, I think we have all data. |