Bug 1353707
| Summary: | MiqVimBrokerWorker uses much more cpu during same workload in 5.6 vs 5.5 (VMware Capacity & Utilization) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat CloudForms Management Engine | Reporter: | Alex Krzos <akrzos> | ||||||
| Component: | Performance | Assignee: | Adam Grare <agrare> | ||||||
| Status: | CLOSED DUPLICATE | QA Contact: | Dave Johnson <dajohnso> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | medium | ||||||||
| Version: | 5.6.0 | CC: | jhardy, obarenbo, simaishi | ||||||
| Target Milestone: | GA | ||||||||
| Target Release: | cfme-future | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2016-11-29 18:32:47 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
It is possible this is also caused by running garbage collection every API call, https://bugzilla.redhat.com/show_bug.cgi?id=1341402#c3 I'll try to reproduce the high cpu load then revert that commit and see if the cpu load returns to normal levels. Created attachment 1216327 [details] VimBroker CPU Usage GC vs No-GC This appears to be the same issue as https://bugzilla.redhat.com/show_bug.cgi?id=1341402, I enabled metrics collection on a 4096 VM/16 Cluster/1352 Datastore VC simulator with and without https://github.com/ManageIQ/manageiq/commit/4bbad2141b4ec820b56885584abd67ba94cf5569 on a current 5.6 appliance and the attachment shows the CPU usage. *** This bug has been marked as a duplicate of bug 1378998 *** |
Created attachment 1177420 [details] Two stacked graphs displaying worker cpu usage during 4 C&U workload scenarios Description of problem: I ran 4 sets of 4 hour long workloads where: 1. Provider is added 2. C&U is turned on 3. Metrics collection and processing is occurring transparently to the user Each workload uses a different provider: Small/Medium/Large/XLarge - all of VMware type Over two different appliance versions: 5.6.0.13 5.5.5.0 I then compared system resource utilization and found that the MiqVimBrokerWorker is utilizing the CPU far more in 5.6.0.13 than it previously did in 5.5.5.0 Version-Release number of selected component (if applicable): 5.6.0.13 How reproducible: When compared correctly it is reproduced Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: This appears to be exacerbating a bottleneck in that we have a single point to communicate to the provider to capture metrics and that single point is using far more CPU than it used to in older builds. Additionally more CPU utilization is occurring on the 5.6 builds (We are using the cpu more) Attached Graph: Top is 5.6, bottom is 5.5, each 4 hour segment is the next workload(C&U) scenario(Different provider) in order of 4 hours of small, 4 hours of medium, 4 hours of large, and 4 hours of xlarge. Note the filling out of available CPU on the 5.6 towards the Large/Xlarge scenarios.