Description of problem: Profiling job should be moved to beginning of the import flow. This change is an enhancement and is required for improving the import performance. With this change coming in, it will make sure that there is no race condition between gluster-integration sds-sync and gluster profiling commands. Version-Release number of selected component (if applicable): How reproducible: 100 % Steps to Reproduce: N/A Actual results: Currently profiling job is executed during the import job as a part of the gluster-integration sds syn. Expected results: Profiling job will be one of the initials jobs of the import flow and gluster-integration sds-sync will be invoked later on in the import flow. Additional info:
I have 2 questions: 1) The import should achieve the same end result, but do it in bit different order. Right? 2) What is the performance improvement you are expecting to achieve? I'm asking 2nd question as I'm not willing to implement this change without expected performance improvement is reached.
(In reply to Martin Bukatovic from comment #2) > I have 2 questions: > > 1) The import should achieve the same end result, but do it in bit different > order. Right? This is correct. > 2) What is the performance improvement you are expecting to achieve? > > I'm asking 2nd question as I'm not willing to implement this change without > expected performance improvement is reached. Earlier profiling was being done during the gluster_integration sds_sync along with the sync. Before this point, high number of volumes were never tested for performance and scale. During the testing we observed that gluster profiling commands were competing with the gluster commands required for the sds sync. This scenario was always present there but never was really observed. When we shift the profiling to the beginning of the import, gluster integration is not started yet, which makes gluster completely free to execute the profiling commands. To give a fare comparison for improvement, for 400 volumes, the orignal code used to take range from 30 mins to more than an hour, but with this shift the profiling gets completed in around 10 mins or even less for the same number of volumes.
This bug is taken out from BU3
I have tested Import task with the same fresh setup with the same 3 volumes but with different versions of tendrl: New version: tendrl-ansible-1.6.3-11.el7rhgs.noarch tendrl-api-1.6.3-12.el7rhgs.noarch tendrl-api-httpd-1.6.3-12.el7rhgs.noarch tendrl-commons-1.6.3-17.el7rhgs.noarch tendrl-grafana-plugins-1.6.3-20.el7rhgs.noarch tendrl-grafana-selinux-1.5.4-3.el7rhgs.noarch tendrl-monitoring-integration-1.6.3-20.el7rhgs.noarch tendrl-node-agent-1.6.3-18.el7rhgs.noarch tendrl-notifier-1.6.3-4.el7rhgs.noarch tendrl-selinux-1.5.4-3.el7rhgs.noarch tendrl-ui-1.6.3-15.el7rhgs.noarch Old version: tendrl-ansible-1.6.3-11.el7rhgs.noarch tendrl-api-1.6.3-10.el7rhgs.noarch tendrl-api-httpd-1.6.3-10.el7rhgs.noarch tendrl-commons-1.6.3-15.el7rhgs.noarch tendrl-grafana-plugins-1.6.3-20.el7rhgs.noarch tendrl-grafana-selinux-1.5.4-3.el7rhgs.noarch tendrl-monitoring-integration-1.6.3-20.el7rhgs.noarch tendrl-node-agent-1.6.3-15.el7rhgs.noarch tendrl-notifier-1.6.3-4.el7rhgs.noarch tendrl-selinux-1.5.4-3.el7rhgs.noarch tendrl-ui-1.6.3-14.el7rhgs.noarch According to test results is cluster import time reduced in newer version. Median of 3 test run times with new version is 63 seconds and median of 3 test run times with older version is 85 seconds. All cluster imports with new version were finished before any import with older version. In task messages is now message `Starting profiling for volumes` before importing hosts. --> VERIFIED
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0660