Description of problem: tcms-run on a run with 1194 test cases takes hours to complete. By investigation, Lukas Zachar found that the culprit is likely this construction in mutable.py, line 375: caseplan = [caseplan for caseplan in self.caseplans if caseplan.testcase == testcase][0] which is of exponential complexity. To be more precise: 1) BaseOS QE Tier1 on RHEL8 with modules, 241 TCs processed, run time ~ 3 min 2) BaseOS QE Tier2 on RHEL8 w/o modules, 880 TCs processed, run time ~ 60 min 3) BaseOS QE Tier2 on RHEL8 with modules, 1194 TCs to process, run time... after ~ 4 hours still not completed Version-Release number of selected component (if applicable): python2-nitrate-1.3-9.fc28 How reproducible: always Steps to Reproduce: 1. run tcms-run on a plan with 10, 100, 1000 TCs 2. 3. Actual results: Expected results: Additional info:
From gdb a bt can be get, showing is where the TR processing is stuck. The bt is most of the time this: Traceback (most recent call first): File "/usr/lib/python2.7/site-packages/nitrate/base.py", line 63, in getter if getattr(self, "_" + field) is NitrateNone: File "/usr/lib/python2.7/site-packages/nitrate/base.py", line 368, in __eq__ return self.id == other.id File "/usr/lib/python2.7/site-packages/nitrate/mutable.py", line 376, in sortkey if caseplan.testcase == testcase][0] File "/usr/lib/python2.7/site-packages/nitrate/containers.py", line 779, in <lambda> self._items, key=lambda x: self._object.sortkey(x)): File "/usr/lib/python2.7/site-packages/nitrate/containers.py", line 779, in __iter__ self._items, key=lambda x: self._object.sortkey(x)): File "/usr/lib/python2.7/site-packages/nitrate/mutable.py", line 185, in __iter__ for testcase in self.testcases: File "/usr/bin/tcms-run", line 367, in link_cases if testcase not in testplan: File "/usr/bin/tcms-run", line 770, in create_update_run link_cases(testplan, testrun, environment) File "/usr/bin/tcms-run", line 844, in process_plan testplan, summary, srpms, distro, notes, errata, environment) File "/usr/bin/tcms-run", line 865, in <module> process_plan(nitrate.TestPlan(testplan))
If someone could try hhttps://github.com/lukaszachy/python-nitrate/commit/3cddb0fd60ff826471a0c3b489f53a636773af46 ... For me tcms-run finished in quite nice time: Linked 1125 relevant cases, skipped 98 irrelevant cases real 3m5.521s user 0m18.295s sys 0m0.517s
Tried, tested. The patched version finished the test run creation with ~ 1200 testcases in few minutes instead of several hours. That's what I call a speedup, really. Thanks!
While patch is being accepted and official release will be made: rhel6, rhel7, fedora 27,28,29 builds are available on https://copr.fedorainfracloud.org/coprs/lzachar/python-nitrate/
python-nitrate-1.4-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-f699a3f38d
python-nitrate-1.4-1.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-14856dab5b
python-nitrate-1.4-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-83a0df283a
python-nitrate-1.4-1.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-654f0dfbff
python-nitrate-1.4-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-83a0df283a
python-nitrate-1.4-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-f699a3f38d
python-nitrate-1.4-1.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-14856dab5b
python-nitrate-1.4-1.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-654f0dfbff
python-nitrate-1.4-1.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.
python-nitrate-1.4-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.
python-nitrate-1.4-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.