Description of problem: Recently we had a customer upload a vmcore split into very small chunks (256KB) and there were thousands of files. From this point on, anyone else using the web UI got either very slow or crashed browser when they tried to load it. I think it may be better to change the default behavior away from loading the full list of ftp files on the manager page. The the manager page can be simple and just show existing tasks. Maybe we update the 'filter' line to have a checkbox to load the files from ftp, or wait for someone to enter something into the filter box, or do some action such as clicking the 'filter' button or a new button to 'load list of ftp files'. Version-Release number of selected component (if applicable): retrace-server-1.11-4.el6.noarch How reproducible: Every time loaded, though the thousands of files may be removed in the next several days. Steps to Reproduce: 1. go to the 'manager' web UI page when there's thousands of files on the ftp server Actual results: web UI is slow, hangs or crashes browser Expected results: web UI is more responsive. If you need to find a file with a specific pattern, that should be a non-default action. Additional info: This is probably a low-frequency problem, but if it happens, it makes the webUI much less usable. We put in a ticket to remove these files, but the request was denied, at least in part due to the mechanics of how the ftp server works. The format of the vmcore split files is vmcore_part_XXXX. I think there are 98,000 of these files.
Would it be possible to have it display files that match some string? Frequently the customer indicates that he uploaded core but fails to list the name of the file. I search for the case number and frequently find the core. Or would this be as slow?
There is already a "filter" field at the top of the page. The problem is that it shows unfiltered as soon as you go to the page. I think making it wait til you put something in the filter field is a great idea. And I suppose it could take an empty filter and show everything. While I'm at it: I can never remember if the filter needs/uses wildcards, or if it just looks for what you enter as a substring. Maybe a bit of label text or a mouseover popup could explain that.
Just to chime in. Since there is the possibility of a future customer uploading an unreasonable number of files in the future, I completely agree that populating the "FTP Files" on load should be avoided. It seems like unnecessary load when you can just use a '*' wildcard after the fact and achieve the same end state. To that note, the filter filed does accept wildcards and usually requires them. For example: 00930333 Returns: No files *00930333* Returns 00930333-vmcore However, in poking around, I don't believe the fix is that straightforward. Searching through the FTP server contents seems to be the bottleneck in this interaction: $ time curl https://optimus.gsslab.rdu2.redhat.com/manager 1>/dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 12.9M 100 12.9M 0 0 2156k 0 0:00:06 0:00:06 --:--:-- 3431k real 0m6.170s user 0m0.119s sys 0m0.103s $ time curl https://optimus.gsslab.rdu2.redhat.com/manager?filter=None 1>/dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 4932 100 4932 0 0 912 0 0:00:05 0:00:05 --:--:-- 1218 real 0m5.409s user 0m0.047s sys 0m0.132s The filter behaviour still needs to traverse the contents of the FTP server and returns in roughly the same amount of time from what I see. It's just my 2 cents, but to bypass the issue at this particular time, I would recommend determining the filename exactly and just plug it into the URL for the task creation page: https://optimus.gsslab.rdu2.redhat.com/manager/<Filename>
Is there some magic javascript library that dynamically loads the list of files as they are received from the FTP server? Such a mechanism would allow the rest of the page to be viewable and usable.
Apparently optimus UI is now suffering from this problem again due to someone uploading a ton of small split files starting with 'x' (for example, "xzaei") though doesn't appear to be quite as bad as before. Last time we opened a ticket with IT and they wouldn't remove the files, stating they would "auto-remove" within 30 days. Maybe we should re-think the web UI though this bug still remains a fairly low priority in relation to other retrace-server bugs.
I went ahead and put together a patch that addresses this condition and submitted it upstream. https://github.com/abrt/retrace-server/pull/108 In short, I moved the FTP query operation into a separate AJAX call. The only additional requirement that this includes is the ability to access the google-hosted jQuery library similar to: <script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script> Kyle Walker Software Maintenance Engineer - SEG North America
I just tested kyle's patch on our development server and it looks like it definitely fixes this problem. Even with a constant creating stream of files on the ftp server: # for i in $(seq 1 10000000); do touch /var/ftp/pub/file.$i; done The 'manager' interface does not hang. I can't claim to fully understand the patch though I do grok enough to see we won't block at the same place and this looks like it fixes the problem. Assuming Jakub and/or Michal is ok with it, hopefully it can be merged soon.
retrace-server-1.16-1.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-4cd8a3c0e5
retrace-server-1.16-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-81eb6f786a
retrace-server-1.16-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-d84c5abe3f
retrace-server-1.16-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-2e49e9538e
retrace-server-1.16-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-81eb6f786a
retrace-server-1.16-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-d84c5abe3f
retrace-server-1.16-1.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-4cd8a3c0e5
retrace-server-1.16-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-2e49e9538e
I checked retrace-server-1.16-1.el6 and the loading of the manager page is much improved in the presence of a large number of ftp files. I simulated the large number of ftp files by having a loop creating a lot of files in the ftp server directory and then on the retrace-server machine, trying to load the 'manager' page.
retrace-server-1.16-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
retrace-server-1.16-2.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-2e49e9538e
retrace-server-1.16-2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-81eb6f786a
retrace-server-1.16-2.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-4cd8a3c0e5
retrace-server-1.16-2.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-4cd8a3c0e5
retrace-server-1.16-2.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-2e49e9538e
retrace-server-1.16-2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-81eb6f786a
retrace-server-1.17.0-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-3d55370e77
retrace-server-1.17.0-1.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-9390d60e0d
retrace-server-1.17.0-1.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-9390d60e0d
retrace-server-1.17.0-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-3d55370e77
retrace-server-1.17.0-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.