Description of problem: abrt-gui is slow when downloading and processing debuginfo. One significant problem is using multiple passes. First the debuginfo.rpm is downloaded and written to a file. Then the .rpm file is converted to cpio, and written to a file. Then the .cpio archive is extracted into files. This is three passes when a one-pass, three-stage pipeline could be used instead: download | rpm2cpio | cpio --extract. The savings in wall-clock time would be significant, probably all overlapped with the download itself. I just got done waiting 20 minutes for debuginfo processing on bug #781729 (SIGSEGV in gnome-shell), and the wait greatly degrades my motivation to use abrt. Version-Release number of selected component (if applicable): abrt-2.0.7-3.fc17.x86_64 How reproducible: every time Steps to Reproduce: 1. Inovke abrt-gui and use local gdb to process debuginfo for an application crash. 2. 3. Actual results: The missing debuginfo for each package is downloaded, converted to cpio, and extracted to files using separate passes with temporary files between each pass. Expected results: The missing debuginfo is obtained much more quickly by using a single pass pipeline. Additional info:
(In reply to comment #0) > Description of problem: abrt-gui is slow when downloading and processing > debuginfo. One significant problem is using multiple passes. First the > debuginfo.rpm is downloaded and written to a file. Then the .rpm file is > converted to cpio, and written to a file. Then the .cpio archive is extracted > into files. This is three passes when a one-pass, three-stage pipeline could > be used instead: download | rpm2cpio | cpio --extract You are right, we do use intermediate files. But I contend you are mistaken about the impact of it: the vast majority of the time (I'd hazard to guess ~95%) is spent on downloading. A pipe download | rpm2cpio | cpio --extract will be harder to debug if (or rather, *when*) problems occur. For starters, standard shell (meaning: not using bash-specific extensions) does not even provide the means of detecting when intermediate steps in pipe exited with nonzero status. Yes, one or other way we can do that, but the small improvements aren't worth the needed significant code complications in this area. > The savings in wall-clock time would be significant, probably all overlapped with the download itself. I just got done waiting 20 minutes for debuginfo processing on bug #781729 (SIGSEGV in gnome-shell), and the wait greatly degrades my motivation to use abrt. We know about it, and we try to improve this situation, but it's not going to happen quickly...
Sometimes downloading is the bottleneck. I suggest: sort the list of debuginfo .rpm by download size, then download using two parallel threads: one processing from the small end of the list, the other processing from the large end of the list. This tends to maintain high throughput [thus earliest finish] despite per-package overhead, and with acceptable server load and traffic congestion.