Bug 619733
| Summary: | fedpkg local holds rpmbuild log | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Petr Pisar <ppisar> | ||||||
| Component: | fedora-packager | Assignee: | Dennis Gilmore <dennis> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | low | ||||||||
| Version: | 13 | CC: | dcantrell, dennis, dmalcolm, dwmw2, mmaslano, sochotni | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | All | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | fedora-packager-0.5.1.2-2.fc14 | Doc Type: | Bug Fix | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2010-08-17 05:36:26 UTC | Type: | --- | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 435553 [details]
Do not redirect local rpmbuild output
This patch removes annoying rpmbuild output redirect.
Created attachment 436934 [details]
Alternate patch, which redirects/captures stdout/stderr as desired
I wrote an implementation of this which used "tee" to capture to the logfile, and also emit to both stdout and stderr.
This merged rpmbuild's stdout and stderr as seen from fedpkg's (as one stdout), and didn't preserve the "log" usage, so I tried again.
The attached patch uses helper threads to process rpmbuild's stdout and stderr separately, redirecting them to a list of file-like objects.
It sends each through to the corresponding fedpkg stream, but also captures both into the logfile.
I also added a ToLog() method, and at first I added a ToLog(log.info) to the tee for stdout, and a ToLog(log.error) to the one for stderr. However, each of these log streams seems to be set up to go to stdout/stderr of fedpkg, so it led to duplication of the data.
This (essentially) eliminates the annoying buffering, whilst capturing the logs, and allowing separate redirection from fedpkg if desired.
Tested lightly, letting builds run to completion, and hitting Ctrl-C. Sometimes need to hit Ctrl-C several times to actually halt a build. Ctrl-C-K in emacs kills it first time.
There may still be some race conditions in error handling.
Hrm, so I fixed this a slightly different way, before I saw any of your code. I think it does what you want, and it still has the log file, and ^c still seems to work. It'll be in the next build. python-kitchen-0.2-0.1.a2.el5,fedora-packager-0.5.1.2-1.el5 has been submitted as an update for Fedora EPEL 5. http://admin.fedoraproject.org/updates/python-kitchen-0.2-0.1.a2.el5,fedora-packager-0.5.1.2-1.el5 fedora-packager-0.5.1.2-1.fc14 has been submitted as an update for Fedora 14. http://admin.fedoraproject.org/updates/fedora-packager-0.5.1.2-1.fc14 fedora-packager-0.5.1.2-1.fc13 has been submitted as an update for Fedora 13. http://admin.fedoraproject.org/updates/fedora-packager-0.5.1.2-1.fc13 fedora-packager-0.5.1.2-1.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/fedora-packager-0.5.1.2-1.fc12 fedora-packager-0.5.1.2-2.fc13 has been pushed to the Fedora 13 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update fedora-packager'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/fedora-packager-0.5.1.2-2.fc13 fedora-packager-0.5.1.2-2.el5, python-kitchen-0.2-0.1.a2.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update fedora-packager python-kitchen'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/python-kitchen-0.2-0.1.a2.el5,fedora-packager-0.5.1.2-2.el5 fedora-packager-0.5.1.2-2.fc13 has been pushed to the Fedora 13 stable repository. If problems still persist, please make note of it in this bug report. fedora-packager-0.5.1.2-2.fc12 has been pushed to the Fedora 12 stable repository. If problems still persist, please make note of it in this bug report. fedora-packager-0.5.1.2-2.el5, python-kitchen-0.2-0.1.a2.el5 has been pushed to the Fedora EPEL 5 stable repository. If problems still persist, please make note of it in this bug report. fedora-packager-0.5.1.2-2.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report. |
fedora-packager-0.5.1.0-1.fc13.noarch The `fedpkg local' command does not pass rpmbuild stdout and stderr to terminal. This is regression against the dist-cvs `make local' command. In addition, the output is saved to .build*.log after rpmbuild termination. Thus there is no way how to watch rpmbuild progress. More ever the stdout and stderr redirection is handled in wrong way: try: proc = subprocess.Popen(' '.join(cmd), stderr=subprocess.PIPE, stdout=subprocess.PIPE, shell=True) output, error = proc.communicate() except OSError, e: raise FedpkgError(e) outfile = open(os.path.join(self.path, '.build-%s-%s.log' % (self.ver, self.rel)), 'w') outfile.writelines(output) log.info(output) if error: outfile.writelines(error) log.error(error) outfile.close() As you can see, the stderr and stdout is saved sequentially making rpmbuild output splitted into stdout and stderr part. The log should be multiplex both streams in line order as they come from rpmbuild. Otherwise user cannot tell which command failed. Other problem is the log file is created ex post. Let's say rpmbuild hangs, user interrupts the process (^C) to find reason in log file, and there is no log file because Python terminates before creating the file (due to SIGINT). Proposed solution: Read rpmbuild output as soon as possible and multiplex the stream lines into log file asynchronously. Also flushing the log file after each line or at least on python exit should be forced to preserve content on abort. Also copying the streams to terminal (by default or when special fedpkg option given) should be implemented. Apparently, one need to operate the log in separate thread or process.