Bug 472702
| Summary: | Fix use of python/generator.py in parallel builds | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Community] Virtualization Tools | Reporter: | Michael Marineau <marineam> | ||||
| Component: | libvirt | Assignee: | Daniel Veillard <veillard> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | unspecified | CC: | berrange, crobinso | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2009-03-12 10:54:27 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Hmm, I think this was already addressed upstream. Are you using stock libvirt 0.4.6 or was this against an upstream checkout? See https://bugzilla.redhat.com/show_bug.cgi?id=463733 for more info. I didn't notice that other bug report, but the fix posted there only partially fixed the issue. Make is still waiting for generator.py to finish but it is still running generator.py as many times as -j allows. From the current CVS: Making all in python make[2]: Entering directory `/tmp/libvirt/python' Making all in . make[3]: Entering directory `/tmp/libvirt/python' /usr/bin/python ./generator.py . /usr/bin/python ./generator.py . /usr/bin/python ./generator.py . So, it is very unlikely to get corruption but the make process is still being dumb. :-) Touching a stamp file as my patch does is an easy way to make it only run once. Hi, would you mind sending the patch to libvir-list? Things get better notice on the list. If not, I can send it along for you if you'd prefer. Okay, that should now be fixed upstream, as I commited the patch in CVS, thanks to both ! Daniel This patch is included in the 0.6.1 release |
Created attachment 324445 [details] Parallel build fix When doing a parallel build of libvirt with make -jX where X is >1 generator.py gets called several times all at once because it generates several files at the same time while make assumes each call will only generate one file. For example with make -j3: Making all in python make[2]: Entering directory `/var/tmp/portage/app-emulation/libvirt-0.4.6/work/libvirt-0.4.6/python' Making all in . make[3]: Entering directory `/var/tmp/portage/app-emulation/libvirt-0.4.6/work/libvirt-0.4.6/python' /usr/bin/python ./generator.py . /usr/bin/python ./generator.py . /usr/bin/python ./generator.py . /usr/lib64/python2.5/xmllib.py:9: DeprecationWarning: The xmllib module is obsolete. Use xml.sax instead. warnings.warn("The xmllib module is obsolete. Use xml.sax instead.", DeprecationWarning) /usr/lib64/python2.5/xmllib.py:9: DeprecationWarning: The xmllib module is obsolete. Use xml.sax instead. warnings.warn("The xmllib module is obsolete. Use xml.sax instead.", DeprecationWarning) /usr/lib64/python2.5/xmllib.py:9: DeprecationWarning: The xmllib module is obsolete. Use xml.sax instead. warnings.warn("The xmllib module is obsolete. Use xml.sax instead.", DeprecationWarning) Found 134 functions in libvirt-api.xml Found 134 functions in libvirt-api.xml Found 1 functions in libvirt-python-api.xml Generated 125 wrapper functions Missing type converters: Found 1 functions in libvirt-python-api.xml Generated 125 wrapper functions Missing type converters: cat ./libvir.py libvirtclass.py > libvirt.py-t mv libvirt.py-t libvirt.py Found 134 functions in libvirt-api.xml Found 1 functions in libvirt-python-api.xml Generated 125 wrapper functions Missing type converters: As you can see above libvirtclass.py is being read by cat and written by one of the generator processes at the same time which sometimes results in a partially truncated libvirt.py. The attached patch fixes this issue by using a dummy stamp file to force generator.py to run only once.