Created attachment 1770254 [details] Patch to redhat/macros and redhat/brp-python-bytecompile Description of problem: If %_topdir contains spaces, then RPM package builds can fail for different reasons at the install stage and the python byte-compilation stage. Most of the issues are simply due to missing quotes around $RPM_BUILD_ROOT. There is a more colourful problem in the python byte-compilation, where the helper script brp-python-bytecompile uses a bash construct "for python_libdir in `find ...`". This is less trivial to fix because the for/find construct cannot distinguish between spaces within filenames versus newlines that separate multiple find results, and quotes alone cannot fix this. I have attached a patch that fix both problems, patched and successfully tested on Fedora 33. This patches two files: /usr/lib/rpm/redhat/macros (for the install issue), and /usr/lib/rpm/redhat/brp-python-bytecompile (for the byte-compilation issue). Most of the fixes just involve adding quotes around $RPM_BUILD_ROOT or $python_libdir. As for the more complex for/find construct, I have fixed this by using "find -print0" to separate multiple find results with nulls, and then replacing the for loop with a "while ... read" loop that uses nulls to separate its inputs. This means that spaces in filenames are successfully preserved, whilst multiple find results are still successfully separated. How reproducible: Consistently reproducible; just use a topdir with spaces and then build any spec file with an install phase and that ships python code. To see the full effect of the patch to /usr/lib/rpm/redhat/macros, the topdir should have spaces in its *dirname* (so use a topdir of "/home/bab/foo bar/rpmbuild", not just "/home/bab/foo bar"). Additional info: For me at least the bug was practical, not just theoretical - I encountered this when building Fedora packages in a continuous integration system, where the RPM build directories were beneath machine-specific directories with names such as "Fedora 33", "Fedora 32", etc. I noticed there was also a different file /usr/lib/rpm/brp-python-bytecompile (not in the redhat/ subdirectory), shipped with the rpm package (not redhat-rpm-config). This other script has similar problems to the one that I have patched here; however, I left it alone because as far as I can tell it is never used. Nevertheless, it does appear to be fixable in a similar manner, and this might or might not be worth forwarding to the RPM upstream.
> Nevertheless, it does appear to be fixable in a similar manner, and this might or might not be worth forwarding to the RPM upstream. The file was removed from RPM upstream and moved to: https://github.com/rpm-software-management/python-rpm-packaging/blob/main/scripts/brp-python-bytecompile which should be more or less kept in sync with what is in redhat-rpm-config.
I'Ve opened a pull request with the proposed changes: https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/127
I've tried to reproduce the issue and I can confirm that a folder with a space in its name can cause the mentioned issues: $ ls test/ one 'thr e e' 'tw o' $ for dir in `find ./test -type d`; do echo $dir ; done ./test ./test/thr e e ./test/tw o ./test/one But the mentioned way with the while loop does not work for me and I honestly don't know why: $ while IFS= read -r -d '' dir; do echo $dir; done < <(find ./test -type d) <nothing> but it works when I omit the delimiter specification: $ while IFS= read -r dir; do echo $dir; done < <(find ./test -type d) ./test ./test/thr e e ./test/tw o ./test/one Anyway, I think that better might be to make it less cryptic with just a basic read and one pipe: $ find ./test -type d | while read dir; do echo $dir; done ./test ./test/thr e e ./test/tw o ./test/one Do you know about any reason why the mentioned way might be worse than what you proposed?
Hello. Are you still interested in this?
Hi - sorry, yes, still very much interested in this. I expect the reason your test didn't work was because you didn't include "-print 0" in your find statement. The delimiter -d '' used in the read statement tells bash to expect null delimiters, and so the find statement needs to produce them to match. The reason for my more convoluted suggestion was to avoid the pipe, so that the inside of the loop didn't run inside a new subshell (which is different from how the original code runs, and I didn't want to cause unexpected problems with things like variable assignments, exit codes and so on). But this was just me being extra cautious, and perhaps your simpler solution that pipes the inside of the loop into a subshell works just fine. Best regards - Ben.
^^ oops, there was a typo in comment 5 posted just now: I meant "-print0", not "-print 0", in the find statement.
Thanks for the explanation and the reasoning. To sum it up, we have two possibilities for how to implement the loop: The first one is more complex to read but runs all the commands in the main shell: while IFS= read -r -d '' python_libdir; do echo $python_libdir # do stuff done < <(find "/usr/lib" -type d -print0|grep -z -E "/(usr|app)/lib(64)?/python[0-9]\.[0-9]+$") The second one is shorter and easier to read but runs all the commands in a subshell: find "/usr/lib" -type d -print0|grep -z -E "/(usr|app)/lib(64)?/python[0-9]\.[0-9]+$" | while read -d "" python_libdir; do echo $python_libdir; # do stuff done I honestly don't have a favorite candidate. I don't think that running the commands in a subshell causes any issues but I also don't consider the first candidate too complex to reject it. @mhroncok what do you think?
I'd prefer to maintain the second option, but it is not a strong opinion.
The pull request is ready from my point of view.
FEDORA-2021-d3e0183946 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-d3e0183946
FEDORA-2021-d3e0183946 has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.