Bug 1947416 - Red Hat RPM macros break package builds where %topdir contains spaces
Summary: Red Hat RPM macros break package builds where %topdir contains spaces
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: redhat-rpm-config
Version: 33
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Florian Festi
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-08 12:42 UTC by Ben Burton
Modified: 2021-06-28 10:52 UTC (History)
10 users (show)

Fixed In Version: redhat-rpm-config-189-1.fc35
Clone Of:
Environment:
Last Closed: 2021-06-28 10:52:36 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Patch to redhat/macros and redhat/brp-python-bytecompile (2.37 KB, patch)
2021-04-08 12:42 UTC, Ben Burton
no flags Details | Diff

Description Ben Burton 2021-04-08 12:42:30 UTC
Created attachment 1770254 [details]
Patch to redhat/macros and redhat/brp-python-bytecompile

Description of problem:

If %_topdir contains spaces, then RPM package builds can fail for different reasons at the install stage and the python byte-compilation stage.

Most of the issues are simply due to missing quotes around $RPM_BUILD_ROOT.  There is a more colourful problem in the python byte-compilation, where the helper script brp-python-bytecompile uses a bash construct "for python_libdir in `find ...`".  This is less trivial to fix because the for/find construct cannot distinguish between spaces within filenames versus newlines that separate multiple find results, and quotes alone cannot fix this.

I have attached a patch that fix both problems, patched and successfully tested on Fedora 33.  This patches two files: /usr/lib/rpm/redhat/macros (for the install issue), and /usr/lib/rpm/redhat/brp-python-bytecompile (for the byte-compilation issue).  Most of the fixes just involve adding quotes around $RPM_BUILD_ROOT or $python_libdir.  As for the more complex for/find construct, I have fixed this by using "find -print0" to separate multiple find results with nulls, and then replacing the for loop with a "while ... read" loop that uses nulls to separate its inputs.  This means that spaces in filenames are successfully preserved, whilst multiple find results are still successfully separated.

How reproducible:

Consistently reproducible; just use a topdir with spaces and then build any spec file with an install phase and that ships python code.  To see the full effect of the patch to /usr/lib/rpm/redhat/macros, the topdir should have spaces in its *dirname* (so use a topdir of "/home/bab/foo bar/rpmbuild", not just "/home/bab/foo bar").

Additional info:

For me at least the bug was practical, not just theoretical - I encountered this when building Fedora packages in a continuous integration system, where the RPM build directories were beneath machine-specific directories with names such as "Fedora 33", "Fedora 32", etc.

I noticed there was also a different file /usr/lib/rpm/brp-python-bytecompile (not in the redhat/ subdirectory), shipped with the rpm package (not redhat-rpm-config). This other script has similar problems to the one that I have patched here; however, I left it alone because as far as I can tell it is never used.  Nevertheless, it does appear to be fixable in a similar manner, and this might or might not be worth forwarding to the RPM upstream.

Comment 1 Miro Hrončok 2021-04-08 13:04:31 UTC
>  Nevertheless, it does appear to be fixable in a similar manner, and this might or might not be worth forwarding to the RPM upstream.

The file was removed from RPM upstream and moved to: https://github.com/rpm-software-management/python-rpm-packaging/blob/main/scripts/brp-python-bytecompile which should be more or less kept in sync with what is in redhat-rpm-config.

Comment 2 Miro Hrončok 2021-04-19 19:05:52 UTC
I'Ve opened a pull request with the proposed changes: https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/127

Comment 3 Lumír Balhar 2021-04-23 10:51:44 UTC
I've tried to reproduce the issue and I can confirm that a folder with a space in its name can cause the mentioned issues:

$ ls test/
 one  'thr e e'  'tw o'

$ for dir in `find ./test -type d`; do echo $dir ; done
./test
./test/thr
e
e
./test/tw
o
./test/one

But the mentioned way with the while loop does not work for me and I honestly don't know why:

$ while IFS= read -r -d '' dir; do echo $dir; done < <(find ./test -type d)
<nothing>

but it works when I omit the delimiter specification:

$ while IFS= read -r dir; do echo $dir; done < <(find ./test -type d)
./test
./test/thr e e
./test/tw o
./test/one

Anyway, I think that better might be to make it less cryptic with just a basic read and one pipe:

$ find ./test -type d | while read dir; do echo $dir; done
./test
./test/thr e e
./test/tw o
./test/one

Do you know about any reason why the mentioned way might be worse than what you proposed?

Comment 4 Lumír Balhar 2021-05-19 05:48:35 UTC
Hello.

Are you still interested in this?

Comment 5 Ben Burton 2021-05-20 01:25:48 UTC
Hi - sorry, yes, still very much interested in this.

I expect the reason your test didn't work was because you didn't include "-print 0" in your find statement. The delimiter -d '' used in the read statement tells bash to expect null delimiters, and so the find statement needs to produce them to match.

The reason for my more convoluted suggestion was to avoid the pipe, so that the inside of the loop didn't run inside a new subshell (which is different from how the original code runs, and I didn't want to cause unexpected problems with things like variable assignments, exit codes and so on).  But this was just me being extra cautious, and perhaps your simpler solution that pipes the inside of the loop into a subshell works just fine.

Best regards - Ben.

Comment 6 Ben Burton 2021-05-20 01:27:54 UTC
^^ oops, there was a typo in comment 5 posted just now: I meant "-print0", not "-print 0", in the find statement.

Comment 7 Lumír Balhar 2021-06-01 06:15:09 UTC
Thanks for the explanation and the reasoning.

To sum it up, we have two possibilities for how to implement the loop:

The first one is more complex to read but runs all the commands in the main shell:

while IFS= read -r -d '' python_libdir;
do
  echo $python_libdir
  # do stuff
done < <(find "/usr/lib" -type d -print0|grep -z -E "/(usr|app)/lib(64)?/python[0-9]\.[0-9]+$")

The second one is shorter and easier to read but runs all the commands in a subshell:

find "/usr/lib" -type d -print0|grep -z -E "/(usr|app)/lib(64)?/python[0-9]\.[0-9]+$" | while read -d "" python_libdir;
do
  echo $python_libdir;
  # do stuff
done

I honestly don't have a favorite candidate. I don't think that running the commands in a subshell causes any issues but I also don't consider the first candidate too complex to reject it.

@mhroncok what do you think?

Comment 8 Miro Hrončok 2021-06-01 07:56:03 UTC
I'd prefer to maintain the second option, but it is not a strong opinion.

Comment 9 Lumír Balhar 2021-06-02 12:06:30 UTC
The pull request is ready from my point of view.

Comment 10 Fedora Update System 2021-06-28 10:43:48 UTC
FEDORA-2021-d3e0183946 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-d3e0183946

Comment 11 Fedora Update System 2021-06-28 10:52:36 UTC
FEDORA-2021-d3e0183946 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.