Bug 783932 - rpmbuild warnings for files that include spaces when using generic auto requires/provides filtering macros
Summary: rpmbuild warnings for files that include spaces when using generic auto requi...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: redhat-rpm-config
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Panu Matilainen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 873847 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-01-23 09:55 UTC by Tadej Janež
Modified: 2012-11-09 22:48 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2012-11-09 19:41:32 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Orange's build.log on Fedora 17 (2.32 MB, text/plain)
2012-01-23 09:57 UTC, Tadej Janež
no flags Details
Patch that fixes the issue (836 bytes, patch)
2012-09-18 14:42 UTC, Tadej Janež
no flags Details | Diff

Description Tadej Janež 2012-01-23 09:55:27 UTC
Description of problem:
I tried to package a python module and I encountered the following errors:

(except from build.log)
Processing files: orange-2.5-0.1.20120119svn.fc17.x86_64
Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.2ZCJXp
+ umask 022
+ cd /builddir/build/BUILD
+ cd Orange-2.5a1
+ DOCDIR=/builddir/build/BUILDROOT/orange-2.5-0.1.20120119svn.fc17.x86_64/usr/share/doc/orange-2.5
+ export DOCDIR
+ /bin/mkdir -p /builddir/build/BUILDROOT/orange-2.5-0.1.20120119svn.fc17.x86_64/usr/share/doc/orange-2.5
+ cp -pr doc/COPYING doc/LICENSES /builddir/build/BUILDROOT/orange-2.5-0.1.20120119svn.fc17.x86_64/usr/share/doc/orange-2.5
+ exit 0
warning: Recognition of file "Qt" failed: mode 000000 cannot open `Qt' (No such file or directory)
warning: Recognition of file "Qt/OWDiscretizeQt.py" failed: mode 000000 cannot open `Qt/OWDiscretizeQt.py' (No such file or directory)
warning: Recognition of file "Qt/OWDiscretizeQt.pyc" failed: mode 000000 cannot open `Qt/OWDiscretizeQt.pyc' (No such file or directory)
warning: Recognition of file "Qt/OWDiscretizeQt.pyo" failed: mode 000000 cannot open `Qt/OWDiscretizeQt.pyo' (No such file or directory)
warning: Recognition of file "Qt/OWDistributionsQt.py" failed: mode 000000 cannot open `Qt/OWDistributionsQt.py' (No such file or directory)
warning: Recognition of file "Qt/OWDistributionsQt.pyc" failed: mode 000000 cannot open `Qt/OWDistributionsQt.pyc' (No such file or directory)
warning: Recognition of file "Qt/OWDistributionsQt.pyo" failed: mode 000000 cannot open `Qt/OWDistributionsQt.pyo' (No such file or directory)
...

The files that rpmbuild complains about have the following paths:
/usr/lib64/python2.7/site-packages/orange/OrangeWidgets/Visualize Qt/OWDiscretizeQt.py
/usr/lib64/python2.7/site-packages/orange/OrangeWidgets/Visualize Qt/OWDiscretizeQt.pyc
/usr/lib64/python2.7/site-packages/orange/OrangeWidgets/Visualize Qt/OWDiscretizeQt.pyo
/usr/lib64/python2.7/site-packages/orange/OrangeWidgets/Visualize Qt/OWDistributionsQt.py
/usr/lib64/python2.7/site-packages/orange/OrangeWidgets/Visualize Qt/OWDistributionsQt.pyc
/usr/lib64/python2.7/site-packages/orange/OrangeWidgets/Visualize Qt/OWDistributionsQt.pyo

In essence, all paths contain a space ("Visualize Qt").


Version-Release number of selected component (if applicable):
rpm-build-4.9.1.2-11.fc17.x86_64

How reproducible:
Always.

Additional info:
The same issue is also encountered on F-15 and F-16.

Comment 1 Tadej Janež 2012-01-23 09:57:27 UTC
Created attachment 556885 [details]
Orange's build.log on Fedora 17

Comment 2 Tadej Janež 2012-01-24 15:33:43 UTC
Today, I discovered that %filter_setup is at fault here.

If I remove the following lines from the SPEC file:
%{?filter_setup:
%filter_provides_in %{python_sitearch}/%{name}/.*\.so$
%filter_from_requires /liborange.so/d
%filter_setup
}

rpmbuild doesn't report any warnings anymore.

Thus, I'm changing the component to redhat-rpm-config.

If it helps, I can provide the build logs and SRPMS of the software I'm packaging.

Comment 3 Robin Lee 2012-08-11 08:49:38 UTC
I met this issue too, under up-to-update F17.

Comment 4 Tadej Janež 2012-09-18 14:42:16 UTC
Created attachment 614038 [details]
Patch that fixes the issue

The $FILE argument has to be in double quotes so it works for file names with spaces. The double quotes in the __find_provides and __find_requires commands were replaced with single quotes for proper shell quotes nesting.

Comment 5 Toshio Ernie Kuratomi 2012-10-03 17:00:09 UTC
I notice that other filter macros which are used in __find_provides and __find_requires currently use single quotes.  Will those need to be changed as well?

For instance:

%filter_provides_in(P) %{expand: \
%global __filter_prov_cmd %{?__filter_prov_cmd} %{__grep} -v %{-P} '%*' | \
}

Comment 6 Tadej Janež 2012-10-04 10:10:03 UTC
(In reply to comment #5)
> I notice that other filter macros which are used in __find_provides and
> __find_requires currently use single quotes.  Will those need to be changed
> as well?

To my understanding, they don't need to be changed.

For example, in my SPEC file I have the following filter set up:
%{?filter_setup:
%filter_provides_in %{python_sitearch}/%{name}/.*\.so$
%filter_from_requires /liborange.so/d
%filter_setup
}

During the building of the RPM, these macros become the following commands:
Finding  Provides: /bin/sh -c ' /usr/bin/grep -v  '/usr/lib64/python2.7/site-packages/orange/.*\.so$' | while read FILE; do /usr/lib/rpm/rpmdeps -P "${FILE}"; done | /bin/sort -u '
Finding  Requires: /bin/sh -c '  while read FILE; do /usr/lib/rpm/rpmdeps -R "${FILE}"; done | /bin/sort -u  | /usr/bin/sed -e '/liborange.so/d''

So, '%*' in %global __filter_prov_cmd %{?__filter_prov_cmd} %{__grep} -v %{-P} '%*' | \ gets expanded to '/usr/lib64/python2.7/site-packages/orange/.*\.so$' and the nested single quotes appear to work ok.

Toshio, thank you for fixing the long standing bugs in redhat-rpm-config!
Will you also include the fix for this bug in the update?

Comment 7 Toshio Ernie Kuratomi 2012-10-04 19:35:31 UTC
I may end up pushing it only to rawhide but my intention is to get it into a package so we can start testing that it works.

Thanks for your expansion!  I think it shows a bug.  If the pattern given to filter_provides_in ends up having spaces then grep (and sed in the filter_from_requires) will end up with multiple arguments.

If you have time to test that corner case, it would help me out.  If not, I can get around to it after a deadline I have for a different project for Tuesday next week.

Comment 8 Tadej Janež 2012-10-05 09:34:34 UTC
(In reply to comment #7)
> I may end up pushing it only to rawhide but my intention is to get it into a
> package so we can start testing that it works.

OK, makes sense.
 
> Thanks for your expansion!  I think it shows a bug.  If the pattern given to
> filter_provides_in ends up having spaces then grep (and sed in the
> filter_from_requires) will end up with multiple arguments.
 
> If you have time to test that corner case, it would help me out.  If not, I
> can get around to it after a deadline I have for a different project for
> Tuesday next week.

I tried testing that corner case. I changed my filter setup to:
%{?filter_setup:
%filter_provides_in %{python_sitearch}/test a space/%{name}/.*\.so$
%filter_from_requires /lib with space.so/d
%filter_setup
}

During rpmbuild, this got translated to (excerpt from build log):
Finding  Provides: /bin/sh -c ' /usr/bin/grep -v  '/usr/lib64/python2.7/site-packages/test a space/orange-test/.*\.so$' | while read FILE; do /usr/lib/rpm/rpmdeps -P "${FILE}"; done | /bin/sort -u '
/usr/bin/sed: -e expression #1, char 4: unterminated address regex
Finding  Requires: /bin/sh -c '  while read FILE; do /usr/lib/rpm/rpmdeps -R "${FILE}"; done | /bin/sort -u  | /usr/bin/sed -e '/lib with space.so/d''

So, the grep command looks fine, while the sed command produces an error.
I'm really a shell or sed expert to know what would be the correct solution for this.

Comment 9 Toshio Ernie Kuratomi 2012-11-08 03:22:26 UTC
Does this fix it for you?


# actually set up the filtering bits 
%filter_setup %{expand: \
%global _use_internal_dependency_generator 0 \
%global __deploop() while read FILE; do echo "${FILE}" | /usr/lib/rpm/rpmdeps -%{1}; done | /bin/sort -u \
%global __find_provides /bin/sh -c "%{?__filter_prov_cmd} %{__deploop P} %{?__filter_from_prov}" \
%global __find_requires /bin/sh -c "%{?__filter_req_cmd}  %{__deploop R} %{?__filter_from_req}" \
}

Note that all quotes are double quotes.  Panu suggested it in: https://bugzilla.redhat.com/show_bug.cgi?id=873847  which I believe is a duplicate of this.

Comment 10 Panu Matilainen 2012-11-08 05:17:39 UTC
*** Bug 873847 has been marked as a duplicate of this bug. ***

Comment 11 Tadej Janež 2012-11-09 09:31:37 UTC
(In reply to comment #9)
> Does this fix it for you?

Yes, it does.

> # actually set up the filtering bits 
> %filter_setup %{expand: \
> %global _use_internal_dependency_generator 0 \
> %global __deploop() while read FILE; do echo "${FILE}" |
> /usr/lib/rpm/rpmdeps -%{1}; done | /bin/sort -u \
> %global __find_provides /bin/sh -c "%{?__filter_prov_cmd} %{__deploop P}
> %{?__filter_from_prov}" \
> %global __find_requires /bin/sh -c "%{?__filter_req_cmd}  %{__deploop R}
> %{?__filter_from_req}" \
> }

I've used the above code and repeated the experiment from comment #8. Not it expands to this:
Finding  Provides: /bin/sh -c " /usr/bin/grep -v  '/usr/lib64/python2.7/site-packages/test a space/orange-test/.*\.so$' | while read FILE; do echo "${FILE}" | /usr/lib/rpm/rpmdeps -P; done | /bin/sort -u "
Finding  Requires: /bin/sh -c "  while read FILE; do echo "${FILE}" | /usr/lib/rpm/rpmdeps -R; done | /bin/sort -u  | /usr/bin/sed -e '/lib with space.so/d'"
Provides: corn.so()(64bit) orangene.so()(64bit) orangeom.so()(64bit) orange.so()(64bit) statc.so()(64bit)

I didn't see any errors.

Comment 12 Toshio Ernie Kuratomi 2012-11-09 19:41:32 UTC
This is now built for rawhide:

http://koji.fedoraproject.org/koji/buildinfo?buildID=365437

I may push it back to F18 as well.  I want to see if we can fix #872737 first, though... that would be the last of the low hanging fruit that's currently opened against redhat-rpm-config

Comment 13 Fedora Update System 2012-11-09 22:48:57 UTC
redhat-rpm-config-9.1.0-37.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/redhat-rpm-config-9.1.0-37.fc18


Note You need to log in before you can comment on or make changes to this bug.