Bug 532118

Summary: Reworking pythondeps.sh to support multiple parallel Python stacks
Product: [Fedora] Fedora Reporter: Dave Malcolm <dmalcolm>
Component: rpmAssignee: Panu Matilainen <pmatilai>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: rawhideCC: a.badger, ffesti, jnovy, pmatilai
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-01-30 10:43:48 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 530636    
Attachments:
Description Flags
Revised version of script
none
New version of brp-python-bytecompile none

Description Dave Malcolm 2009-10-30 18:22:48 UTC
Created attachment 366839 [details]
Revised version of script

Sent upstream here: http://lists.rpm.org/pipermail/rpm-maint/2009-October/002592.html
but filing in bz so I can add it to my "Python 3 in Fedora 13" tracker bug.

rpm.org's scripts/pythondeps.sh can be used to autogenerate Provides and
Requires items of the form:
   python(abi) = ${PYVER}
where PYVER is of the form MAJOR.MINOR

"pythondeps.sh" can take a list of buildroot paths as lines on stdin and
generate the "python(abi) = foo" line on stdout.

As I understand things, it's invoked due to these lines in
macros/macros.in:
%__python_provides %{_rpmconfigdir}/pythondeps.sh --provides
%__python_requires %{_rpmconfigdir}/pythondeps.sh --requires
(which is invoked by rpmfcHelper/rpmfcGenerateDepends in build/rpmfc.c
when "_use_internal_dependency_generator" is enabled)

Incidentally, my reading of the above code [1] is that only files that
are categorized as python files get passed to the script.  Since the
python binary is not a python file, it's not passed to the script, so
the provides line is never generated.  In Fedora's "python.spec" we get
around this by manually supplying a couple of provides:
  Provides: python-abi = %{pybasever}
  Provides: python(abi) = %{pybasever}

The current implementation invokes "python" to determine PYVER.  This
assumes that /usr/bin/python is actually installed in the environment,
which isn't the case when rebuilding a "python" rpm in a clean
environment (e.g. mock chroot).

Attached is a reimplementation of the code which allows for multiple
python implementations and removes the assumption that /usr/bin/python
is installed (although in Fedora we don't actually use this for the
provides case as described above).

The code assumes that Python binaries are of the form
    /usr/bin/pythonMAJOR.MINOR
and generate "provides" items of the form
    python(abi) = MAJOR.MINOR

Python libraries are assumed to be of the form
    /usr/lib/pythonMAJOR.MINOR/  and
    /usr/lib64/pythonMAJOR.MINOR/ 
generating (uniquely) lines of the form:
    python(abi) = MAJOR.MINOR    

Examples:
$ echo "/home/david/rpmbuild/BUILDROOT/python3-3.1.1-7.fc11.i386/usr/bin/python3.1" \
  | ./pythondeps.sh --provides
Output: python(abi) = 3.1
(though this path is never actually passed; see note above)

$ echo "/home/david/rpmbuild/BUILDROOT/python3-3.1.1-7.fc11.i386/usr/bin/python3.1-config" \
  | ./pythondeps.sh --provides
(no output)

$ echo "/home/david/rpmbuild/BUILDROOT/python3-3.1.1-7.fc11.i386/usr/lib/python3.1/test/test_bisect.py" \
  | ./pythondeps.sh --requires
Output: python(abi) = 3.1


[1] plus adding a "tee -a /tmp/whatever" to the file to capture it, for
playback into my test scripts

Comment 1 Bug Zapper 2009-11-16 14:43:07 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 12 development cycle.
Changing version to '12'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 2 Dave Malcolm 2010-01-19 23:35:06 UTC
Panu: how do-able is this for Fedora 13?   

This is needed for this Fedora 13 feature:
https://fedoraproject.org/wiki/Features/Python3F13

Comment 3 Toshio Ernie Kuratomi 2010-01-27 00:12:21 UTC
Created attachment 386947 [details]
New version of brp-python-bytecompile

This version of the script has a few bug fixes.  Tested that it works on a python2/python3 module package (python-setuptools with a python3 subpackage).  No private libraries.  Also didn't test what happens when SyntaxErrors are found.

Comment 4 Toshio Ernie Kuratomi 2010-01-27 00:17:02 UTC
Comment on attachment 366839 [details]
Revised version of script

Sorry, wrong bug

Comment 5 Panu Matilainen 2010-01-30 10:43:48 UTC
Dave, apologies for late response. IIRC there was something here that made me postpone applying it "until I've had a chance to look closer" but can't for the life of me remember what it was.

Built into rpm-4.8.0-4.fc13, if something breaks ... shrug, we'll fix it then.