Bug 220766 - Review Request: ScientificPython - a collection of Python modules that are useful for scientific computing
Review Request: ScientificPython - a collection of Python modules that are u...
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: Package Review (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Chitlesh GOORAH
Fedora Package Reviews List
:
Depends On:
Blocks: FE-ACCEPT
  Show dependency treegraph
 
Reported: 2006-12-26 02:31 EST by Jef Spaleta
Modified: 2012-05-15 09:29 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-12-31 03:31:40 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
limburgher: fedora‑cvs+


Attachments (Terms of Use)

  None (edit)
Description Jef Spaleta 2006-12-26 02:31:19 EST
Spec URL: http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython.spec

SRPM URL: 
http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython-2.6-2.src.rpm

Description:
ScientificPython is a collection of Python modules that are useful for scientific computing. In this collection you will find modules that cover basic geometry (vectors, tensors, transformations, vector and tensor fields), quaternions, automatic derivatives, (linear) interpolation, polynomials, elementary statistics, nonlinear least-squares fits, unit calculations, Fortran-compatible text formatting, 3D visualization via VRML, and two Tk widgets for simple line plots and 3D wireframe models. There are also interfaces to the netCDF library (portable structured binary files), to MPI (Message Passing Interface, message-based parallel programming), and to BSPlib (Bulk Synchronous Parallel programming)
Comment 1 Jef Spaleta 2006-12-26 04:23:09 EST
Updated srpm and spec:
http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython-2.6-3.src.rpm
http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython.spec

Changelog
* Mon Dec 25 2006 Jef Spaleta <jspaleta@gmail.com> 2.6-3
- Add tk and qt subpackages
- Add bsp and mpi subpackages
- Requires clean-ups for all subpackages
- Patch to correctly locate netcdf.a and netcdf.h

* Mon Dec 25 2006 Jef Spaleta <jspaleta@gmail.com> 2.6-2
- Add docs subpackage

* Sun Dec 24 2006 Jef Spaleta <jspaleta@gmail.com> 2.6-1
- Initial ScientificPython Package
Comment 2 Chitlesh GOORAH 2006-12-27 08:04:43 EST
Some rpmlints issues which need to be corrected:

rpmlint /home/chitlesh/rpmbuild/SRPMS/ScientificPython-2.6-3.src.rpm
W: ScientificPython summary-not-capitalized a collection of Python modules that
are useful for scientific computing
E: ScientificPython description-line-too-long ScientificPython is a collection
of Python modules that are useful for scientific computing. In this collection
you will find modules that cover basic geometry (vectors, tensors,
transformations, vector and tensor fields), quaternions, automatic derivatives,
(linear) interpolation, polynomials, elementary statistics, nonlinear
least-squares fits, unit calculations, Fortran-compatible text formatting, 3D
visualization via VRML, and two Tk widgets for simple line plots and 3D
wireframe models. There are also interfaces to the netCDF library (portable
structured binary files), to MPI (Message Passing Interface, message-based
parallel programming), and to BSPlib (Bulk Synchronous Parallel programming)

chitlesh(SPECS)[1]$rpmlint
/home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-mpi-2.6-3.i386.rpm
E: ScientificPython-mpi description-line-too-long This package contains the
ScientificPython mpi enabled python intepreter and associated modules

rpmlint /home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-bsp-2.6-3.i386.rpm
E: ScientificPython-bsp description-line-too-long This package contains the
necessary ScientificPython modules for virtual bsp. This is useful for running
multiple virtual processes in a bsp manner on a single cpu.

chitlesh(SPECS)[1]$rpmlint
/home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-tk-2.6-3.i386.rpm
W: ScientificPython-tk summary-not-capitalized tk widgets from ScientificPython

chitlesh(SPECS)[1]$rpmlint
/home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-qt-2.6-3.i386.rpm
W: ScientificPython-qt summary-not-capitalized qt widgets from ScientificPython


shouldn't the sub packages be a dependency of the ScientificPython package ?

you missed Doc/BSP_Tutorial.pdf in ScientificPython-doc package

Comment 3 Jef Spaleta 2006-12-27 15:08:34 EST
(In reply to comment #2)
I'll clean up the descriptive tag crap before I submit.

> shouldn't the sub packages be a dependency of the ScientificPython package ?
> 

No, its the other way around.  Subpackages depend on the main package.  I broke
these out as sub-packages specifically because they drag in additional
requirements which may or may not be needed and as a result they should be
optional. This is a codebase aimed at people writing homebrew scientific
simulation code, not an end-user application. I expect everyone using this to
have enough grey matter to look for subpackages as needed.  I've no desire to
delibrately force all subpackages to install dragging in tk and qt and openmpi
on every system, systems which be delibrately streamlined for batched scientific
computing. You'll notice this sort of thing is already done for
python-matplotlib and python-matplotlib-tk so I'm not setting a precendent here. 

> you missed Doc/BSP_Tutorial.pdf in ScientificPython-doc package

Crap thats suppose to be in -BSP subpackage, I missed it when I split off BSP.
I'm on the fence about the BSP stuff in general because libBSP is not available
in Fedora yet. I'm not even sure what the licensing conditions on libBSP are. 
The only reason I'm including the BSP python modules at all is because
ScientificPython includes a virtual BSP utility which allows you to simulate the
use of the BSP protocal on a single processor without the need of libBSP. Cute,
but I'm not sure how useful packaging that actually is. Since I've no experience
with libBSP installs yet I wasn't going to hold up packaging ScientificPython
for this optional functionality. I'm primarily interested in the provided netCDF
support, and secondarily the mpi support.  I probably need to add a
README.Fedora to the -bsp subpackage stating that the libBSP support isn't
available yet.


-jef
Comment 4 Jef Spaleta 2006-12-28 02:14:04 EST
Okay here's the latest version of the package. This should take care of
everything brought up so far in the comments.

http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython-2.6-4.src.rpm

http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython.spec

Changelog:
* Wed Dec 27 2006 Jef Spaleta <jspaleta@gmail.com> 2.6-4
- move impipython to docs section of mpi subpackage 
-   this is a script which must be editted by hand per system.
- Added Doc/BSP_Tutorial.pdf to docs subpackage
- Fixed description text line wrapping.

rpmlint warnings from All binary packages under mock development:
W: ScientificPython* invalid-license CeCILL
  This is a bogus warning. CeCILL is explicitly compatible with the GPL 
  http://www.fsf.org/licensing/licenses/index_html#GPLCompatibleLicenses
W: ScientificPython-devel no-documentation
  Bogus, I made a seperate docs package to hold the html and pdf documentation
W: ScientificPython-qt no-documentation
  Bogus
W: ScientificPython-tk no-documentation
  Bogus

W: ScientificPython-mpi doc-file-dependency
/usr/share/doc/ScientificPython-mpi-2.6/impipython /bin/csh
  Do I need to strip the executable bits from this file.  I've moved this script
into documentation because its not strictly necessary for operation and is meant
as a troubleshooting aid when using the mpi capability. This is stated in the
README.MPI file included in the mpi subpackage docs.  In any event, this has to
be hand editted to pass the correct number of processors to use to mpirun so its
system dependant. Anyone have thoughts concerning any additional effort I need
to make with the impipython script?



-jef
Comment 5 Chitlesh GOORAH 2006-12-28 05:58:19 EST
I'll review it
Comment 6 Chitlesh GOORAH 2006-12-28 06:51:01 EST
# for the ScientificPython-mpi package:

rpm -qlvp /home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-mpi-2.6-4.i386.rpm
-rwxr-xr-x    1 root    root          1090528 Dec 28 12:39 /usr/bin/mpipython
[..]
-rwxr-xr-x    1 root    root              111 Oct  6 12:49
/usr/share/doc/ScientificPython-mpi-2.6/impipython

contents of : /usr/share/doc/ScientificPython-mpi-2.6/impipython
#!/bin/csh
mpirun -np 2 /usr/local/bin/mpipython
/usr/lib/python2.1/site-packages/Scientific/BSP/Console.py $*

you should adjust the path of mpipython since it is found at /usr/bin/mpipython
instead of /usr/local/bin/mpipython 

/usr/lib/python2.1/site-packages/Scientific/BSP/Console.py is incorrect as well
since the 
rpm -qvlp /home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-bsp-2.6-4.i386.rpm
gives
-rw-r--r--    1 root    root             5278 Oct  6 12:49
/usr/lib/python2.4/site-packages/Scientific/BSP/Console.py
[..]

Hence the ScientificPython-bsp is a dependecy of ScientificPython-mpi

in your spec file you should also add tcsh as requires for the
ScientificPython-mpi package, like 
Requires:       openmpi-libs tcsh ScientificPython-bsp
Comment 7 Chitlesh GOORAH 2006-12-28 06:54:39 EST
(In reply to comment #6)
> /usr/lib/python2.1/site-packages/Scientific/BSP/Console.py is incorrect as well
> since the 
> rpm -qvlp /home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-bsp-2.6-4.i386.rpm
> gives
> -rw-r--r--    1 root    root             5278 Oct  6 12:49
> /usr/lib/python2.4/site-packages/Scientific/BSP/Console.py
> [..]

Beware in rawhide, it will be
usr/lib/python2.5/site-packages/Scientific/BSP/Console.py
Comment 8 Chitlesh GOORAH 2006-12-28 07:09:37 EST
(In reply to comment #3)
> I probably need to add a README.Fedora to the -bsp subpackage stating that the 
> libBSP support isn't available yet.

Or perhaps document it somewhere in the fedoraproject.org/wiki :)
Since in the /usr/share/doc/ScientificPython-mpi-2.6/README.MPI, it states,

The module Scientific.MPI is documented in the ScientificPython
manual. The main purpose of this file is to explain how to install
ScientificPython with MPI support.[...]

You could possibly add useful information from that file to the wiki page
without those installation notes. Then
/usr/share/doc/ScientificPython-mpi-2.6/README.MPI might be useless, afterwards 
/usr/share/doc/ScientificPython-mpi-2.6/impipython could be at /usr/bin/impipython

What do you think ?
Since, I'm concerned too with scientific packages at Fedora, possibly we could
gather some packagers (scientific packages) to document their changes during
packaging on the wiki.

http://fedoraproject.org/wiki/Extras/SIGs/SciTech
We already have SciTech SIG, we could bring it to life, just like the PHP SIG is
doing a great job. And help each other during reviews, troubleshooting, etc...
Comment 9 Chitlesh GOORAH 2006-12-28 07:15:36 EST
# for the qt package

Requires:       pyQt

The real name is PyQt is not yum won't be able to find that dependency :)
Comment 10 Jef Spaleta 2006-12-28 12:47:46 EST
(In reply to comment #6)
> # for the ScientificPython-mpi package:
> 
> rpm -qlvp /home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-mpi-2.6-4.i386.rpm
> -rwxr-xr-x    1 root    root          1090528 Dec 28 12:39 /usr/bin/mpipython
> [..]
> -rwxr-xr-x    1 root    root              111 Oct  6 12:49
> /usr/share/doc/ScientificPython-mpi-2.6/impipython
> 
> contents of : /usr/share/doc/ScientificPython-mpi-2.6/impipython
> #!/bin/csh
> mpirun -np 2 /usr/local/bin/mpipython
> /usr/lib/python2.1/site-packages/Scientific/BSP/Console.py $*


> 
> you should adjust the path of mpipython since it is found at /usr/bin/mpipython
> instead of /usr/local/bin/mpipython 

I noticed this... it was yet another reason I put this into the documentation
section.  It has to be hand adjusted for number of cpus to use regardless.

I'm just going to end up just replacing this file completely and generating a
new one. The patch for this file would be longer than the inline cat command to
produce a new one.

> 
> /usr/lib/python2.1/site-packages/Scientific/BSP/Console.py is incorrect as
well since the  rpm -qvlp
/home/chitlesh/rpmbuild/RPMS/i386/ScientificPython-bsp-2.6-4.i386.rpm

> Hence the ScientificPython-bsp is a dependecy of ScientificPython-mpi
I think I'm going to merge the bsp and mpi stuff into one subpackage. Should I
just call the new subpackage ScientificPython-mpi or should I be more
encompassing and call it ScientificPython-parallelprocessing ?  If we ever get
libbsp in Fedora Extras I can build the additional libbsp support into the
subpackage. Once you are doing parallelization having libbsp installed is
probably acceptable. 


> 
> in your spec file you should also add tcsh as requires for the
> ScientificPython-mpi package, like 
> Requires:       openmpi-libs tcsh ScientificPython-bsp

technically I dont think so since the script is placed in as part of the
documetnation.  It is a reference script, its not critical.. I would even call
it trivial. No matter what you do you have to edit this by hand to at least set
the number of processors for mpirun to use. My understanding is that reference
scripts or examples included as documentation in a packages %doc section do not
need to include their intepreter as a hard requirement on the package.

Comment 11 Jef Spaleta 2006-12-28 12:58:33 EST
(In reply to comment #8)
> Or perhaps document it somewhere in the fedoraproject.org/wiki :)
> Since in the /usr/share/doc/ScientificPython-mpi-2.6/README.MPI, it states,> 
> The module Scientific.MPI is documented in the ScientificPython
> manual. The main purpose of this file is to explain how to install
> ScientificPython with MPI support.[...]
> 
> You could possibly add useful information from that file to the wiki page
> without those installation notes. 

No, I'm not thrilled at using the wiki as the primary source of information. If
you want to make a scientific computing wiki section that provides more
expansive information across the spectrum of available software tools... go
right ahead. But my primary concern is to provide the necessary information in
the packaging documentation for this package.  Other packages already use
README.Fedora files to document Fedora specific changes. If you have suggestions
on what I should be including in a README.Fedora file, I'll do that. But I'm not
going to try to divide my time between the package CVS and the wiki for package
specific documentation.

Then
> /usr/share/doc/ScientificPython-mpi-2.6/README.MPI might be useless, afterwards 
> /usr/share/doc/ScientificPython-mpi-2.6/impipython could be at /usr/bin/impipython

No impipython cannot be in /usr/bin/  because you still have to set the number
of processors to use with mpirun by hand. There is no way around it, the script
is clearly a trivial reference script. I'm not going to work on enhancing this
script into a generally useful executable as part of downstream packaging activity. 

> 
> What do you think ?
> Since, I'm concerned too with scientific packages at Fedora, possibly we could
> gather some packagers (scientific packages) to document their changes during
> packaging on the wiki.
> 
> http://fedoraproject.org/wiki/Extras/SIGs/SciTech
> We already have SciTech SIG, we could bring it to life, 

I'm not interested in participating in any coordinated SIG activities at this
point. 
Comment 12 Jef Spaleta 2006-12-29 01:49:34 EST
Here is the updated srpm to address outstanding issues. PyQt requires has been
fixed as well as removal of redudant requires: openmpi-libs and netcdf.

I've re-integrated the mpi and bsp subpackages back into the main package.
Picking up the depchain for openmpi-libs isn't a significant burden. But I still
feel its appropriate to leave the Qt and tk toolkit modules as subpackages.

I've also spun up a fedora-specific impipython.sh file to replace the upstream
one. It has the correct filepath information generated at package buildtime.
It's still included as a doc file, because you still have to hand edit it for
the number or processors you want to run it with via mpirun.  You'll notice I
added a nice verbose comment header in the script explaining what its there for.

This is my best effort to cover all the problems pointed out by Chitlesh. Is
there anything else I need to fix?

I've look a bit more at the Visualization subdirectory of included python code.
At the moment they pretty much require someone to have additional non Fedora
space items installs to operate correctly. They do have runtime detection of the
needed helper programs so they don't fall over and die with python tracebacks.
They should exit gracefully and tell you you don't have the needed additional
software installed. I'm still inclined to include them since they do no harm,
but I'm not sure if I should split this off as a subpackage. There is no real
gain in splitting them off as a subpackage at the moment. If at some point we
get the 3d visualization programs in Fedora packaging space we may consider it
if we want to make one the 3d visualization stacks a hard requirement on the
package,  I don't consider this a blocker issue, but I'm open to dealing with
this a different way if the reviewer(s) think otherwise.

SRPM:
http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython-2.6-5.src.rpm
Spec:
http://jspaleta.thecodergeek.com/Fedora%20SRPMS/ScientificPython/ScientificPython.spec

Changelog:
* Thu Dec 28 2006 Jef Spaleta <jspaleta@gmail.com> 2.6-5
- remove mpi and bsp subpackages. On more thought, 
-   it makes more sense to have the parallel computing items
-   in the main package.
- Added inline impipython.sh reference script
-   Replaces upstreams impipython reference file
-   This will have the correct path statements generated at
-   package buildtime. Still included as a doc item
Comment 13 Chitlesh GOORAH 2006-12-30 06:57:02 EST
MUST Items:

- MUST: rpmlint's output isn' clean but can be ignored, since CeCILL is a free
software license, explicitly compatible with the GNU GPL.
http://www.cecill.info/licences.en.html
- MUST: The package is named according to the Package Naming Guidelines.
- MUST: The spec file name matches the base package %{name}
- MUST: The package meets the Packaging Guidelines.
- MUST: The package is licensed (CeCill) with an open-source compatible license
and meet other legal requirements as defined in the legal section of Packaging
Guidelines.
- MUST: The License field in the package spec file matches the actual license.
- MUST: the source package includes the text of the license(s) in its own file,
then that file, containing the text of the license(s) for the package is
included in %doc.
- MUST: The spec file must be written in American English.
- MUST: The spec file for the package is be legible. 
- MUST: The sources used to build the package must matches the upstream source,
as provided in the spec URL.
- MUST: The package successfully compiles and builds into binary rpms on at
least i386.
- MUST: All build dependencies is listed in BuildRequires.
- MUST: If the package does not contain shared library files located in the
dynamic linker's default paths
- MUST: the package is not designed to be relocatable
- MUST: the package owns all directories that it creates.
- MUST: the package does not contain any duplicate files in the %files listing.
- MUST: Permissions on files are set properly.
- MUST: The package has a %clean section, which contains rm -rf %{buildroot} (or
$RPM_BUILD_ROOT).
- MUST: The package consistently uses macros, as described in the macros section
of Packaging Guidelines.
- MUST: The package contains code, or permissable content. This is described in
detail in the code vs. content section of Packaging Guidelines.
- MUST: There are no Large documentation files
- MUST: %doc does not affect the runtime of the application. To summarize: If it
is in %doc, the program must run properly if it is not present.
- MUST: There are no Header files or static libraries 
- MUST: The package does not contain library files with a suffix 
- MUST: Package does NOT contain any .la libtool archives
- MUST: Package containing GUI applications includes a %{name}.desktop file, and
that file must be properly installed with desktop-file-install in the %install
section.
- MUST: Package does not own files or directories already owned by other packages. 

SHOULD Items:

 - SHOULD: The source package does include license text(s) as Licence
 - SHOULD: mock builds succcessfully in i386.
 - SHOULD: The reviewer tested that the package functions as described. A
package should not segfault instead of running, for example.
 - SHOULD: No scriptlets were used

APPROVED
Comment 14 Jef Spaleta 2006-12-31 01:06:12 EST
Well I'm an idiot.

failed to build on 64bit.  I hard coded /usr/lib/ into my patch for the netcdf
library location. I'll need to fix that so it catches /usr/lib64/ if its applicable.

-jef
Comment 15 Jef Spaleta 2006-12-31 03:31:40 EST
fixed and built in the development tree, awaiting signing.

-jef
Comment 16 Terje Røsten 2007-01-01 17:02:51 EST
(In reply to comment #15)
> fixed and built in the development tree, awaiting signing.

Thanks for the great work Jef, would you push this package for FC6 too?




Comment 17 Chitlesh GOORAH 2007-01-01 17:05:29 EST
Soon it will be, Terje :)
http://fedoraproject.org/wiki/Extras/CVSSyncNeeded
Comment 18 Jonathan Underwood 2012-05-13 16:29:49 EDT
Package Change Request
======================
Package Name: ScientificPython
New Branches: el6
Owners: jgu
InitialCC:
Comment 19 Jon Ciesla 2012-05-15 09:29:25 EDT
Git done (by process-git-requests).

Note You need to log in before you can comment on or make changes to this bug.