Bug 1087812

Summary: Review Request: gpaw - A grid-based real-space PAW method DFT code
Product: [Fedora] Fedora Reporter: marcindulak <Marcin.Dulak>
Component: Package ReviewAssignee: Susi Lehtola <susi.lehtola>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: package-review, susi.lehtola
Target Milestone: ---Keywords: Reopened
Target Release: ---Flags: susi.lehtola: fedora-review+
gwync: fedora-cvs+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: gpaw-0.10.0.11364-5.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-07-01 17:05:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1090070    
Bug Blocks:    

Description marcindulak 2014-04-15 11:24:30 UTC
Spec URL: http://marcindulak.fedorapeople.org/packages/gpaw/r05/gpaw.spec
SRPM URL: http://marcindulak.fedorapeople.org/packages/gpaw/r05/gpaw-0.10.0.11364-5.fc21.src.rpm
Description: GPAW is a density-functional theory (DFT) Python code based on the projector-augmented wave (PAW) method and the atomic simulation environment (ASE). It uses real-space uniform grids and multigrid methods or atom-centered basis-functions.

Fedora Account System Username: marcindulak

koji: http://koji.fedoraproject.org/koji/taskinfo?taskID=6740101

Note that gpaw depends on gpaw-setups package, which contains about 50MB of data under /usr/share/gpaw-setups - the way of packaging gpaw-setups needs to be decided yet. A separate gpaw-setups package preferred, due to the fact that any version of gpaw-setups can be used with any version of gpaw, and versioning of gpaw and gpaw-setups is different.

Comment 1 marcindulak 2014-04-22 09:30:07 UTC

*** This bug has been marked as a duplicate of bug 983608 ***

Comment 2 Susi Lehtola 2014-04-22 11:52:23 UTC
*** Bug 983608 has been marked as a duplicate of this bug. ***

Comment 3 Susi Lehtola 2014-04-22 11:54:01 UTC
gpaw-setups needs to be a wholly separate package, as you mention in comment #0.

Comment 4 marcindulak 2014-04-22 12:43:51 UTC
OK: gpaw is waiting for review here, and gpaw-setups coming shortly.

Comment 6 Susi Lehtola 2014-04-24 09:50:53 UTC
You still need to get rid of the gpaw-setups stuff here.

Comment 7 marcindulak 2014-04-24 10:22:37 UTC
Every gpaw release is bound for the specific gpaw-setups for the gpaw-test to pass, and therefore gpaw must have access to gpaw-setups during the rpmbuild.
gpaw-setups is an independent package.

Comment 8 Susi Lehtola 2014-04-24 12:13:55 UTC
Oh. Then please make a note of this in the spec file.

Or, you could just stop bundling gpaw-setups, and either BuildRequire the wanted version of gpaw-setups, or run the tests only if the existing gpaw-setups package is of the matching version.

In any case, I think bundling the sources for gpaw-setups is not allowed...

Comment 9 Susi Lehtola 2014-04-24 12:51:48 UTC
You can achieve the latter by e.g. implementing a _gpaw_setups_version rpm macro in gpaw-setups, i.e. in gpaw-setups

%global macrosdir %(d=%{_rpmconfigdir}/macros.d; [ -d $d ] || d=%{_sysconfdir}/rpm; echo $d)

%install
mkdir -p %{buildroot}%{macrosdir}
cat > %{buildroot}%{macrosdir}/macros.libint << EOF
# Current version of gpaw-setups is
%_gpaw_setups_version %{version}
EOF

Comment 10 marcindulak 2014-04-24 13:08:02 UTC
I can't BuildRequires the specific version of the setups because there will be only one version of gpaw-setups available in the Fedora repos at a time.

If i modify gpaw.spec to run gpaw-test only if specific version of gpaw-setups is present in the repositories, then after new gpaw-setups are released but still no new gpaw i won't be able to run gpaw-test during rebuild of gpaw (e.g. due to mass rebuild, or update of some other BuildRequires).

I'm not sure how clearly the concept of bundling is defined but what happens here
is using static data (gpaw-setups) for tests. I guess gpaw-setups is not a library. Is there a specific forum/mailing-list where we could clarify this - or do we need to go directly for bundling exception?

Comment 11 Susi Lehtola 2014-04-24 13:25:52 UTC
I think it'd be good to ask the FPC for an exception here. There are no security issues I can think of. The main thing is maybe that the sources and the SRPM are 5000% bigger if the setups are included.

Comment 12 marcindulak 2014-04-25 11:58:07 UTC
Asked the question here first: https://lists.fedoraproject.org/pipermail/packaging/2014-April/010145.html

Comment 13 marcindulak 2014-04-30 15:13:14 UTC
gpaw.spec has been updated and it does not bundle gpaw-setups anymore.

Comment 14 Susi Lehtola 2014-04-30 15:41:27 UTC
Please change the revision and changelog accordingly.

Also, I suggest you use ATLAS on architectures that don't have OpenBLAS.  The link should then be made against
 -L%{_libdir} -llapack -lf77blas -latlas
on older Fedoras / RHEL, and
 -L%{_libdir} -lsatlas
on newer ones. I think all the other requirements are available on arm and ppc.

Comment 15 Susi Lehtola 2014-05-01 10:15:44 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable

===== MUST items =====

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: If (and only if) the source package includes the text of the license(s)
     in its own file, then that file, containing the text of the license(s)
     for the package is included in %doc.
[x]: License field in the package spec file matches the actual license.
[-]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[-]: Package contains desktop file if it is a GUI application.
[-]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.

[?]: Requires correct, justified where necessary.
- I'm not sure about the openssh-clients buildrequire.
- You can drop BR: gcc, it's always included. https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Exceptions_2

[x]: Spec file is legible and written in American English.
[-]: Package contains systemd file(s) if in need.
[x]: Package is not known to require an ExcludeArch tag.
[x]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least one
     supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: All build dependencies are listed in BuildRequires, except for any that
     are listed in the exceptions section of Packaging Guidelines.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Package does not contain duplicates in %files.

[!]: Permissions on files are set properly.
- The .so library needs 755 permissions.

[x]: Package use %makeinstall only when make install' ' DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package do not use a name that already exist
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as provided
     in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
[x]: Packages must not store files under /srv, /opt or /usr/local

===== SHOULD items =====

Generic:
[!]: If the source package does not include license text(s) as a separate file
     from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane.
[x]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[-]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[?]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed files.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: Dist tag is present (not strictly required in GL).
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

Rpmlint
-------
gpaw.src: W: spelling-error %description -l en_US multigrid -> multitude
gpaw.x86_64: W: spelling-error %description -l en_US multigrid -> multitude
gpaw.x86_64: W: unstripped-binary-or-object /usr/lib64/python2.7/site-packages/_gpaw.so
gpaw.x86_64: W: no-documentation
gpaw.x86_64: W: no-manual-page-for-binary gpaw-setup
gpaw.x86_64: W: no-manual-page-for-binary gpaw-mpisim
gpaw.x86_64: W: no-manual-page-for-binary gpaw-run
gpaw.x86_64: W: no-manual-page-for-binary gpaw-mapfile-cray
gpaw.x86_64: W: no-manual-page-for-binary gpaw-basis
gpaw.x86_64: W: no-manual-page-for-binary gpaw-test
gpaw.x86_64: W: no-manual-page-for-binary gpaw-runscript
gpaw.x86_64: W: no-manual-page-for-binary gpaw-mapfile-bgp
gpaw.x86_64: W: no-manual-page-for-binary gpaw-install-setups
gpaw-common.noarch: W: summary-not-capitalized C gpaw - common files
gpaw-common.noarch: W: spelling-error %description -l en_US multigrid -> multitude
gpaw-mpich.x86_64: W: summary-not-capitalized C gpaw - mpich version
gpaw-mpich.x86_64: W: spelling-error %description -l en_US multigrid -> multitude
gpaw-mpich.x86_64: W: no-documentation
gpaw-openmpi.x86_64: W: summary-not-capitalized C gpaw - openmpi version
gpaw-openmpi.x86_64: W: spelling-error %description -l en_US multigrid -> multitude
gpaw-openmpi.x86_64: W: no-documentation
6 packages and 0 specfiles checked; 0 errors, 21 warnings.

These are OK, except for the unstripped-binary-or-object caused by the wrong permissions of the .so file.

Comment 16 Susi Lehtola 2014-05-01 10:22:28 UTC
The MPI stuff is OK as well.

I'd ask you to be a little bit more verbose in %files, since free usage of wildcards can have devastating results. I suggest

%files
%{_bindir}/%{name}*
%{python_sitearch}/%{name}/
%{python_sitearch}/%{name}-%{version}-py*.egg-info

This will also catch if the egg-info isn't produced.

Comment 17 marcindulak 2014-05-02 18:18:46 UTC
I start addressing the problems. Can you make libxc available on el7.ppc64?
It also looks to me that *mpich2, scipy are not available on el6.ppc64, is there an easy %if macro to say (not el6.ppc64)?

Comment 18 marcindulak 2014-05-02 19:10:52 UTC
What's your opinion about getting openblas on el7.x86_64?

Comment 19 Susi Lehtola 2014-05-02 22:45:45 UTC
(In reply to marcindulak from comment #17)
> I start addressing the problems. Can you make libxc available on el7.ppc64?

I haven't bothered with epel7 yet, since rhel7 isn't out yet and so it'd be difficult to debug the builds if necessary.

Same for openblas.

> It also looks to me that *mpich2, scipy are not available on el6.ppc64, is
> there an easy %if macro to say (not el6.ppc64)?

Uhm, I think not.
%ifarch ppc64
%if 0%{?rhel} == 6
foo
%endif
%endif

Comment 20 marcindulak 2014-05-06 16:18:55 UTC
I prefer to wait with epel7 until openblas/libxc are in epel7.x86_64.
I also exclude el6.ppc64 from the spec.
Fedora %arm builds (with atlas).

Other problems fixed:
- openssh-clients BR is needed for mpiexec to run (see http://koji.fedoraproject.org/koji/taskinfo?taskID=6817438 - this is an epel6 build where there is no openssh-clients dependency pulled in and mpiexec fails with "ORTE_ERROR_LOG: Not found in file ess_hnp_module.c at line 194").
I experienced the same issue on the opensuse build system.
- gcc BR removed
- _gpaw.so permissions fixed
- COPYING will be included in the gpaw tarball (from the next release) https://trac.fysik.dtu.dk/projects/gpaw/changeset/11439
- %files glob more explicit

Spec URL: http://marcindulak.fedorapeople.org/packages/gpaw/r02/gpaw.spec
SRPM URL: http://marcindulak.fedorapeople.org/packages/gpaw/r02/gpaw-0.10.0.11364-2.fc21.src.rpm

Comment 21 Susi Lehtola 2014-05-20 18:27:01 UTC
Are you sure gpaw doesn't need scipy to run?

**

I believe hdf5 (and its MPI flavors) should be added as BuildRequirements, so that gpaw gets built support for these.

**

If you can, you should modify the upstream setup.py so that the libraries are always installed as 755. It's standard for *nix shared libraries.

**

Actually,
MUST: Each package must consistently use macros.

is not OK. You are mixing %{buildroot} (1 occurrence) and %{optflags} (2 occurrences) vs $RPM_BUILD_ROOT (4 occurrences) and $RPM_OPT_FLAGS (0 occurrences).

http://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Using_.25.7Bbuildroot.7D_and_.25.7Boptflags.7D_vs_.24RPM_BUILD_ROOT_and_.24RPM_OPT_FLAGS

To fix this, replace the 3 occurrences of $RPM_BUILD_ROOT with %{buildroot}. If you like, you can also replace %{buildroot} and %{optflags} with $RPM_BUILD_ROOT and $RPM_OPT_FLAGS.

Comment 22 Susi Lehtola 2014-05-20 18:44:36 UTC
The only blocker is the last one, which I trust you'll fix before git import. This package has been

APPROVED

Comment 23 marcindulak 2014-05-22 10:57:18 UTC
New Package SCM Request
=======================
Package Name: gpaw
Short Description: A grid-based real-space PAW method DFT code
Owners: marcindulak
Branches: f19 f20 el6 epel7
InitialCC:

Comment 24 Gwyn Ciesla 2014-05-22 11:49:25 UTC
Git done (by process-git-requests).

Comment 25 Fedora Update System 2014-05-28 11:26:32 UTC
gpaw-0.10.0.11364-3.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/gpaw-0.10.0.11364-3.el6

Comment 26 Fedora Update System 2014-05-28 11:26:40 UTC
gpaw-0.10.0.11364-3.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/gpaw-0.10.0.11364-3.fc19

Comment 27 Fedora Update System 2014-05-28 11:26:49 UTC
gpaw-0.10.0.11364-3.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/gpaw-0.10.0.11364-3.fc20

Comment 28 Fedora Update System 2014-05-28 18:02:29 UTC
gpaw-0.10.0.11364-3.el6 has been pushed to the Fedora EPEL 6 testing repository.

Comment 29 Fedora Update System 2014-06-09 15:13:20 UTC
gpaw-0.10.0.11364-5.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/gpaw-0.10.0.11364-5.el6

Comment 30 Fedora Update System 2014-06-09 15:13:26 UTC
gpaw-0.10.0.11364-5.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/gpaw-0.10.0.11364-5.fc19

Comment 31 Fedora Update System 2014-06-09 15:13:33 UTC
gpaw-0.10.0.11364-5.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/gpaw-0.10.0.11364-5.fc20

Comment 32 Fedora Update System 2014-07-01 17:05:08 UTC
gpaw-0.10.0.11364-5.el6 has been pushed to the Fedora EPEL 6 stable repository.

Comment 33 Fedora Update System 2014-07-01 23:27:48 UTC
gpaw-0.10.0.11364-5.fc19 has been pushed to the Fedora 19 stable repository.

Comment 34 Fedora Update System 2014-07-01 23:29:24 UTC
gpaw-0.10.0.11364-5.fc20 has been pushed to the Fedora 20 stable repository.