Bug 1389971 - Review Request: elemental - distributed-memory dense and sparse-direct linear algebra and optimizatio
Summary: Review Request: elemental - distributed-memory dense and sparse-direct linea...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Antonio T. (sagitter)
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-10-30 05:52 UTC by Ryan H. Lewis (rhl)
Modified: 2016-12-16 22:22 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-16 20:58:38 UTC
Type: ---
Embargoed:
anto.trande: fedora-review+


Attachments (Terms of Use)

Description Ryan H. Lewis (rhl) 2016-10-30 05:52:08 UTC
Spec URL: https://github.com/elemental/Elemental/blob/master/redhat/elemental.spec
SRPM URL: http://koji.fedoraproject.org/koji/taskinfo?taskID=16253578
          https://copr.fedorainfracloud.org/coprs/rhl/elemental/build/471052/
Description: Elemental is an open-source library for distributed-memory dense and sparse-direct linear algebra and optimization which builds on top of BLAS, LAPACK, and MPI using modern C++ and additionally exposes interfaces to C and Python (with a Julia interface beginning development).

The development of Elemental has led to a number of research articles and a number of related projects, such as the parallel sweeping preconditioner, PSP, and a parallel algorithm for Low-rank Plus Sparse MRI, RT-LPS-MRI.

Fedora Account System Username: rhl

Comment 1 Ryan H. Lewis (rhl) 2016-10-30 06:00:46 UTC
[rpmbuild@9e172a3d127e x86_64]$ rpmlint elemental-openmpi-0.87-2.fc24.x86_64.rpm 
elemental-openmpi.x86_64: E: invalid-soname /usr/lib64/libElSuiteSparse_openmpi.so libElSuiteSparse_openmpi.so
elemental-openmpi.x86_64: E: invalid-soname /usr/lib64/libpmrrr_openmpi.so libpmrrr_openmpi.so
elemental-openmpi.x86_64: W: shared-lib-calls-exit /usr/lib64/libpmrrr_openmpi.so exit.5
elemental-openmpi.x86_64: E: invalid-soname /usr/lib64/libEl_openmpi.so libEl_openmpi.so
[I have supressed a large number of  missing man page warnings]
1 packages and 0 specfiles checked; 3 errors, 144 warnings.

the mpich subpackage looks the same as the openmpi subpackage.

[rpmbuild@9e172a3d127e x86_64]$ rpmlint elemental-python2-0.87-2.fc24.x86_64.rpm 
elemental-python2.x86_64: E: description-line-too-long C This package contains the python bindings for using Elemental through a python shell
elemental-python2.x86_64: W: no-documentation
1 packages and 0 specfiles checked; 1 errors, 1 warnings.

[rpmbuild@9e172a3d127e x86_64]$ rpmlint -v elemental-devel-0.87-2.fc24.x86_64.rpm 
elemental-devel.x86_64: I: checking
elemental-devel.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-devel.x86_64: W: no-documentation
elemental-devel.x86_64: W: non-standard-dir-in-usr etc
1 packages and 0 specfiles checked; 0 errors, 2 warnings.

[rpmbuild@9e172a3d127e x86_64]$ rpmlint -v elemental-common-0.87-2.fc24.x86_64.rpm 
elemental-common.x86_64: I: checking
elemental-common.x86_64: W: spelling-error Summary(en_US) mpich -> chimp
elemental-common.x86_64: W: spelling-error Summary(en_US) openmpi -> opening
elemental-common.x86_64: W: spelling-error %description -l en_US mpich -> chimp
elemental-common.x86_64: W: spelling-error %description -l en_US openmpi -> opening
elemental-common.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
1 packages and 0 specfiles checked; 0 errors, 4 warnings.


I will try to address the few errors and warning listed in short order. Also, we need to make an official release of Elemental and ask the spec file to reflect that new release version.

Comment 2 Ryan H. Lewis (rhl) 2016-10-30 06:01:29 UTC
[rpmbuild@9e172a3d127e SRPMS]$ rpmlint -v elemental-0.87-2.fc24.src.rpm 
elemental.src: I: checking
elemental.src: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental.src: I: checking-url https://github.com/elemental/Elemental/archive/master.zip (timeout 10 seconds)
1 packages and 0 specfiles checked; 0 errors, 0 warnings.

Comment 3 Antonio T. (sagitter) 2016-11-15 16:43:33 UTC
- Please, post direct links to 'raw' spec file and srpm.

- MPI compilers are not correct: you are using always '/usr/bin/c++'. You have to set CMAKE_CXX_COMPILER:FILEPATH option at least inside MPI build directories.

'lib*_openmpi' and 'lib*_mpich' libraries must be installed under '/usr/lib/openmpi/lib' and '/usr/lib/mpich/lib' respectively. Like them, even binary files must be installed under '/usr/lib/openmpi/bin' and '/usr/lib/mpich/bin' respectively.
See http://fedoraproject.org/wiki/Packaging:MPI

- If 

// Build METIS
BUILD_METIS:BOOL=ON

means build Metis from source, you can disable it.

- CMake looks searching for GMP, too.

-- Could NOT find GMP (missing:  GMP_INCLUDES GMP_LIBRARIES GMP_VERSION_OK) (Required is at least version "6.0.0")

- Python package name looks better like python2-elemental

- Libraries are not "sonamed"; could be a good suggestion for upstream.

Comment 4 Antonio T. (sagitter) 2016-11-15 16:48:17 UTC
Take a look to proposed 'Build Modes': http://libelemental.org/documentation/0.85/build.html#build-modes

Comment 5 Ryan H. Lewis (rhl) 2016-11-15 17:43:38 UTC
Thanks Antonio, I am adjusting the raw spec file, located in the repository itself: 

Spec URL: https://github.com/elemental/Elemental/blob/master/redhat/elemental.spec

The copr builds happen every time we push to master: https://copr.fedorainfracloud.org/coprs/rhl/elemental/build/477011/

I have sonamed libEl with the major version (and issued a PR for this, since I don't knwo what other affects this may have) and disabled BUILD_METIS (even though it wasn't building), and added gmp-devel as a build dependency.

I've renamed the python subpackges to python2-elemental-openmpi and added a corresponding mpich one.

I believe I have set the compiler flags correctly now, and moved the binaries/libraries to the correct locations. We will see what happens once the above build completes.

Comment 6 Antonio T. (sagitter) 2016-11-15 18:56:06 UTC
(In reply to Ryan H. Lewis (rhl) from comment #5)
> Thanks Antonio, I am adjusting the raw spec file, located in the repository
> itself: 
> 
> Spec URL:
> https://github.com/elemental/Elemental/blob/master/redhat/elemental.spec
> 
> The copr builds happen every time we push to master:
> https://copr.fedorainfracloud.org/coprs/rhl/elemental/build/477011/

When the package is ready, post links to the files.
(This is the raw file https://raw.githubusercontent.com/elemental/Elemental/master/redhat/elemental.spec)
> 
> I have sonamed libEl with the major version (and issued a PR for this, since
> I don't knwo what other affects this may have) and disabled BUILD_METIS
> (even though it wasn't building), and added gmp-devel as a build dependency.
> 
> I've renamed the python subpackges to python2-elemental-openmpi and added a
> corresponding mpich one.
> 
> I believe I have set the compiler flags correctly now, and moved the
> binaries/libraries to the correct locations. We will see what happens once
> the above build completes.

Do you really need to compile examples (bin/examples-*) and tests files (bin/tests-*)? If yes, those files should be packaged separately.

Also, you can perform 'make test' for testing Elemental.
http://libelemental.org/documentation/dev/build.html#testing-the-c-11-installation

Comment 7 Antonio T. (sagitter) 2016-11-17 18:47:09 UTC
> Do you really need to compile examples (bin/examples-*) and tests files 
> (bin/tests-*)? If yes, those files should be packaged separately.

Test files looks to be just for testing. There is no need to pack them.
Sorry for misunderstanding.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd); \
export  CTEST_OUTPUT_ON_FAILURE=1; \
ctest; \

'ctest' command goes under %check section.
See http://fedoraproject.org/wiki/Packaging:Guidelines#Scriplets_are_only_allowed_to_write_in_certain_directories

Comment 8 Ryan H. Lewis (rhl) 2016-11-18 00:06:33 UTC
can you explain how the ctest in %check would work for each of the two MPI variants, and on the _installed_ binaries or on the compiled but not installed ones?

Comment 9 Antonio T. (sagitter) 2016-11-18 11:30:57 UTC
(In reply to Ryan H. Lewis (rhl) from comment #8)
> can you explain how the ctest in %check would work for each of the two MPI
> variants, and on the _installed_ binaries or on the compiled but not
> installed ones?

Your latest SPEC files is good: http://copr-dist-git.fedorainfracloud.org/cgit/rhl/elemental/elemental.git/tree/elemental.spec?id=9e5397b387a7171c08d4c7f5d9f0af23e5adf742

We need to know why 'Examples/lapack_like/SimpleSVD' is failing, it looks like that it's faulted.


'%{_prefix}/%{_sysconfdir}/elemental/CMake/*' path is wrong to me; usually, CMake files go under a private directory of '%{_libdir}/cmake' like '%{_libdir}/cmake/elemental'.

Comment 10 Ryan H. Lewis (rhl) 2016-11-18 16:39:19 UTC
Upstream reported that SimpleSVD had a bug, and was redundant so it was removed. I am seeing that the tests are timing out/failing to find libEl.so.0 This is some sort of LD_LIBRARY_PATH issue. I am looking into it now.

I've tried to move that path as well.

Comment 11 Ryan H. Lewis (rhl) 2016-11-18 23:17:43 UTC
I believe the latest spec should work.

Comment 12 Antonio T. (sagitter) 2016-11-19 11:35:42 UTC
(In reply to Ryan H. Lewis (rhl) from comment #11)
> I believe the latest spec should work.

<
$ mock -r fedora-rawhide-x86_64 --no-clean --shell 'rpm -q --list elemental-openmpi| grep lib64'
Start: shell
/usr/lib64/openmpi/lib/libEl.so
/usr/lib64/openmpi/lib/libEl.so.0
/usr/lib64/openmpi/lib/libEl.so.86-dev
/usr/lib64/openmpi/lib/libElSuiteSparse.so
/usr/lib64/openmpi/lib/libpmrrr.so

$ mock -r fedora-rawhide-x86_64 --no-clean --shell 'rpm -q --list elemental-mpich| grep lib64'
Start: shell
/usr/lib64/mpich/lib/libEl.so
/usr/lib64/mpich/lib/libEl.so.0
/usr/lib64/mpich/lib/libEl.so.86-dev
/usr/lib64/mpich/lib/libElSuiteSparse.so
/usr/lib64/mpich/lib/libpmrrr.so

'/usr/lib64/mpich/lib/libEl.so' and '/usr/lib64/openmpi/lib/libEl.so' must be packed in 'elemental-mpich-devel' and 'elemental-openmpi-devel' respectively.
>

<
%package openmpi-examples
Summary: OpenMPI variant of Elemental
Group: Development/Libraries
** Requires: %{name}-openmpi = %{version}-%{release} **
%description openmpi-examples
Contains the example drivers built against OpenMPI

Required package is not arched, use
Requires: %{name}-openmpi%{?_isa} = %{version}-%{release}
Requires: %{name}-mpich%{?_isa} = %{version}-%{release}
>

<
$ mock -r fedora-rawhide-x86_64 --no-clean --shell 'rpmlint elemental-mpich'
Start: shell
elemental-mpich.x86_64: E: invalid-soname /usr/lib64/mpich/lib/libElSuiteSparse.so libElSuiteSparse.so
elemental-mpich.x86_64: W: undefined-non-weak-symbol /usr/lib64/mpich/lib/libElSuiteSparse.so sqrt
elemental-mpich.x86_64: W: unused-direct-shlib-dependency /usr/lib64/mpich/lib/libElSuiteSparse.so /usr/lib64/mpich/lib/libmpi.so.12
elemental-mpich.x86_64: E: invalid-soname /usr/lib64/mpich/lib/libpmrrr.so libpmrrr.so
elemental-mpich.x86_64: W: unused-direct-shlib-dependency /usr/lib64/mpich/lib/libpmrrr.so /lib64/libopenblas.so.0
elemental-mpich.x86_64: W: unused-direct-shlib-dependency /usr/lib64/mpich/lib/libpmrrr.so /lib64/libgfortran.so.3
elemental-mpich.x86_64: W: unused-direct-shlib-dependency /usr/lib64/mpich/lib/libpmrrr.so /lib64/libpthread.so.0
elemental-mpich.x86_64: W: unused-direct-shlib-dependency /usr/lib64/mpich/lib/libpmrrr.so /lib64/libqd.so.0
elemental-mpich.x86_64: W: shared-lib-calls-exit /usr/lib64/mpich/lib/libpmrrr.so exit.5
elemental-mpich.x86_64: W: no-documentation
elemental-mpich.x86_64: W: devel-file-in-non-devel-package /usr/lib64/mpich/lib/libEl.so
elemental-mpich.x86_64: E: library-without-ldconfig-postin /usr/lib64/mpich/lib/libEl.so.86-dev
elemental-mpich.x86_64: E: library-without-ldconfig-postun /usr/lib64/mpich/lib/libEl.so.86-dev
1 packages and 0 specfiles checked; 4 errors, 9 warnings.
Finish: shell

rpmlint shows an sqrt 'undefined-non-weak-symbol' warning, the library looks correctly linked with libm.so.

'unused-direct-shlib-dependency /usr/lib64/mpich/lib/libElSuiteSparse.so /usr/lib64/mpich/lib/libmpi.so.12' looks wrong, too.

Comment 13 Ryan H. Lewis (rhl) 2016-11-19 18:21:31 UTC
where do the so-versioned parts of libEl go? how about the the other shared objects? 

why do they belong in a devel package? What belongs in the elemental-openmpi package? there would be nothing left in elemental-mpich/openmpi packages then. 

Similar to the base package `elemental`. There is nothing there for it.


<
rpmlint shows an sqrt 'undefined-non-weak-symbol' warning, the library looks correctly linked with libm.so.

<
'unused-direct-shlib-dependency /usr/lib64/mpich/lib/libElSuiteSparse.so /usr/lib64/mpich/lib/libmpi.so.12' looks wrong, too.

Do we need to address these issues?

Comment 14 Ryan H. Lewis (rhl) 2016-11-19 18:33:08 UTC
Also, what do you mean by "this looks wrong." what is wrong?

Comment 15 Ryan H. Lewis (rhl) 2016-11-19 19:24:04 UTC
I've edited the spec for:

1. use -Wl, --as-needed (should clean up unused-direct-shlib-dependency)
2. no postin/postun
3. arch versioning with %{?_isa}

I've issued a code fix for:

1. PMRRR calling exit
2. invalid-soname
3. undefined-non-weak-symbol `sqrt` which an explicit link against libm on LINUX

Comment 16 Antonio T. (sagitter) 2016-11-19 19:45:57 UTC
(In reply to Ryan H. Lewis (rhl) from comment #13)
> where do the so-versioned parts of libEl go? how about the the other shared
> objects? 
> 
> why do they belong in a devel package? What belongs in the elemental-openmpi
> package? there would be nothing left in elemental-mpich/openmpi packages
> then. 
> 
> Similar to the base package `elemental`. There is nothing there for it.

Unversioned libraries (*.so, symbolic links to the versioned ones) go in devel packages.

> 
> 
> <
> rpmlint shows an sqrt 'undefined-non-weak-symbol' warning, the library looks
> correctly linked with libm.so.

This warning shouldn't be there since library is correctly linked to libm (-lm).

> 
> <
> 'unused-direct-shlib-dependency /usr/lib64/mpich/lib/libElSuiteSparse.so
> /usr/lib64/mpich/lib/libmpi.so.12' looks wrong, too.
> 
> Do we need to address these issues?

I'm not sure they are real bugs, but it's strange that library has an unused dependence against libmpi.so .

Comment 17 Ryan H. Lewis (rhl) 2016-11-19 22:34:19 UTC
Ok, I've gone ahead and separated out the devel package, and the rpms are building

Comment 19 Antonio T. (sagitter) 2016-11-21 19:41:32 UTC
- Code under external/suite_sparse is under LGPLv2+ license.
  Other code/cmake files is MIT, Boost.
  Please, update License tag and pack debian/copyright by using
  %license tag.

- Don't need to execute ldconfig scripts.

- Some tests are failed on PPC64: http://koji.fedoraproject.org/koji/taskinfo?taskID=16555354

- I see you're using a master.zip archive source code. It would be better indicate the date of the commit, elemental-0.87-2.%{checkout}git%{commitnumber}%{?dist}
  See https://fedoraproject.org/wiki/Packaging:Versioning#Snapshot_packages

x MUST: The package must be named according to the Package Naming Guidelines .
x MUST: The spec file name must match the base package %{name}, in the format %{name}.spec unless your package has an exemption. [2] .
x MUST: The package must meet the Packaging Guidelines .
x MUST: The package must be licensed with a Fedora approved license and meet the Licensing Guidelines .
! MUST: The License field in the package spec file must match the actual license. [3]
x MUST: If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package must be included in %license.[4]

x MUST: The spec file must be written in American English. [5]
x MUST: The spec file for the package MUST be legible. [6]
? MUST: The sources used to build the package must match the upstream source, as provided in the spec URL. Reviewers should use sha256sum for this task as it is used by the sources file once   imported into git. If no upstream URL can be specified for this package, please see the Source URL Guidelines for how to deal with this.

! MUST: The package MUST successfully compile and build into binary rpms on at least one primary architecture. [7]
- MUST: If the package does not successfully compile, build or work on an architecture, then those architectures should be listed in the spec in ExcludeArch. Each architecture listed in ExcludeArch MUST have a bug filed in bugzilla, describing the reason that the package does not compile/build/work on that architecture. The bug number MUST be placed in a comment, next to the corresponding ExcludeArch line. [8]

x MUST: All build dependencies must be listed in BuildRequires, except for any that are listed in the exceptions section of the Packaging Guidelines ; inclusion of those as BuildRequires is optional. Apply common sense.

- MUST: The spec file MUST handle locales properly. This is done by using the %find_lang macro. Using %{_datadir}/locale/* is strictly forbidden.[9]
! MUST: Every binary RPM package (or subpackage) which stores shared library files (not just symlinks) in any of the dynamic linker's default paths, must call ldconfig in %post and %postun. [10]
x MUST: Packages must NOT bundle copies of system libraries.[11]
- MUST: If the package is designed to be relocatable, the packager must state this fact in the request for review, along with the rationalization for relocation of that specific package. Without this, use of Prefix: /usr is considered a blocker. [12]

x MUST: A package must own all directories that it creates. If it does not create a directory that it uses, then it should require a package which does create that directory. [13]
x MUST: A Fedora package must not list a file more than once in the spec file's %files listings. (Notable exception: license texts in specific situations)[14]
x MUST: Permissions on files must be set properly. Executables should be set with executable permissions, for example. [15]
x MUST: Each package must consistently use macros. [16]
x MUST: The package must contain code, or permissible content. [17]
- MUST: Large documentation files must go in a -doc subpackage. (The definition of large is left up to the packager's best judgement, but is not restricted to size. Large can refer to either size or quantity). [18]

x MUST: If a package includes something as %doc, it must not affect the runtime of the application. To summarize: If it is in %doc, the program must run properly if it is not present. [18]
- MUST: Static libraries must be in a -static package. [19]
x MUST: Development files must be in a -devel package. [20]
x MUST: In the vast majority of cases, devel packages must require the base package using a fully versioned dependency: Requires: %{name}%{?_isa} = %{version}-%{release} [21]
x MUST: Packages must NOT contain any .la libtool archives, these must be removed in the spec if they are built.[19]
- MUST: Packages containing GUI applications must include a %{name}.desktop file, and that file must be properly installed with desktop-file-install in the %install section. If you feel that your packaged GUI application does not need a .desktop file, you must put a comment in the spec file with your explanation. [22]

x MUST: Packages must not own files or directories already owned by other packages. The rule of thumb here is that the first package to be installed should own the files or directories that other packages may rely upon. This means, for example, that no package in Fedora should ever share ownership with any of the files or directories owned by the filesystem or man package. If you feel that you have a good reason to own a file or directory that another package owns, then please present that at package review time. [23]

x MUST: All filenames in rpm packages must be valid UTF-8.

- SHOULD: If the source package does not include license text(s) as a separate file from upstream, the packager SHOULD query upstream to include it. [25]
- SHOULD: The description and summary sections in the package spec file should contain translations for supported Non-English languages, if available. [26]
x SHOULD: The reviewer should test that the package builds in mock. [27]
x SHOULD: The package should compile and build into binary rpms on all supported architectures. [28]
x SHOULD: The reviewer should test that the package functions as described. A package should not segfault instead of running, for example.
- SHOULD: If scriptlets are used, those scriptlets must be sane. This is vague, and left up to the reviewers judgement to determine sanity. [29]
x SHOULD: Usually, subpackages other than devel should require the base package using a fully versioned dependency. [21]
- SHOULD: The placement of pkgconfig(.pc) files depends on their usecase, and this is usually for development purposes, so should be placed in a -devel pkg. A reasonable exception is that the main pkg itself is a devel tool not installed in a user runtime, e.g. gcc or gdb. [30]

- SHOULD: If the package has file dependencies outside of /etc, /bin, /sbin, /usr/bin, or /usr/sbin consider requiring the package which provides the file instead of the file itself. [31]
- SHOULD: your package should contain man pages for binaries/scripts. If it doesn't, work with upstream to add them where they make sense

$ mock -r fedora-rawhide-x86_64 --no-clean --shell 'rpmlint elemental-mpich elemental-mpich-devel  elemental-openmpi-devel elemental-common elemental-mpich-examples elemental-openmpi-examples python2-elemental-mpich python2-elemental-openmpi elemental elemental-devel'
Start: shell
elemental-mpich.x86_64: W: no-documentation
elemental-mpich-devel.x86_64: W: only-non-binary-in-usr-lib
elemental-mpich-devel.x86_64: W: no-documentation
elemental-openmpi-devel.x86_64: W: only-non-binary-in-usr-lib
elemental-openmpi-devel.x86_64: W: no-documentation
elemental-openmpi-devel.x86_64: E: non-empty-%postun /sbin/ldconfig
elemental-common.x86_64: W: spelling-error Summary(en_US) mpich -> chimp
elemental-common.x86_64: W: spelling-error Summary(en_US) openmpi -> opening
elemental-common.x86_64: W: spelling-error %description -l en_US mpich -> chimp
elemental-common.x86_64: W: spelling-error %description -l en_US openmpi -> opening
elemental-mpich-examples.x86_64: W: no-documentation
elemental-openmpi-examples.x86_64: W: no-documentation
python2-elemental-mpich.x86_64: E: description-line-too-long C This package contains the python bindings for using Elemental through a python shell with MPICH
python2-elemental-mpich.x86_64: W: only-non-binary-in-usr-lib
python2-elemental-mpich.x86_64: W: no-documentation
python2-elemental-openmpi.x86_64: E: description-line-too-long C This package contains the python bindings for using Elemental through a python shell with OpenMPI
python2-elemental-openmpi.x86_64: W: only-non-binary-in-usr-lib
python2-elemental-openmpi.x86_64: W: no-documentation
(none): E: no installed packages by name elemental
elemental-devel.x86_64: W: only-non-binary-in-usr-lib
elemental-devel.x86_64: W: no-documentation
9 packages and 0 specfiles checked; 3 errors, 17 warnings.
Finish: shell

$ rpmlint elemental-0.87-2.fc26.src.rpm
elemental.src: E: unknown-key RSA#83ec784d (MD5
elemental.src:139: W: mixed-use-of-spaces-and-tabs (spaces: line 139, tab: line 1)

Comment 20 Ryan H. Lewis (rhl) 2016-11-22 08:17:08 UTC
1) Are PPC64 builds a necessity? 

Perhaps we can just file a bug for this and fix it later? Initial investigation suggests its likely a toolchain issue. We need to run valgrind on it simulating the ppc64 environment..

2) We just releaseed Elemental 0.87 itself in the last 24 hours. 

I am now updating the spec to reflect that. It's probably best that we package a released version of Elemental. 

3) I've updated the licenses accordingly.

4) I'm not sure if there is a facility for a rawhide package that captures the latest of master?

Comment 21 Antonio T. (sagitter) 2016-11-22 10:39:29 UTC
(In reply to Ryan H. Lewis (rhl) from comment #20)
> 1) Are PPC64 builds a necessity? 
> 
> Perhaps we can just file a bug for this and fix it later? Initial
> investigation suggests its likely a toolchain issue. We need to run valgrind
> on it simulating the ppc64 environment..

You need to build successfully your package on rawhide; then, fix issues on PPC or exclude these architectures temporarily and open a bug report (http://fedoraproject.org/wiki/Packaging:Guidelines#Architecture_Build_Failures).

> 
> 2) We just releaseed Elemental 0.87 itself in the last 24 hours. 
> 
> I am now updating the spec to reflect that. It's probably best that we
> package a released version of Elemental. 
> 
> 3) I've updated the licenses accordingly.
> 
> 4) I'm not sure if there is a facility for a rawhide package that captures
> the latest of master?

Comment 22 Ryan H. Lewis (rhl) 2016-11-30 02:14:04 UTC
I've spent some quality time debugging Elementals failing PPC examples and tests. 

There is nothing obvious to point to in Elemental. There appears to be some vanilla C++ code which works on every architecture but is producing a segmentation fault on PPC64. I suspect this is a gcc compiler bug (in the optimizer) when building for ppc, as it only appears in Release mode. 

I cannot seem to produce a minimal test case which actually replicates this issue (without Elemental that is), so I am not sure it even makes sense yet to file a bug.

Hopefully this warrants us not needing to support PPC builds in order to get through the review. As soon as this is fixed we should definitely build El for PPC.

I've submitted a scratch build to koji for the release version of Elemental. Also, I have produced a second spec file, instead of the one used with master: 

https://raw.githubusercontent.com/elemental/Elemental/master/redhat/elemental-release.spec

I have a scratch build running (including all architectures, even ppc which I know will fail). I'll post the results shortly.

Comment 23 Ryan H. Lewis (rhl) 2016-11-30 02:27:54 UTC
http://koji.fedoraproject.org/koji/taskinfo?taskID=16676132 

At least x86_64 builds green.

Comment 24 Ryan H. Lewis (rhl) 2016-11-30 05:10:36 UTC
http://koji.fedoraproject.org/koji/taskinfo?taskID=16676472 Here is a build where PPC is excluded.

Comment 25 Ryan H. Lewis (rhl) 2016-12-01 17:39:56 UTC
I believe i've done what you've asked now. The output of rpmlint is below:

You wrote:
> - Don't need to execute ldconfig scripts.

However, this is generating rpmlint errors. I'm not sure if you want me to add them back.

elemental-common.x86_64: I: checking
elemental-common.x86_64: W: spelling-error Summary(en_US) mpich -> chimp
elemental-common.x86_64: W: spelling-error Summary(en_US) openmpi -> opening
elemental-common.x86_64: W: spelling-error %description -l en_US mpich -> chimp
elemental-common.x86_64: W: spelling-error %description -l en_US openmpi -> opening
elemental-common.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-devel.x86_64: I: checking
elemental-devel.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-devel.x86_64: W: only-non-binary-in-usr-lib
elemental-devel.x86_64: W: no-documentation
elemental-mpich.x86_64: I: checking
elemental-mpich.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-mpich.x86_64: W: no-documentation
elemental-mpich.x86_64: E: library-without-ldconfig-postin /usr/lib64/mpich/lib/libElSuiteSparse.so.87-dev
elemental-mpich.x86_64: E: library-without-ldconfig-postun /usr/lib64/mpich/lib/libElSuiteSparse.so.87-dev
elemental-mpich.x86_64: E: library-without-ldconfig-postin /usr/lib64/mpich/lib/libpmrrr.so.87-dev
elemental-mpich.x86_64: E: library-without-ldconfig-postun /usr/lib64/mpich/lib/libpmrrr.so.87-dev
elemental-mpich.x86_64: E: library-without-ldconfig-postin /usr/lib64/mpich/lib/libEl.so.87-dev
elemental-mpich.x86_64: E: library-without-ldconfig-postun /usr/lib64/mpich/lib/libEl.so.87-dev
elemental-mpich-devel.x86_64: I: checking
elemental-mpich-devel.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-mpich-devel.x86_64: W: only-non-binary-in-usr-lib
elemental-mpich-devel.x86_64: W: no-documentation
elemental-mpich-examples.x86_64: I: checking
elemental-mpich-examples.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-mpich-examples.x86_64: W: no-documentation
elemental-openmpi.x86_64: I: checking
elemental-openmpi.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-openmpi.x86_64: W: no-documentation
elemental-openmpi.x86_64: E: library-without-ldconfig-postin /usr/lib64/openmpi/lib/libpmrrr.so.87-dev
elemental-openmpi.x86_64: E: library-without-ldconfig-postun /usr/lib64/openmpi/lib/libpmrrr.so.87-dev
elemental-openmpi.x86_64: E: library-without-ldconfig-postin /usr/lib64/openmpi/lib/libElSuiteSparse.so.87-dev
elemental-openmpi.x86_64: E: library-without-ldconfig-postun /usr/lib64/openmpi/lib/libElSuiteSparse.so.87-dev
elemental-openmpi.x86_64: E: library-without-ldconfig-postin /usr/lib64/openmpi/lib/libEl.so.87-dev
elemental-openmpi.x86_64: E: library-without-ldconfig-postun /usr/lib64/openmpi/lib/libEl.so.87-dev
elemental-openmpi-devel.x86_64: I: checking
elemental-openmpi-devel.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-openmpi-devel.x86_64: W: only-non-binary-in-usr-lib
elemental-openmpi-devel.x86_64: W: no-documentation
elemental-openmpi-examples.x86_64: I: checking
elemental-openmpi-examples.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
elemental-openmpi-examples.x86_64: W: no-documentation
python2-elemental-mpich.x86_64: I: checking
python2-elemental-mpich.x86_64: E: description-line-too-long C This package contains the python bindings for using Elemental through a python shell with MPICH
python2-elemental-mpich.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
python2-elemental-mpich.x86_64: W: only-non-binary-in-usr-lib
python2-elemental-mpich.x86_64: W: no-documentation
python2-elemental-openmpi.x86_64: I: checking
python2-elemental-openmpi.x86_64: E: description-line-too-long C This package contains the python bindings for using Elemental through a python shell with OpenMPI
python2-elemental-openmpi.x86_64: I: checking-url http://libelemental.org (timeout 10 seconds)
python2-elemental-openmpi.x86_64: W: only-non-binary-in-usr-lib
python2-elemental-openmpi.x86_64: W: no-documentation
10 packages and 0 specfiles checked; 14 errors, 18 warnings.

Comment 26 Ryan H. Lewis (rhl) 2016-12-01 18:02:39 UTC
I've also reported the PPC failure to the GCC folks: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78636

Comment 27 Antonio T. (sagitter) 2016-12-01 18:04:22 UTC
(In reply to Ryan H. Lewis (rhl) from comment #25)
> I believe i've done what you've asked now. The output of rpmlint is below:
> 
> You wrote:
> > - Don't need to execute ldconfig scripts.
> 
> However, this is generating rpmlint errors. I'm not sure if you want me to
> add them back.
> 

'rpmlint' recognizes libs and does not see any 'ldconfig' commands, it's an error for it but 'ldconfig' does not work with private directories (or at least this is what i see in manpages), so are false positive for us.

Also, mpi libraries contain rpaths set toward /usr/lib(64)/openmpi(mpich) which are directories where all MPI libs are shared.

Please, post links to the new SPEC and SRPM files.

Comment 29 Antonio T. (sagitter) 2016-12-01 18:23:29 UTC
Add 

ExcludeArch: %{power64}

until PPC builds are fixed.

Comment 30 Ryan H. Lewis (rhl) 2016-12-01 23:37:54 UTC
done.

Comment 31 Antonio T. (sagitter) 2016-12-02 14:17:38 UTC
(In reply to Ryan H. Lewis (rhl) from comment #28)
> spec:
> https://raw.githubusercontent.com/elemental/Elemental/master/redhat/
> elemental-release.spec
> srpm: https://rhl.fedorapeople.org/elemental/elemental-0.87.5-2.fc24.src.rpm

Please, leave a comment about why you have excluded PPC including eventual links.

'elemental' does not build on epel6/7 because of missing 'devtoolset-4-toolchain':

http://koji.fedoraproject.org/koji/taskinfo?taskID=16707202

Comment 32 Ryan H. Lewis (rhl) 2016-12-02 15:36:14 UTC
Should I just excludearch them as well? They do not have new enough compilers to build Elemental. devtoolset-4 makes this possible, and it works in the copr build. #fedora-devel says that the SCLs are not currently usable in fedora epel packages.

Comment 33 Antonio T. (sagitter) 2016-12-02 17:30:24 UTC
>Should I just excludearch them as well?

No.
I thought you wanted to compile on epel, too.

The review is almost done:

- Your are not using %license to pack the license files.

- You should use %doc or %_pkgdocdir macros for documentation, not %{_docdir}/Elemental (http://fedoraproject.org/wiki/Packaging:Guidelines#Documentation):

$ rpm --eval %_pkgdocdir
%{_docdir}/%{name}

- 'elemental-common' provides arch-independent files, so it is a 'noarch' package.
Please, modify in this sense

%package common
Summary: Files in common between mpich and openmpi
Group: Development/Libraries
BuildArch: noarch                         <---
Requires: qt5-qtbase
%description common 
Files not specific to mpich or openmpi

Consequently, other packages which require 'elemental-common' use the line

Requires: %{name}-common = %{version}-%{release}

(without '%{?_isa}')

Comment 34 Ryan H. Lewis (rhl) 2016-12-02 18:07:35 UTC
I definitely do want to compile on EPEL, but, it appears without devtoolset-4 this will not be possible.

I'm not sure what you want me to do with: 


> - Your are not using %license to pack the license files.

> - You should use %doc or %_pkgdocdir macros for documentation, not %{_docdir}/Elemental (http://fedoraproject.org/wiki/Packaging:Guidelines#Documentation):

I don't seem to be using ` %{_docdir}/Elemental`

Comment 35 Ryan H. Lewis (rhl) 2016-12-02 18:09:23 UTC
Oh I see you want me do to:

mv %{_docdir}/Elemental %_pkgdocdir

and then replace the %{_datadir}/doc/Elemental with %_pkgdocdir

Comment 36 Antonio T. (sagitter) 2016-12-02 18:19:53 UTC
%_pkgdocdir = %{_docdir}/%{name} = %{_datadir}/doc/elemental != %{_datadir}/doc/Elemental

:)

Use %license to tag the license files like:

%license COPYING.txt
or
%license LICENSE
or
%license COPYRIGHT

Comment 37 Ryan H. Lewis (rhl) 2016-12-02 18:25:49 UTC
I have this line: %license debian/copyright

Is that the wrong location somehow? rpmlint isn't complaining about it obviously, so, i'm not sure offhand. 

Or do you mean that you want: the other COPYRIGHT file? that one just suggests you read the debian/copyright file so i'm not sure what the value is of that..

Comment 38 Ryan H. Lewis (rhl) 2016-12-02 18:29:23 UTC
Ok, I believe I have addressed your concerns now.

Comment 39 Antonio T. (sagitter) 2016-12-02 18:35:22 UTC
(In reply to Ryan H. Lewis (rhl) from comment #38)
> Ok, I believe I have addressed your concerns now.

Not there. In this way:

%files common 
%{_datadir}/elemental/*
%_pkgdocdir/
%license debian/copyright
%license LICENSE

'debian/copyright' and 'LICENSE' will be installed with 'elemental-common' package, so they will be always present when 'elemental' libraries are installed.

Comment 40 Ryan H. Lewis (rhl) 2016-12-02 18:43:59 UTC
done.

Comment 41 Antonio T. (sagitter) 2016-12-02 19:40:56 UTC
Please, post new links (rebuild the src-rpm from new SPEC file).

Comment 43 Antonio T. (sagitter) 2016-12-04 13:54:43 UTC
Following line can be removed: 


> %files -n python2-elemental-mpich 
> %{python2_sitearch}/mpich/*
>
%license debian/copyright         <----
>
> %changelog
...

Package approved.

Comment 44 Gwyn Ciesla 2016-12-05 14:45:19 UTC
Package request has been approved: https://admin.fedoraproject.org/pkgdb/package/rpms/elemental

Comment 45 Fedora Update System 2016-12-06 21:11:12 UTC
elemental-0.87.5-2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-563959753e

Comment 46 Fedora Update System 2016-12-06 21:11:21 UTC
elemental-0.87.5-2.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-6ca5b05dc6

Comment 47 Fedora Update System 2016-12-08 04:51:34 UTC
elemental-0.87.5-2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-563959753e

Comment 48 Fedora Update System 2016-12-08 04:54:56 UTC
elemental-0.87.5-2.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-6ca5b05dc6

Comment 49 Fedora Update System 2016-12-16 20:58:38 UTC
elemental-0.87.5-2.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

Comment 50 Fedora Update System 2016-12-16 22:22:40 UTC
elemental-0.87.5-2.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.