Bug 2253356 - Review Request: lusol - LU factors of a square or rectangular sparse matrix
Summary: Review Request: lusol - LU factors of a square or rectangular sparse matrix
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Benson Muite
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 2253360
TreeView+ depends on / blocked
 
Reported: 2023-12-06 21:46 UTC by Jerry James
Modified: 2024-02-09 21:10 UTC (History)
2 users (show)

Fixed In Version: lusol-20160126-1.20221005gite35b66d.fc40
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2024-02-09 21:10:15 UTC
Type: ---
Embargoed:
benson_muite: fedora-review+


Attachments (Terms of Use)

Description Jerry James 2023-12-06 21:46:17 UTC
Spec URL: https://jjames.fedorapeople.org/lusol/lusol.spec
SRPM URL: https://jjames.fedorapeople.org/lusol/lusol-20160126-1.20221005gite35b66d.fc40.src.rpm
Fedora Account System Username: jjames
Description: LUSOL computes LU factors of a square or rectangular sparse matrix.

This package is part of an effort to add the SoPlex and SCIP solvers to Fedora.  The entire collection of packages is available in a COPR: https://copr.fedorainfracloud.org/coprs/jjames/SCIP/.

Comment 1 Fedora Review Service 2023-12-06 21:51:14 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/6729876
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2253356-lusol/fedora-rawhide-x86_64/06729876-lusol/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 2 Benson Muite 2023-12-08 06:06:17 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed



===== MUST items =====

C/C++:
[-]: Provides: bundled(gnulib) in place as required.
     Note: Sources not installed
[x]: Package does not contain kernel modules.
[x]: If your application is a C or C++ application you must list a
     BuildRequires against gcc, gcc-c++ or clang.
[x]: Header files in -devel subpackage, if present.
[x]: ldconfig not called in %post and %postun for Fedora 28 and later.
[x]: Package does not contain any libtool archives (.la)
[x]: Package contains no static executables.
[x]: Rpath absent or only used for internal libs.
[x]: Development (unversioned) .so files in -devel subpackage, if present.

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[ ]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses
     found: "Unknown or generated", "MIT License", "*No copyright* MIT
     License". 36 files have unknown license. Detailed output of
     licensecheck in /home/fedora/2253356-lusol/licensecheck.txt
[x]: License file installed when any subpackage combination is installed.
[x]: If the package is under multiple licenses, the licensing breakdown
     must be documented in the spec.
[x]: %build honors applicable compiler flags or justifies otherwise.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[-]: Package contains desktop file if it is a GUI application.
[x]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[ ]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[ ]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[-]: Package contains systemd file(s) if in need.
[ ]: Useful -debuginfo package or justification otherwise.
[ ]: Package is not known to require an ExcludeArch tag.
[ ]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least
     one supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: If (and only if) the source package includes the text of the
     license(s) in its own file, then that file, containing the text of the
     license(s) for the package is included in %license.
[x]: The License field must be a valid SPDX expression.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Dist tag is present.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package must not depend on deprecated() packages.
[x]: Package use %makeinstall only when make install DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package does not use a name that already exists.
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as
     provided in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
     Note: Documentation size is 4374 bytes in 2 files.
[x]: Packages must not store files under /srv, /opt or /usr/local

===== SHOULD items =====

Generic:
[-]: If the source package does not include license text(s) as a separate
     file from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[ ]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[x]: Patches link to upstream bugs/comments/lists or are otherwise
     justified.
[-]: Sources are verified with gpgverify first in %prep if upstream
     publishes signatures.
     Note: gpgverify is not used.
[ ]: Package should compile and build into binary rpms on all supported
     architectures.
[-]: %check is present and all tests pass.
[ ]: Packages should try to preserve timestamps of original installed
     files.
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Fully versioned dependency in subpackages if applicable.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on debuginfo package(s).
     Note: No rpmlint messages.
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Large data in /usr/share should live in a noarch subpackage if package
     is arched.
[x]: Spec file according to URL is the same as in SRPM.


Rpmlint
-------
Checking: lusol-20160126-1.20221005gite35b66d.fc40.x86_64.rpm
          lusol-devel-20160126-1.20221005gite35b66d.fc40.x86_64.rpm
          lusol-debuginfo-20160126-1.20221005gite35b66d.fc40.x86_64.rpm
          lusol-debugsource-20160126-1.20221005gite35b66d.fc40.x86_64.rpm
          lusol-20160126-1.20221005gite35b66d.fc40.src.rpm
============================ rpmlint session starts ============================
rpmlint: 2.4.0
configuration:
    /usr/lib/python3.12/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora-legacy-licenses.toml
    /etc/xdg/rpmlint/fedora-spdx-licenses.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
rpmlintrc: [PosixPath('/tmp/tmpy8buk4kt')]
checks: 31, packages: 5

lusol.spec: W: patch-not-applied Patch0: 0001-Add-missing-parameter-to-lu1rec-calls.patch
lusol.spec: W: patch-not-applied Patch1: 0002-Give-zero-a-value-before-use-in-lu8mod.patch
lusol.spec: W: patch-not-applied Patch2: lusol-uninit.patch
lusol-devel.x86_64: W: no-documentation
 5 packages and 0 specfiles checked; 0 errors, 4 warnings, 0 badness; has taken 2.3 s 




Rpmlint (debuginfo)
-------------------
Checking: lusol-debuginfo-20160126-1.20221005gite35b66d.fc40.x86_64.rpm
============================ rpmlint session starts ============================
rpmlint: 2.4.0
configuration:
    /usr/lib/python3.12/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora-legacy-licenses.toml
    /etc/xdg/rpmlint/fedora-spdx-licenses.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
rpmlintrc: [PosixPath('/tmp/tmpfe50eazq')]
checks: 31, packages: 1

 1 packages and 0 specfiles checked; 0 errors, 0 warnings, 0 badness; has taken 0.8 s 





Rpmlint (installed packages)
----------------------------
============================ rpmlint session starts ============================
rpmlint: 2.5.0
configuration:
    /usr/lib/python3.12/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora-legacy-licenses.toml
    /etc/xdg/rpmlint/fedora-spdx-licenses.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
checks: 32, packages: 4

lusol-devel.x86_64: W: no-documentation
 4 packages and 0 specfiles checked; 0 errors, 1 warnings, 27 filtered, 0 badness; has taken 2.5 s 



Source checksums
----------------
https://github.com/nwh/lusol/archive/e35b66d23729936bd475b31d933feca2b0021a13/lusol-e35b66d23729936bd475b31d933feca2b0021a13.tar.gz :
  CHECKSUM(SHA256) this package     : 1d4a51aa49871bc7f4f4eb9340a465ed16f2ccb059d2bf52564aa64a768f578a
  CHECKSUM(SHA256) upstream package : 1d4a51aa49871bc7f4f4eb9340a465ed16f2ccb059d2bf52564aa64a768f578a


Requires
--------
lusol (rpmlib, GLIBC filtered):
    libc.so.6()(64bit)
    libgfortran.so.5()(64bit)
    libgfortran.so.5(GFORTRAN_8)(64bit)
    rtld(GNU_HASH)

lusol-devel (rpmlib, GLIBC filtered):
    libclusol.so.0()(64bit)
    lusol(x86-64)

lusol-debuginfo (rpmlib, GLIBC filtered):

lusol-debugsource (rpmlib, GLIBC filtered):



Provides
--------
lusol:
    libclusol.so.0()(64bit)
    lusol
    lusol(x86-64)

lusol-devel:
    lusol-devel
    lusol-devel(x86-64)

lusol-debuginfo:
    debuginfo(build-id)
    libclusol.so.0.0.0-20160126-1.20221005gite35b66d.fc40.x86_64.debug()(64bit)
    lusol-debuginfo
    lusol-debuginfo(x86-64)

lusol-debugsource:
    lusol-debugsource
    lusol-debugsource(x86-64)



Generated by fedora-review 0.10.0 (e79b66b) last change: 2023-07-24
Command line :/usr/bin/fedora-review -b 2253356
Buildroot used: fedora-rawhide-x86_64
Active plugins: Generic, C/C++, Shell-api
Disabled plugins: Ocaml, R, Python, Java, Perl, Haskell, SugarActivity, PHP, fonts
Disabled flags: EXARCH, EPEL6, EPEL7, DISTTAG, BATCH

Comments:
a) Can the file src/lusol.txt be added to the documentation?
%doc src/lusol.txt
b) Should the mod files lusol.mod and lusol_precision.mod be packaged?
These are development files typically placed in %{_fmoddir} and devel package
must then have
Requires: gcc-gfortran%{_isa}
see
https://docs.fedoraproject.org/en-US/packaging-guidelines/Fortran/
c) Maybe also send a pull request to add soname upstream?
d) patch lusol-uninit.patch seems logical, but the code has no tests, can check
this, but would need a few days.
e) Build process can sometimes fail - not sure why:
https://kojipkgs.fedoraproject.org//work/tasks/8555/110038555/build.log
Does not happen consistently though:
https://koji.fedoraproject.org/koji/taskinfo?taskID=110040031

Comment 3 Jerry James 2023-12-08 21:39:58 UTC
Thank you for the review, Benson.

(In reply to Benson Muite from comment #2)
> a) Can the file src/lusol.txt be added to the documentation?
> %doc src/lusol.txt

Sure.  Wow, that's some ancient history in that file. :-)

> b) Should the mod files lusol.mod and lusol_precision.mod be packaged?
> These are development files typically placed in %{_fmoddir} and devel package
> must then have
> Requires: gcc-gfortran%{_isa}
> see
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Fortran/

Well, we could.  I personally am interested in this package only for purposes of unbundling it from papilo.  For that, I only need the C interface.  We could provide the Fortran interface too, but I'm not sure I want to encourage anyone to use this package.  It seems unmaintained.  As you probably noticed, I've got 2 pull requests from January and a bug report from August that have had no activity at all.  There have been no commits for over a year (and that commit was just adding one line to .gitignore).  I'm thinking of suggesting to the papilo maintainers that they find a more maintained LU solver.

Anyway, I personally have no motivation to package the Fortran interface.  What do you think?  Is it okay to just package the C interface, and add the Fortran interface if somebody shows up who needs it?  (I think that is highly unlikely, by the way.)

> c) Maybe also send a pull request to add soname upstream?

Upstream supports OS X builds as well as Linux builds.  Do you know how sonames are added for OS X?  An upstream patch would have to handle both.

> d) patch lusol-uninit.patch seems logical, but the code has no tests, can
> check
> this, but would need a few days.

Yes, I need to pull my linear algebra textbook off the shelf and figure out if that patch is correct.

> e) Build process can sometimes fail - not sure why:
> https://kojipkgs.fedoraproject.org//work/tasks/8555/110038555/build.log
> Does not happen consistently though:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=110040031

Ah, it's a mistake in the makefile.  Line 146:

# extra fortran dependencies
lusol.o : lusol_precision.mod

But both objects are actually in the src directory, so it's a race to see if lusol_precision.mod is built in time.  I sent a patch upstream: https://github.com/nwh/lusol/pull/14

So I added src/lusol.txt to %doc and added a patch for the makefile race.  The URLs are the same as above.

Comment 4 Jerry James 2024-01-10 21:51:56 UTC
In case it helps this review move forward, I have added the Fortran mods to the devel subpackage.

I looked at the uninit patch again, and I still *think* it is correct, and I am still not 100% sure that it is correct.  As far as I can tell, papilo (which will be the only Fedora user of this package) does not trigger that code path.

Comment 5 Jerry James 2024-01-16 16:01:51 UTC
@benson_muite what else do you want me to do before we can move forward with this review?  It's blocking the entire SoPlex/SCIP effort.

Comment 6 Benson Muite 2024-01-24 07:42:14 UTC
Sorry for the delay. Thanks for adding Fortran modules. Going through it.

Comment 7 Benson Muite 2024-01-28 16:50:26 UTC
The relevant section of code is:
https://github.com/nwh/lusol/blob/master/src/lusol8b.f#L739-L786
A similar section of code is:
https://github.com/nwh/lusol/blob/master/src/lusol.f90#L6479-L6528

In lusol.f90 in routine lu8rpc jrep corresponds to the column being replaced in A and
krep is the modified column in U, A=LU. The main steps are
i) declare singular if all entries in the column krep are zero
ii) if not all entries in column krep are zero, declare singular if the index of the column
number in U being modified is not the same as jrep, the column being replaced in A
iii) if not declared singular, declare singular if the diagonal entry of U corresponding to the
column index is less than some threshold, otherwise it is not singular.

In this case singular is approximately singular, so entry is close enough to zero for numerical
problems.

In lusol8b.f  the routine lu8mod updates the LU factorization A = L*U  when the m by n matrix  A  is
subjected to a rank-one modification to become A  +  beta * v * w(transpose)
kfirst to klast are the modified columns in U. 
i) declare singular if all entries in the column klast are zero
ii) jrep is used here but not declared. Would expect if not all entries in column klast are zero, declare
singular if the index of the last column number in U being modified klast is not the same as the corresponding
location of the modified column in A - this may not be klast due to permutations. A loop is used to find krep
in lusol.f90 :
https://github.com/nwh/lusol/blob/master/src/lusol.f90#L6333-L6336
Probably something similar is needed such as:
       jrep   = n + 1

600     jrep   = jrep - 1
       if (iq(jrep) /= klast) go to 600
This differs slightly from the earlier loop to find klast:
https://github.com/nwh/lusol/blob/master/src/lusol8b.f#L608-L612
iii) if not declared singular, declare singular if the diagonal entry of U corresponding to the column
klast is less than some threshold, otherwise it is not singular.

Description of a portion of the work:
https://apps.dtic.mil/sti/tr/pdf/ADA169255.pdf

Comment 8 Jerry James 2024-01-29 22:34:37 UTC
Benson, you are fantastic!  Thank you so much for the detailed analysis.  I have updated the patch.  If it looks right to you, would you like to submit it upstream so it has your name on it?  I think upstream is dead, so I'm not sure there's much glory to be had, but at least you could get the credit if somebody else looks at the open PRs.

Comment 9 Benson Muite 2024-01-30 17:46:42 UTC
Thanks for the update.  Will go through it. Determining near non-invertibility
of a matrix through approximate sparsity preserving rank revealing factorizations
seems to have less literature than sparsity preserving matrix factorizations. An
earlier Fortran code is at:
https://web.stanford.edu/group/SOL/software/lusol/
A C version seems to also be used in:
https://sourceforge.net/projects/lpsolve/

Comment 10 Jerry James 2024-02-05 15:58:11 UTC
The window of opportunity is closing for me to get SoPlex and SCIP into Fedora before the Fedora 40 beta freeze.  This review has dragged on for 2 months now, not due to any packaging issues, but because of an upstream bug.  I would like to propose some courses of action that would let us move forward.

1. I comment out the known-buggy function.  The consuming project I care about, papilo, does not use the function in question.  We could add a note to %description or to a README noting that the function is unavailable in the Fedora build because of a known bug.

2. We build the upstream code the way it is.  If there is any code out there in the wide world that calls that function, it already has to deal with the consequences of the bug.

3. Like #2, but we continue to work on fixing the bug after this package is added to Fedora.

Can we do one of those please?  The whole SoPlex/SCIP effort is being held up by a bug that will never manifest when they are used.

Comment 11 Benson Muite 2024-02-06 03:53:34 UTC
Option 1 or 2 are fine. In both cases adding a note in the %description or README is good and in both cases 3 is good, will continue to examine the code and pre-print.

Comment 12 Jerry James 2024-02-09 20:48:58 UTC
Thank you, Benson.  I appreciate all the work you have put into analyzing the bug.  I'm going to go with option 2 for now.

Comment 13 Fedora Admin user for bugzilla script actions 2024-02-09 20:49:32 UTC
The Pagure repository was created at https://src.fedoraproject.org/rpms/lusol

Comment 14 Jerry James 2024-02-09 21:10:15 UTC
This package has been built in Rawhide.


Note You need to log in before you can comment on or make changes to this bug.