Bug 1489668 - Review Request: slurm - HPC cluster management and job scheduling
Summary: Review Request: slurm - HPC cluster management and job scheduling
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Robert-André Mauchin 🐧
QA Contact: Fedora Extras Quality Assurance
URL: https://copr.fedorainfracloud.org/cop...
Whiteboard:
: 1149566 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-08 05:33 UTC by Philip Kovacs
Modified: 2017-11-29 08:33 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-11-01 03:43:21 UTC
Type: ---
Embargoed:
eclipseo: fedora-review+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1149566 0 medium CLOSED Review Request: slurm - Simple LinUx Resource Manager 2021-02-22 00:41:40 UTC

Description Philip Kovacs 2017-09-08 05:33:03 UTC
Spec URL: https://github.com/pkovacs/fedora-slurm/blob/master/slurm.spec

To generate the SRPM:

https://github.com/pkovacs/fedora-slurm.git
cd fedora-slurm
spectool -g slurm.spec
fedpkg --release f27 --module slurm srpm

Description: Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters.

https://slurm.schedmd.com/overview.html

Fedora Account System Username: pkfed

Koji scratch builds:

https://koji.fedoraproject.org/koji/taskinfo?taskID=21716655   (f26)
https://koji.fedoraproject.org/koji/taskinfo?taskID=21717543   (f27)

Comment 1 Philip Kovacs 2017-09-08 05:35:31 UTC
To generate the SRPM:

I meant:

git clone https://github.com/pkovacs/fedora-slurm.git

Comment 2 Philip Kovacs 2017-09-08 23:46:58 UTC
I added this project to Copr and will maintain builds there for the time being.

https://copr.fedorainfracloud.org/coprs/pkfed/slurm/

Comment 3 Robert-André Mauchin 🐧 2017-09-11 09:41:25 UTC
 - You need to post a link to the SRPM, not instruction on how to build it, otherwise we can't run fedora-review directly on the bug.

 - Similarly, you need to post a direct link to the spec.

 - You need to close the other bug as duplicate of this one if it's a stalled review.

 - The title of the bug should be: "Review Request: slurm - HPC cluster management and job scheduling" Otherwise you'll have an error message when trying to create the repo with fedrepo-req. So please edit the bug title.

 - You need to remove %defattr(-,root,root,-) in the %files sections, it's not used anymore.

 - This needs to be removed too:

>%clean
>rm -rf %{buildroot}

 - This is not needed:

>Requires(post): /sbin/ldconfig
>Requires(postun): /sbin/ldconfig

 - Don't use:

>Requires(post):   systemd
>Requires(preun):  systemd
>Requires(postun): systemd

But use the special macro:

%{?systemd_requires}
BuildRequires: systemd

See https://fedoraproject.org/wiki/Packaging:Scriptlets?rd=Packaging:ScriptletSnippets#Systemd

 - Replace make %{?_smp_mflags} with the macro %make_build

 - Replace make install DESTDIR=%{buildroot} with the macro %make_install

 - Use pkgconfig when you can for the -devel BR:

BuildRequires:  pkgconfig(gtk+-2.0)
BuildRequires:  hdf5-devel
BuildRequires:  pkgconfig(hwloc)
BuildRequires:  pkgconfig(libfreeipmi)
BuildRequires:  pkgconfig(libcurl)
BuildRequires:  pkgconfig(lua)
BuildRequires:  pkgconfig(mariadb)
BuildRequires:  pkgconfig(munge)
BuildRequires:  pkgconfig(ncurses)
BuildRequires:  pkgconfig(openssl)
BuildRequires:  pam-devel
BuildRequires:  pmix-devel
BuildRequires:  readline-devel
BuildRequires:  pkgconfig(librrd)
BuildRequires:  pkgconfig(zlib)

 - Not everything is under GPLv2, some parts are BSD:

BSD (2 clause)
--------------
slurm-17.02.7/src/common/log.c
slurm-17.02.7/src/common/log.h

BSD (unspecified)
-----------------
slurm-17.02.7/src/common/uthash/LICENSE
slurm-17.02.7/src/common/uthash/uthash.h

  It should be reflected in the License: field

 - If there are GUI app (in slurm-gui) they must have a Desktop file.

 - You need to add:

Requires:       perl(:MODULE_COMPAT_%(eval "`perl -V:version`"; echo $version))

  for your Perl dependent subpackages.


Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed


===== MUST items =====

C/C++:
[x]: Package does not contain kernel modules.
[x]: Package contains no static executables.
[x]: Development (unversioned) .so files in -devel subpackage, if present.
     Note: Unversioned so-files in private %_libdir subdirectory (see
     attachment). Verify they are not in ld path.
[x]: Header files in -devel subpackage, if present.
[x]: ldconfig called in %post and %postun if required.
[x]: Package does not contain any libtool archives (.la)
[x]: Rpath absent or only used for internal libs.

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[!]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses
     found: "GPL", "GPL (v2 or later)", "Unknown or generated", "MIT/X11
     (BSD like)", "BSD (unspecified)", "zlib/libpng", "GPL (v2 or later)
     (with incorrect FSF address)", "*No copyright* GPL (with incorrect FSF
     address)", "ISC", "BSD (2 clause)", "LGPL (v2.1 or later)", "*No
     copyright* GPL (v2 or later)", "GPL (v2)", "PostgreSQL GPL (v2 or
     later)". 1084 files have unknown license. Detailed output of
     licensecheck in /home/bob/packaging/review/slurm/review-
     slurm/licensecheck.txt
[x]: License file installed when any subpackage combination is installed.
[x]: Package must own all directories that it creates.
     Note: Directories without known owners: /usr/lib/.build-id/c5,
     /usr/lib64/perl5, /usr/lib64/perl5/vendor_perl/auto, /usr/lib/.build-
     id/a8, /usr/lib/.build-id/ec, /usr/lib/.build-id/2c, /usr/lib/.build-
     id/90, /usr/lib/.build-id/a9, /usr/lib/.build-id/7b, /etc/slurm,
     /usr/lib/.build-id/d0, /usr/lib/.build-id/ab, /etc/logrotate.d,
     /usr/lib64/security, /usr/lib64/slurm, /usr/lib/.build-id/4f, /usr/lib
     /.build-id/ad, /usr/lib64/perl5/vendor_perl, /usr/lib/.build-id/c6,
     /usr/lib/.build-id/d5, /usr/lib/.build-id/76
[x]: %build honors applicable compiler flags or justifies otherwise.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[!]: Each %files section contains %defattr if rpm < 4.4
     Note: %defattr present but not needed
[!]: Package contains desktop file if it is a GUI application.
[x]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[x]: Package contains systemd file(s) if in need.
[x]: Useful -debuginfo package or justification otherwise.
[x]: Package is not known to require an ExcludeArch tag.
[x]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
     Note: Documentation size is 542720 bytes in 8 files.
[x]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least
     one supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: Package requires other packages for directories it uses.
[x]: Package does not own files or directories owned by other packages.
[x]: All build dependencies are listed in BuildRequires, except for any
     that are listed in the exceptions section of Packaging Guidelines.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: %config files are marked noreplace or the reason is justified.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Dist tag is present.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package use %makeinstall only when make install DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: No %config files under /usr.
[x]: Package does not use a name that already exists.
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as
     provided in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

Perl:
[!]: Package contains the mandatory BuildRequires and Requires:.
     Note: Requires: perl(:MODULE_COMPAT_%(eval "`%{__perl} -V:version`";
     echo $version)) missing?

===== SHOULD items =====

Generic:
[x]: Avoid bundling fonts in non-fonts packages.
     Note: Package contains font files
[-]: If the source package does not include license text(s) as a separate
     file from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[x]: Fully versioned dependency in subpackages if applicable.
     Note: No Requires: %{name}%{?_isa} = %{version}-%{release} in slurm-
     devel , slurm-doc , slurm-gui , slurm-libs , slurm-plugins , slurm-
     plugins-auth_none , slurm-plugins-lua , slurm-plugins-munge , slurm-
     plugins-mysql , slurm-plugins-pbs , slurm-plugins-rrdtool , slurm-
     slurmdbd , slurm-contribs , slurm-openlava , slurm-perlapi , slurm-
     plugins-pam_slurm , slurm-torque , slurm-debuginfo
[?]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[x]: Patches link to upstream bugs/comments/lists or are otherwise
     justified.
[x]: Scriptlets must be sane, if used.
[-]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[x]: Package should compile and build into binary rpms on all supported
     architectures.
[-]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed
     files.
[x]: Files in /run, var/run and /var/lock uses tmpfiles.d when appropriate
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: The placement of pkgconfig(.pc) files are correct.
[x]: Sources can be downloaded from URI in Source: tag
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

===== EXTRA items =====

Generic:
[ ]: Large data in /usr/share should live in a noarch subpackage if package
     is arched.
     Note: Arch-ed rpms have a total of 4229120 bytes in /usr/share
[x]: Rpmlint is run on debuginfo package(s).
     Note: There are rpmlint messages (see attachment).
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Spec file according to URL is the same as in SRPM.

Comment 4 Christopher Brown 2017-09-11 15:29:28 UTC
*** Bug 1149566 has been marked as a duplicate of this bug. ***

Comment 5 Philip Kovacs 2017-09-12 01:11:43 UTC
Hi Robert-Andre,

Thanks for helping with this one.

I made the changes that you asked:

- removed the defattr's
- removed the %clean section
- removed unneeded Requires: ldconfig
- added %systemd_requires
- added %make_build and %make_install
- changed -devel BR's to use pkgconfig()
- added perl Requires to the perlapi sub package
- added BSD to the License
- added .desktop and an icon for the gui sub package
- added BR desktop-file-utils
- added %post gui, %postun gui, %posttrans gui for the icon cache

The spec URL is here:

https://raw.githubusercontent.com/pkovacs/fedora-slurm/master/slurm.spec

The SRPM URL is here:

https://copr-be.cloud.fedoraproject.org/results/pkfed/slurm/fedora-26-x86_64/00601084-slurm/slurm-17.02.7-1.fc26.src.rpm

I just ran an f28 scratch build with koji here (all arches are green):

https://koji.fedoraproject.org/koji/taskinfo?taskID=21805579

Hopefully we're getting close now.  

Do we need to request a reservation of uid/gid from the FPC?   I just used uid/gid 942 as temporary values.

Regards,

Phil

Comment 6 Robert-André Mauchin 🐧 2017-09-12 06:00:21 UTC
>Do we need to request a reservation of uid/gid from the FPC?

I think so.

 - rm -rf %{buildroot} is unneeded in %install too.

The rest is good.

Comment 7 Philip Kovacs 2017-09-13 02:01:40 UTC
I pushed that last little change you requested (remove rm -rf %{buildroot}).

I've also open a ticket with the packaging committee to try and get a dedicated uid/gid for slurm.  Let's see how they respond.

https://pagure.io/packaging-committee/issue/716

Comment 8 Philip Kovacs 2017-09-14 19:11:41 UTC
After a discussion on an upstream forum, it's clear to me now that adding a dedicated slurm user with a static uid is not the best approach for this package.
We don't to make incorrect assumptions about how installations will manage their users and slurm clusters are often managed with ldap-type tools.  End-users of slurm must also have consistent uids on all nodes, so installers will likely turn to ldap/nis, etc. anyway to create a working system.

I am going to alter the rpm spec to remove the slurm user creation, install for root, and leave the user-management issues to the installers.  

I will also add a "bonus" script for Fedora that eases the ownership change of a node to whatever user/group the installer wants after installation.

Comment 9 Philip Kovacs 2017-09-15 00:57:48 UTC
Latest spec URL: https://raw.githubusercontent.com/pkovacs/fedora-slurm/master/slurm.spec

Latest SRPM: https://copr-be.cloud.fedoraproject.org/results/pkfed/slurm/fedora-rawhide-x86_64/00602696-slurm/slurm-17.02.7-1.fc28.src.rpm

Latest koji scratch for f28: https://koji.fedoraproject.org/koji/taskinfo?taskID=21870948

Changes:

 - installs for root
 - no slurm account creation
 - added slurm_setuser script

Comment 10 Robert-André Mauchin 🐧 2017-09-15 03:34:17 UTC
Everything seems good, package accepted.

Comment 11 Adam Huffman 2017-09-15 06:19:25 UTC
Hi Philip,

Many thanks for working on this.

May I ask whether you're planning on creating an EL7 branch?

If not, I'm happy to create one and maintain it.

Cheers,
Adam

Comment 12 Philip Kovacs 2017-09-15 06:52:43 UTC
I was hoping to maintain the Fedora side of things, if I get sponsored.  I have plans to work with openmpi and get that package integrated with these new slurm packages. It's exciting to get slurm going finally.  This package was a lot of work and I'd like to maintain it on the Fedora side if I could.  I'm happy to work with you and if you want to get going on the EL side, be my guest.

Comment 13 Philip Kovacs 2017-09-15 17:05:48 UTC
OK I'm sponsored.  Thank you tibbs.  Will move this ticket along soon.

Comment 14 Gwyn Ciesla 2017-09-15 19:27:56 UTC
(fedrepo-req-admin):  The Pagure repository was created at https://src.fedoraproject.org/rpms/slurm

Comment 15 Fedora Update System 2017-10-27 07:00:12 UTC
slurm-17.02.8-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-a746d91931

Comment 16 Fedora Update System 2017-10-27 18:49:45 UTC
slurm-17.02.8-1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-a746d91931

Comment 17 Fedora Update System 2017-11-11 03:07:15 UTC
slurm-17.02.8-1.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 18 Dmitrij S. Kryzhevich 2017-11-29 04:08:47 UTC
Adam, are you going to maintain epel branch for slurm still? I'm interesting in it too and can pick it up.

Comment 19 Dmitrij S. Kryzhevich 2017-11-29 04:11:22 UTC
Philip, clear the Blocks section of request please as you are not needed to be sponsored anymore.

PS
Sorry for the spam, missed that in prev comment.

Comment 20 Philip Kovacs 2017-11-29 05:47:01 UTC
I'm happy to build for EPEL7 also.

Comment 21 Dmitrij S. Kryzhevich 2017-11-29 08:33:24 UTC
(In reply to Philip Kovacs from comment #20)
> I'm happy to build for EPEL7 also.

You are the owner. You can do it already.

So I'l just wait for it though.


Note You need to log in before you can comment on or make changes to this bug.