Bug 2239811 - Review Request: python-pynsgr - Interface to the NeuroScience Gateway REST interface
Summary: Review Request: python-pynsgr - Interface to the NeuroScience Gateway REST in...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Sandro
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: Trivial
Depends On:
Blocks: fedora-neuro, NeuroFedora
TreeView+ depends on / blocked
 
Reported: 2023-09-20 10:23 UTC by Ankur Sinha (FranciscoD)
Modified: 2023-10-18 01:46 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2023-10-09 10:52:10 UTC
Type: ---
Embargoed:
gui1ty: fedora-review+


Attachments (Terms of Use)
Forge macro compatible use of tag (473 bytes, patch)
2023-09-21 10:57 UTC, Sandro
no flags Details | Diff

Description Ankur Sinha (FranciscoD) 2023-09-20 10:23:39 UTC
Spec URL: https://ankursinha.fedorapeople.org/python-pynsgr/python-pynsgr.spec
SRPM URL: https://ankursinha.fedorapeople.org/python-pynsgr/python-pynsgr-1.0.3-1.fc40.src.rpm

Description:
Python interface to the NeuroScience Gateway REST interface, based on pycipres

Fedora Account System Username: ankursinha

Comment 1 Ankur Sinha (FranciscoD) 2023-09-20 10:23:42 UTC
This package built on koji:  https://koji.fedoraproject.org/koji/taskinfo?taskID=106425893

Comment 2 Sandro 2023-09-21 10:55:38 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed

Issues
======

[x]: License file installed when any subpackage combination is installed.
=> Are you intending to have the doc subpackage installable standalone? If you make it require the main package you could do without the duplicate license file, which the doc subpackage installs separately.

===== MUST items =====

Generic:
[x]: Package successfully compiles and builds into binary rpms on at least
     one supported primary architecture.
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: License field in the package spec file matches the actual license.
[x]: License file installed when any subpackage combination is installed.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[-]: Package contains desktop file if it is a GUI application.
[-]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[?]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[-]: Package contains systemd file(s) if in need.
[x]: Package is not known to require an ExcludeArch tag.
[x]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
[x]: Package complies to the Packaging Guidelines
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
[x]: If (and only if) the source package includes the text of the
     license(s) in its own file, then that file, containing the text of the
     license(s) for the package is included in %license.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Dist tag is present.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package must not depend on deprecated() packages.
[x]: Package use %makeinstall only when make install DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package does not use a name that already exists.
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as
     provided in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

Python:
[x]: Python eggs must not download any dependencies during the build
     process.
[x]: A package which is used by another package via an egg interface should
     provide egg info.
[ ]: Package meets the Packaging Guidelines::Python
[x]: Package contains BR: python2-devel or python3-devel
[x]: Packages MUST NOT have dependencies (either build-time or runtime) on
     packages named with the unversioned python- prefix unless no properly
     versioned package exists. Dependencies on Python packages instead MUST
     use names beginning with python2- or python3- as appropriate.
[x]: Python packages must not contain %{pythonX_site(lib|arch)}/* in %files
[x]: Binary eggs must be removed in %prep

===== SHOULD items =====

Generic:
[x]: Reviewer should test that the package builds in mock.
[-]: If the source package does not include license text(s) as a separate
     file from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[x]: Fully versioned dependency in subpackages if applicable.
[?]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[-]: Sources are verified with gpgverify first in %prep if upstream
     publishes signatures.
[x]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed
     files.
[x]: Spec use %global instead of %define unless justified.
     Note: %define requiring justification: %define autorelease(e:s:pb:n)
     %{?-p:0.}%{lua:
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: SourceX is a working URL.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).


Rpmlint
-------
Checking: python3-pynsgr-1.0.3-1.fc38.noarch.rpm
          python-pynsgr-doc-1.0.3-1.fc38.noarch.rpm
          python-pynsgr-1.0.3-1.fc38.src.rpm
============================ rpmlint session starts ============================
rpmlint: 2.4.0
configuration:
    /usr/lib/python3.11/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora-legacy-licenses.toml
    /etc/xdg/rpmlint/fedora-spdx-licenses.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
rpmlintrc: [PosixPath('/tmp/tmp7gzt0rhf')]
checks: 31, packages: 3

python-pynsgr-doc.noarch: W: file-not-utf8 /usr/share/doc/python-pynsgr-doc/example/example.zip
 3 packages and 0 specfiles checked; 0 errors, 1 warnings, 0 badness; has taken 0.2 s 




Rpmlint (installed packages)
----------------------------
============================ rpmlint session starts ============================
rpmlint: 2.4.0
configuration:
    /usr/lib/python3.11/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora-legacy-licenses.toml
    /etc/xdg/rpmlint/fedora-spdx-licenses.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
checks: 31, packages: 2

python-pynsgr-doc.noarch: W: file-not-utf8 /usr/share/doc/python-pynsgr-doc/example/example.zip
 2 packages and 0 specfiles checked; 0 errors, 1 warnings, 0 badness; has taken 0.0 s 



Source checksums
----------------
https://github.com/OpenSourceBrain/pynsgr/archive/v1.0.3/pynsgr-1.0.3.tar.gz :
  CHECKSUM(SHA256) this package     : e6801db13f9af70303cb4f24c27750d1a881e48c31aa62c89c0062144e404531
  CHECKSUM(SHA256) upstream package : e6801db13f9af70303cb4f24c27750d1a881e48c31aa62c89c0062144e404531


Requires
--------
python3-pynsgr (rpmlib, GLIBC filtered):
    /usr/bin/python3
    python(abi)
    python3.11dist(requests)

python-pynsgr-doc (rpmlib, GLIBC filtered):



Provides
--------
python3-pynsgr:
    python-pynsgr
    python3-pynsgr
    python3.11-pynsgr
    python3.11dist(pynsgr)
    python3dist(pynsgr)

python-pynsgr-doc:
    python-pynsgr-doc

Comment 3 Sandro 2023-09-21 10:57:30 UTC
Created attachment 1989819 [details]
Forge macro compatible use of tag

This way you only ever need to change `Version:`, which makes the package compatible with Anitya and Packit.

Comment 4 Ben Beasley 2023-09-21 11:43:24 UTC
I had started having a look at this yesterday, and I found a couple of additional things:

===== Issues =====

- While all of the files from which you remove shebangs have “main routines”
  and are able to be used as scripts (via e.g. 'python3 -m pynsgr.client' or
  'python3 /path/to/pynsgr/client.py'), none of them has the executable bit set
  in the filesystem permissions even in the upstream git repository. The
  shebangs are therefore useless even to upstream; in the spirit of staying
  close to upstream, I suggest sending a PR to remove them and applying the PR
  as a patch.

- The generated man pages are broken, e.g.:

    NSGR_JOB(1)                      User Commands                     NSGR_JOB(1)

    NAME   
           nsgr_job - manual page for nsgr_job 1.0.3

    DESCRIPTION
       Traceback (most recent call last):
                  File                                     "/builddir/build/BUILD‐
                  ROOT/python-pynsgr-1.0.3-1.fc40.x86_64//usr/bin/nsgr_job",  line
                  5, in <module>

                  from pynsgr.commands.nsgr_job import main

           ModuleNotFoundError: No module named 'pynsgr'

    nsgr_job 1.0.3                  September 2023                     NSGR_JOB(1)


  You must set the environment variable PYTHONPATH to
  '%{buildroot}%{python3_sitelib}' when invoking help2man.

  Once this is fixed, you don’t need --no-discard-stderr.

  While you are at it, consider using a shell loop to reduce repetition and
  make it immediately clear that both man pages are generated with the same
  options. Below, I have also fiddled with the formatting of the help2man
  options, but that is much deeper into the territory of personal preferences.

    install -d '%{buildroot}%{_mandir}/man1'
    for cmd in nsgr_job nsgr_submit
    do
      help2man \
          --no-info \
          --version-string=%{version} \
          --help-option=-h \
          --output="%{buildroot}%{_mandir}/man1/${cmd}.1" \
          "%{buildroot}/%{_bindir}/${cmd}"
    done

  The result is still far from beautiful, but at least does contain useful
  text.

===== Notes (no change required for approval) =====

- In Issues, I requested that the shebang removal be upstreamed, which makes
  the following commentary useless to this package (but relevant in general):

  In the shebang removal, I would tend to pass all the files to a single sed
  invocation to avoid having to repeat the expression, and so that a person
  reading the spec file can immediately see that all files are transformed in
  exactly the same way.

    # remove shebangs
    sed -i '/^#!.*\/usr\/bin\/env.*python/ d' \
        pynsgr/client.py \
        pynsgr/commands/nsgr_submit.py \
        pynsgr/commands/nsgr_job.py \
        pynsgr/pyjavaproperties.py

  Furthermore, for shebang removal, I like matching on just '^#!', but
  requiring the match to be on the first line of the file; after all, if
  upstream changed the shebangs in non-executable modules to /usr/bin/python3,
  we would still want to remove them.

    # remove shebangs
    sed -i -r '1{/^#!/d}' \
        pynsgr/client.py \
        pynsgr/commands/nsgr_submit.py \
        pynsgr/commands/nsgr_job.py \
        pynsgr/pyjavaproperties.py

Comment 5 Sandro 2023-09-23 21:17:04 UTC
(In reply to Sandro from comment #3)
> This way you only ever need to change `Version:`, which makes the package
> compatible with Anitya and Packit.

I just found out, plumbing on another package, that uses the 'v' prefix for the tags, that %forgemeta is smart enough to determine the prefix all by itself for composing the source url. So, in fact, you can omit the `%global tag` completely. I assume, haven't verified, this works as long as upstream releases on GitHub. I have a faint memory of it being needed when upstream doesn't release on GitHub.

Comment 6 Ankur Sinha (FranciscoD) 2023-10-02 09:53:35 UTC
Thanks for the reviews! Here are the updated spec/srpm:

Spec URL: https://ankursinha.fedorapeople.org/python-pynsgr/python-pynsgr.spec
SRPM URL: https://ankursinha.fedorapeople.org/python-pynsgr/python-pynsgr-1.0.3-2.fc40.src.rpm

Comment 7 Sandro 2023-10-04 14:04:16 UTC
Based on Ben's feedback, I took a closer look at the manpages. I agree:

> The result is still far from beautiful, but at least does contain useful text.

At least it's no longer broken. I wonder how helpful it is having a man page when the output is almost identical to the output of the '--help' switch. But that's more a philosophical question and `nsgr_job` appears to lack a `--help` switch all together (it uses '-h' instead).

While I was busy I noted that `nsgr_job` crashes without any options. I'd say that's a bug:

Traceback (most recent call last):
  File "/usr/bin/nsgr_job", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/pynsgr/commands/nsgr_job.py", line 162, in main
    sys.exit(nsgr_job(sys.argv))
             ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pynsgr/commands/nsgr_job.py", line 86, in nsgr_job
    properties = CipresClient.Application(conf_filepath).getProperties()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pynsgr/client.py", line 499, in __init__
    raise Exception(
Exception: Didn't find the file: ~/.nsgrest.conf or ~/nsgrest.conf (which should contain properties {'USERNAME', 'URL', 'APPID', 'APPNAME', 'PASSWORD'}) or /opt/user/nsgrest.conf on Open Source Brain v2.

Probably not a reason to not grant the review, but I'll ponder that while I'm afk and decide later

Comment 8 Ben Beasley 2023-10-04 14:16:22 UTC
(In reply to Sandro from comment #7)
> Based on Ben's feedback, I took a closer look at the manpages. I agree:
> 
> > The result is still far from beautiful, but at least does contain useful text.
> 
> At least it's no longer broken. I wonder how helpful it is having a man page
> when the output is almost identical to the output of the '--help' switch.
> But that's more a philosophical question and `nsgr_job` appears to lack a
> `--help` switch all together (it uses '-h' instead).

As a matter of purely personal preference, I am *much* happier typing

  man foobar

than

  foobar --help | less

or when the command doesn’t follow conventions, maybe something like

  foobar -h 2>&1 | less -S

I find a man page more ergonomic even if it is merely a lightly-formatted regurgitation of the help text.  

> While I was busy I noted that `nsgr_job` crashes without any options. I'd
> say that's a bug:
> 
> [...]
> Exception: Didn't find the file: ~/.nsgrest.conf or ~/nsgrest.conf (which
> should contain properties {'USERNAME', 'URL', 'APPID', 'APPNAME',
> 'PASSWORD'}) or /opt/user/nsgrest.conf on Open Source Brain v2.
> 
> Probably not a reason to not grant the review, but I'll ponder that while
> I'm afk and decide later

I would say it’s a bug, because spewing a traceback is an ugly way to report an error, but at least the exception message is useful (basically: “you need to configure authentication credentials first, and here’s where you should put them.”)

Comment 9 Sandro 2023-10-04 21:58:54 UTC
(In reply to Ben Beasley from comment #8)
> I find a man page more ergonomic even if it is merely a lightly-formatted
> regurgitation of the help text.

But what we have now looks kinda ugly. And in this specific case, I would resort to the help text, rather than using the man pages. The formatting (linebreaks, etc,) is just not very readable. Compare:

    -r
        remove a job.  Deletes input and output data and all info about the job.  Cancels
        the job if it's waiting to run or running.

    For example:
        nsgr_job -l
            list the user's jobs
        nsgr_job -j JOBHANDLE
            shows status of the job whose jobhandle is JOBHANDLE
        nsgr_job -j JOBHANDLE -d
            download's results of the job whose jobhandle is JOBHANDLE
        nsgr_job -j JOBHANDLE -r
            cancel and remove the specified job.

to:

       -r

       remove a job.
              Deletes input and output data and all info about the job.  Cancels

              the job if it's waiting to run or running.

              For example:

              nsgr_job -l

              list the user's jobs

              nsgr_job -j JOBHANDLE

              shows status of the job whose jobhandle is JOBHANDLE

              nsgr_job -j JOBHANDLE -d

              download's results of the job whose jobhandle is JOBHANDLE

              nsgr_job -j JOBHANDLE -r

              cancel and remove the specified job.

The summum of doing this right is the way `git` does it, in my opinion. But, as I already mentioned, at least the man pages are no longer broken. And they are not blocking this review being granted.

> I would say it’s a bug, because spewing a traceback is an ugly way to report
> an error, but at least the exception message is useful (basically: “you need
> to configure authentication credentials first, and here’s where you should
> put them.”)

But this may well be. I think the exception should be caught and only the message should be shown to the user.

I'll sleep over it and give Ankur a chance to comment.

Comment 10 Ben Beasley 2023-10-05 11:18:41 UTC
I’m happy to write beautiful man pages by hand if someone wants to maintain them.

I’m also happy to write PR’s like https://github.com/Donders-Institute/multiecho/pull/19 in cases where upstream wants it, but in this case it would require rewriting the option parsing to use argparse.

Comment 11 Sandro 2023-10-05 11:37:52 UTC
(In reply to Ben Beasley from comment #10)
> I’m also happy to write PR’s like
> https://github.com/Donders-Institute/multiecho/pull/19 in cases where
> upstream wants it, but in this case it would require rewriting the option
> parsing to use argparse.

I was wondering, why not use argparse. It's my go to choice when it comes to Python command line utilities. I was quite surprised that the two tools provided behaved differently wrt the switches (`-h` vs. `--help`).

Since the person requesting this review is also involved upstream, let's hear him.

Comment 12 Ankur Sinha (FranciscoD) 2023-10-05 13:01:15 UTC
Thanks for the reviews, folks.

Yeh, quite a few things can be done to modernise the code. It's basically something that was written years ago (by someone else) that we've now started to re-maintain. You'll see that I've done the basics of adding tests and all that but we haven't touched the original code much yet. I think the way to go here would be to file issues for what we've found here so I can work on them for the next release. That way we don't block this review, and it'll give me time to work on things. We've got a few urgent deadlines coming up and I'm still trying to work on vxl and the other packages, so this will take a couple of weeks to get to.

How does that sound?

Comment 13 Sandro 2023-10-08 17:32:26 UTC
(In reply to Ankur Sinha (FranciscoD) from comment #12)
> Thanks for the reviews, folks.
> 
> Yeh, quite a few things can be done to modernise the code. It's basically
> something that was written years ago (by someone else) that we've now
> started to re-maintain. You'll see that I've done the basics of adding tests
> and all that but we haven't touched the original code much yet. I think the
> way to go here would be to file issues for what we've found here so I can
> work on them for the next release. That way we don't block this review, and
> it'll give me time to work on things. We've got a few urgent deadlines
> coming up and I'm still trying to work on vxl and the other packages, so
> this will take a couple of weeks to get to.
> 
> How does that sound?

Sounds good to me. Package is APPROVED.

Comment 14 Fedora Admin user for bugzilla script actions 2023-10-09 10:32:08 UTC
The Pagure repository was created at https://src.fedoraproject.org/rpms/python-pynsgr

Comment 15 Ankur Sinha (FranciscoD) 2023-10-09 10:33:28 UTC
Thanks very much. I've filed upstream issues for what we've discussed here. I'll try to fix them in the next release:

https://github.com/OpenSourceBrain/pynsgr/issues/13
https://github.com/OpenSourceBrain/pynsgr/issues/12

Comment 16 Fedora Update System 2023-10-09 10:49:25 UTC
FEDORA-2023-a9159663c1 has been submitted as an update to Fedora 40. https://bodhi.fedoraproject.org/updates/FEDORA-2023-a9159663c1

Comment 17 Fedora Update System 2023-10-09 10:52:10 UTC
FEDORA-2023-a9159663c1 has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 18 Fedora Update System 2023-10-09 12:17:56 UTC
FEDORA-2023-4691e80fd9 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-4691e80fd9

Comment 19 Fedora Update System 2023-10-10 02:37:45 UTC
FEDORA-2023-4691e80fd9 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf install --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-4691e80fd9 \*`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-4691e80fd9

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 20 Fedora Update System 2023-10-18 01:46:16 UTC
FEDORA-2023-4691e80fd9 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.