Bug 2283064 - Review Request: python-detect-secrets - Detect secrets within a code base
Summary: Review Request: python-detect-secrets - Detect secrets within a code base
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: wojnilowicz
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/Yelp/detect-secrets
Whiteboard:
Depends On:
Blocks: 2283066
TreeView+ depends on / blocked
 
Reported: 2024-05-23 21:33 UTC by Dominik 'Rathann' Mierzejewski
Modified: 2024-09-30 18:41 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2024-09-30 18:41:46 UTC
Type: ---
Embargoed:
lukasz.wojnilowicz: fedora-review+


Attachments (Terms of Use)
The .spec file difference from Copr build 7482992 to 8042289 (2.05 KB, patch)
2024-09-20 08:53 UTC, Fedora Review Service
no flags Details | Diff
The .spec file difference from Copr build 8042289 to 8091460 (3.73 KB, patch)
2024-09-30 06:00 UTC, Fedora Review Service
no flags Details | Diff

Description Dominik 'Rathann' Mierzejewski 2024-05-23 21:33:17 UTC
Spec URL: https://rathann.fedorapeople.org/review/python-detect-secrets/python-detect-secrets.spec
SRPM URL: https://rathann.fedorapeople.org/review/python-detect-secrets/python-detect-secrets-1.5.0-1.fc41.src.rpm
Description:
detect-secrets is an aptly named module for (surprise, surprise) detecting
secrets within a code base.

However, unlike other similar packages that solely focus on finding secrets,
this package is designed with the enterprise client in mind: providing a
backwards compatible, systematic means of:

* Preventing new secrets from entering the code base,
* Detecting if such preventions are explicitly bypassed, and
* Providing a checklist of secrets to roll, and migrate off to a more secure
  storage.

Fedora Account System Username: rathann

Comment 1 Fedora Review Service 2024-05-23 21:40:09 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/7482992
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2283064-python-detect-secrets/fedora-rawhide-x86_64/07482992-python-detect-secrets/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 2 wojnilowicz 2024-09-10 15:56:15 UTC
Do you still want it to be reviewed?
Would you like to do a swap review with https://bugzilla.redhat.com/show_bug.cgi?id=2296174 ?

Comment 3 Dominik 'Rathann' Mierzejewski 2024-09-20 08:47:37 UTC
Spec URL: https://rathann.fedorapeople.org/review/python-detect-secrets/python-detect-secrets.spec
SRPM URL: https://rathann.fedorapeople.org/review/python-detect-secrets/python-detect-secrets-1.5.0-1.fc42.src.rpm

Fixed build on rawhide.

@lukasz.wojnilowicz Yes, I'll review yours.

Comment 4 Fedora Review Service 2024-09-20 08:53:30 UTC
Created attachment 2047821 [details]
The .spec file difference from Copr build 7482992 to 8042289

Comment 5 Fedora Review Service 2024-09-20 08:53:32 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/8042289
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2283064-python-detect-secrets/fedora-rawhide-x86_64/08042289-python-detect-secrets/fedora-review/review.txt

Found issues:

- python3-pytest7 is deprecated, you must not depend on it.
  Read more: https://docs.fedoraproject.org/en-US/packaging-guidelines/deprecating-packages/

Please know that there can be false-positives.

---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 6 wojnilowicz 2024-09-21 13:36:45 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed


Issues:
=======
- Package must not depend on deprecated() packages.
  Note: python3-pytest7 is deprecated, you must not depend on it.
  See: https://docs.fedoraproject.org/en-US/packaging-
  guidelines/deprecating-packages/

Shouldn't you use "BuildRequires: python3dist(pytest)" instead of "BuildRequires:  python3-pytest" as mentioned at https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_test_dependencies_2 to make this issue go away?

Could you fix those two warnings with the help of https://docs.fedoraproject.org/en-US/packaging-guidelines/#_manpages ?
python3-detect-secrets.noarch: W: no-manual-page-for-binary detect-secrets
python3-detect-secrets.noarch: W: no-manual-page-for-binary detect-secrets-hook

===== MUST items =====

Generic:
[x]: Package successfully compiles and builds into binary rpms on at least
     one supported primary architecture.
     Note: Using prebuilt packages
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses
     found: "Unknown or generated", "*No copyright* Apache License 2.0",
     "*No copyright* Apache License", "MIT License". 200 files have unknown
     license. Detailed output of licensecheck in /var/lib/copr-
     rpmbuild/results/python-detect-secrets/licensecheck.txt
[x]: Package must own all directories that it creates.
     Note: Directories without known owners: /usr/lib/python3.13,
     /usr/lib/python3.13/site-packages
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[x]: Macros in Summary, %description expandable at SRPM build time.
     Note: Macros in: python3-detect-secrets (description)
[-]: Package contains desktop file if it is a GUI application.
[-]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[!]: Requires correct, justified where necessary.
Builds without "BuildRequires:  python3-pytest-xdist" and not justified.

The following ones are picked up by "%pyproject_buildrequires -x word_list"
BuildRequires:  python3dist(pyahocorasick)
BuildRequires:  python3dist(pyyaml)
BuildRequires:  python3dist(requests)
Build fails without them for you? If not, they should be removed.

The following ones should be justified (tests fail?)
BuildRequires:  python3dist(responses)
BuildRequires:  python3dist(unidiff)

[x]: Spec file is legible and written in American English.
[-]: Package contains systemd file(s) if in need.
[x]: Package is not known to require an ExcludeArch tag.
[-]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
     Note: Documentation size is 67783 bytes in 4 files.
[x]: Package complies to the Packaging Guidelines
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: If (and only if) the source package includes the text of the
     license(s) in its own file, then that file, containing the text of the
     license(s) for the package is included in %license.
[x]: The License field must be a valid SPDX expression.
[x]: Package requires other packages for directories it uses.
[x]: Package does not own files or directories owned by other packages.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Dist tag is present.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package use %makeinstall only when make install DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package does not use a name that already exists.
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as
     provided in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

Python:
[-]: Python eggs must not download any dependencies during the build
     process.
[-]: A package which is used by another package via an egg interface should
     provide egg info.
[x]: Package meets the Packaging Guidelines::Python
[x]: Package contains BR: python2-devel or python3-devel
[x]: Packages MUST NOT have dependencies (either build-time or runtime) on
     packages named with the unversioned python- prefix unless no properly
     versioned package exists. Dependencies on Python packages instead MUST
     use names beginning with python2- or python3- as appropriate.
[x]: Python packages must not contain %{pythonX_site(lib|arch)}/* in %files
[x]: Binary eggs must be removed in %prep

===== SHOULD items =====

Generic:
[x]: Reviewer should test that the package builds in mock.
[-]: If the source package does not include license text(s) as a separate
     file from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[?]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[-]: Sources are verified with gpgverify first in %prep if upstream
     publishes signatures.
     Note: gpgverify is not used.
[-]: Package should compile and build into binary rpms on all supported
     architectures.
[!]: %check is present and all tests pass.
https://github.com/Yelp/detect-secrets/issues/875 justifies exclusion of only 9 failing test. You exclude 33 of them. What's the justification for the remaining tests? 
[x]: Packages should try to preserve timestamps of original installed
     files.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
     Note: No rpmlint messages.


Rpmlint
-------
Checking: python3-detect-secrets-1.5.0-1.fc42.noarch.rpm
          python-detect-secrets-1.5.0-1.fc42.src.rpm
============================ rpmlint session starts ============================
rpmlint: 2.5.0
configuration:
    /usr/lib/python3.12/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora-legacy-licenses.toml
    /etc/xdg/rpmlint/fedora-spdx-licenses.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
rpmlintrc: [PosixPath('/tmp/tmp509cqb75')]
checks: 32, packages: 2

python-detect-secrets.src: E: spelling-error ('preventions', "%description -l en_US preventions -> prevention, prevention's, prevent ions")
python3-detect-secrets.noarch: E: spelling-error ('preventions', "%description -l en_US preventions -> prevention, prevention's, prevent ions")
python3-detect-secrets.noarch: W: no-manual-page-for-binary detect-secrets
python3-detect-secrets.noarch: W: no-manual-page-for-binary detect-secrets-hook
 2 packages and 0 specfiles checked; 2 errors, 2 warnings, 9 filtered, 2 badness; has taken 0.5 s 




Rpmlint (installed packages)
----------------------------
(none): E: there is no installed rpm "python3-detect-secrets".
There are no files to process nor additional arguments.
Nothing to do, aborting.
============================ rpmlint session starts ============================
rpmlint: 2.5.0
configuration:
    /usr/lib/python3.13/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora-spdx-licenses.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
checks: 32, packages: 1

 0 packages and 0 specfiles checked; 0 errors, 0 warnings, 0 filtered, 0 badness; has taken 0.0 s 



Source checksums
----------------
https://github.com/Yelp/detect-secrets/archive/v1.5.0/python-detect-secrets-1.5.0.tar.gz :
  CHECKSUM(SHA256) this package     : 4789cadf12468316f09317b825c4b4beb869bd9777afcbf842079c5d18f07107
  CHECKSUM(SHA256) upstream package : 4789cadf12468316f09317b825c4b4beb869bd9777afcbf842079c5d18f07107


Requires
--------
python3-detect-secrets (rpmlib, GLIBC filtered):
    /usr/bin/python3
    python(abi)
    python3.13dist(pyyaml)
    python3.13dist(requests)



Provides
--------
python3-detect-secrets:
    python-detect-secrets
    python3-detect-secrets
    python3.13-detect-secrets
    python3.13dist(detect-secrets)
    python3dist(detect-secrets)



Generated by fedora-review 0.10.0 (e79b66b) last change: 2023-07-24
Command line :/bin/fedora-review --no-colors --prebuilt --rpm-spec --name python-detect-secrets --mock-config /var/lib/copr-rpmbuild/results/configs/child.cfg
Buildroot used: fedora-rawhide-x86_64
Active plugins: Python, Shell-api, Generic
Disabled plugins: fonts, Ocaml, R, SugarActivity, Java, Perl, Haskell, C/C++, PHP
Disabled flags: EXARCH, EPEL6, EPEL7, DISTTAG, BATCH

Comment 7 Dominik 'Rathann' Mierzejewski 2024-09-23 11:32:51 UTC
(In reply to wojnilowicz from comment #6)
> Package Review
> ==============
> 
> Legend:
> [x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
> [ ] = Manual review needed
> 
> 
> Issues:
> =======
> - Package must not depend on deprecated() packages.
>   Note: python3-pytest7 is deprecated, you must not depend on it.
>   See: https://docs.fedoraproject.org/en-US/packaging-
>   guidelines/deprecating-packages/
> 
> Shouldn't you use "BuildRequires: python3dist(pytest)" instead of
> "BuildRequires:  python3-pytest" as mentioned at
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/
> #_test_dependencies_2 to make this issue go away?

Switched to python3dist(foo) scheme.

> Could you fix those two warnings with the help of
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#_manpages ?
> python3-detect-secrets.noarch: W: no-manual-page-for-binary detect-secrets
> python3-detect-secrets.noarch: W: no-manual-page-for-binary
> detect-secrets-hook
> 
> ===== MUST items =====
[...] 
> [!]: Requires correct, justified where necessary.
> Builds without "BuildRequires:  python3-pytest-xdist" and not justified.

Removed.

> The following ones are picked up by "%pyproject_buildrequires -x word_list"
> BuildRequires:  python3dist(pyahocorasick)
> BuildRequires:  python3dist(pyyaml)
> BuildRequires:  python3dist(requests)
> Build fails without them for you? If not, they should be removed.

Removed.

> The following ones should be justified (tests fail?)
> BuildRequires:  python3dist(responses)
> BuildRequires:  python3dist(unidiff)

Yes.
tests/conftest.py:6: in <module>
    import responses
E   ModuleNotFoundError: No module named 'responses'
and
E           NotImplementedError: SecretsCollection.scan_diff requires `unidiff` to work. Try pip installing that package, and try again.

> ===== SHOULD items =====
[...]
> [!]: %check is present and all tests pass.
> https://github.com/Yelp/detect-secrets/issues/875 justifies exclusion of
> only 9 failing test. You exclude 33 of them. What's the justification for
> the remaining tests? 

Some require an unpackaged dependency (gibberish-detector).
Others require git and upstream git repo clone as part of the tarball, but
GitHub-generated tarball doesn't include git metadata. The standard way
is to use a tarball, so I disabled the git tests and did not include git-core
as a build dependency. What do you suggest instead?

Comment 8 wojnilowicz 2024-09-24 16:04:03 UTC
(In reply to Dominik 'Rathann' Mierzejewski from comment #7)
> > The following ones should be justified (tests fail?)
> > BuildRequires:  python3dist(responses)
> > BuildRequires:  python3dist(unidiff)
> 
> Yes.
> tests/conftest.py:6: in <module>
>     import responses
> E   ModuleNotFoundError: No module named 'responses'
> and
> E           NotImplementedError: SecretsCollection.scan_diff requires
> `unidiff` to work. Try pip installing that package, and try again.
Ok, then please add a comment to the SPEC file so that they'll be justified.

> > ===== SHOULD items =====
> [...]
> > [!]: %check is present and all tests pass.
> > https://github.com/Yelp/detect-secrets/issues/875 justifies exclusion of
> > only 9 failing test. You exclude 33 of them. What's the justification for
> > the remaining tests? 
> 
> Some require an unpackaged dependency (gibberish-detector).
> Others require git and upstream git repo clone as part of the tarball, but
> GitHub-generated tarball doesn't include git metadata. The standard way
> is to use a tarball, so I disabled the git tests and did not include git-core
> as a build dependency. What do you suggest instead?

I just tried to run test_modifies_baseline (it was not listed on github) and it fails due to git as well. It seems that you did the right thing.

Please update your SPEC file, and I'll approve it.

Comment 9 wojnilowicz 2024-09-24 16:05:57 UTC
Did you miss that?

> Could you fix those two warnings with the help of
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#_manpages ?
> python3-detect-secrets.noarch: W: no-manual-page-for-binary detect-secrets
> python3-detect-secrets.noarch: W: no-manual-page-for-binary
> detect-secrets-hook

Comment 10 Dominik 'Rathann' Mierzejewski 2024-09-29 20:04:20 UTC
(In reply to wojnilowicz from comment #8)
> (In reply to Dominik 'Rathann' Mierzejewski from comment #7)
> > > The following ones should be justified (tests fail?)
> > > BuildRequires:  python3dist(responses)
> > > BuildRequires:  python3dist(unidiff)
> > 
> > Yes.
> > tests/conftest.py:6: in <module>
> >     import responses
> > E   ModuleNotFoundError: No module named 'responses'
> > and
> > E           NotImplementedError: SecretsCollection.scan_diff requires
> > `unidiff` to work. Try pip installing that package, and try again.
> Ok, then please add a comment to the SPEC file so that they'll be justified.

It's actually the %pyproject_buildrequires -t that tries to install pytest7
(and fails), because pytest=7 is listed in requirements-dev.txt referenced
in tox.ini.

So, I fixed tox.ini to use requirements-dev-minimal.txt and dropped a few
unnecessary ones from there, as well.

> > > ===== SHOULD items =====
> > [...]
> > > [!]: %check is present and all tests pass.
> > > https://github.com/Yelp/detect-secrets/issues/875 justifies exclusion of
> > > only 9 failing test. You exclude 33 of them. What's the justification for
> > > the remaining tests? 
> > 
> > Some require an unpackaged dependency (gibberish-detector).
> > Others require git and upstream git repo clone as part of the tarball, but
> > GitHub-generated tarball doesn't include git metadata. The standard way
> > is to use a tarball, so I disabled the git tests and did not include git-core
> > as a build dependency. What do you suggest instead?
> 
> I just tried to run test_modifies_baseline (it was not listed on github) and
> it fails due to git as well. It seems that you did the right thing.
> 
> Please update your SPEC file, and I'll approve it.

Done.

Spec URL: https://rathann.fedorapeople.org/review/python-detect-secrets/python-detect-secrets.spec
SRPM URL: https://rathann.fedorapeople.org/review/python-detect-secrets/python-detect-secrets-1.5.0-2.fc42.src.rpm

* Mon Sep 23 2024 Dominik Mierzejewski <dominik> 1.5.0-2
- use automated dependency generator for test deps, too
- drop unnecessary explicit BuildRequires
- drop incomplete support for building with alternative pythons
- generate a manpage with help2man

Comment 11 Fedora Review Service 2024-09-30 06:00:36 UTC
Created attachment 2049472 [details]
The .spec file difference from Copr build 8042289 to 8091460

Comment 12 Fedora Review Service 2024-09-30 06:00:38 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/8091460
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2283064-python-detect-secrets/fedora-rawhide-x86_64/08091460-python-detect-secrets/fedora-review/review.txt

Please take a look if any issues were found.


---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 13 wojnilowicz 2024-09-30 07:31:06 UTC
Every issue is solved now. You even removed this odd looking "python3_pkgversion" :) Great. The package is APPROVED.

Could you just add a comment to this line 
"sed -i -e 's/requirements-dev.txt/requirements-dev-minimal.txt/g' tox.ini"
with your above explanation before commiting it? Without knowing about this review request it's a bit unclear why did you do that.

Comment 14 Fedora Admin user for bugzilla script actions 2024-09-30 11:16:40 UTC
The Pagure repository was created at https://src.fedoraproject.org/rpms/python-detect-secrets

Comment 15 Fedora Update System 2024-09-30 12:51:01 UTC
FEDORA-2024-d1cb2e55ca (python-detect-secrets-1.5.0-1.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2024-d1cb2e55ca

Comment 16 Fedora Update System 2024-09-30 18:41:46 UTC
FEDORA-2024-d1cb2e55ca (python-detect-secrets-1.5.0-1.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.