Bug 2115066 - Review Request: yle-dl - Download videos from Yle servers
Summary: Review Request: yle-dl - Download videos from Yle servers
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Ville-Pekka Vainio
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-08-03 19:35 UTC by Otto Liljalaakso
Modified: 2022-09-24 18:09 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2022-09-24 18:09:21 UTC
Type: ---
Embargoed:
vpvainio: fedora-review+


Attachments (Terms of Use)

Description Otto Liljalaakso 2022-08-03 19:35:50 UTC
Spec URL: http://oturpe.kapsi.fi/fedora/rpms/yle-dl/yle-dl.spec
SRPM URL: http://oturpe.kapsi.fi/fedora/rpms/yle-dl/yle-dl-20220704-1.fc37.src.rpm
Description: yle-dl is a command-line program for downloading media files from the three video
streaming services of the Finnish national broadcasting company Yle: Yle Areena,
Elävä arkisto, and Yle news.
Fedora Account System Username: oturpe

This is my first Python package.

Scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=90440048

Comment 1 Salman Butt 2022-08-04 02:51:21 UTC
UNOFFICIAL REVIEW:
------------------

POSSIBLE ISSUES:

Diff spec file in url and in SRPM.
May need to check re upload the specfile
---------------------------------
--- /review-yle-dl/srpm/yle-dl.spec	2022-08-03 22:15:29.014435833 -0400
+++ /review-yle-dl/srpm-unpacked/yle-dl.spec	2022-08-03 15:19:53.000000000 -0400
@@ -16,5 +16,5 @@
@IBLE 
 %description
-Command-line program for downloading media files from the three video
+yle-dl is a command-line program for downloading media files from the two video
 streaming services of the Finnish national broadcasting company Yle: Yle Areena,
 Elävä arkisto, and Yle news.

-----------------------------------

Requires:       /usr/bin/ffmpeg
Will possibly be need to be changed to ffpmeg-free

Recommends:     /usr/bin/wget
Will possibly be need to be changed to wget 

However file/dir  dependency is valid: https://docs.fedoraproject.org/en-US/packaging-guidelines/#_file_and_directory_dependencies
Will leave it to the official reviewer

----------------------------------------------------

Licnese Unknown or generated
--------------------
yle-dl-20220704/COPYING
yle-dl-20220704/MANIFEST.in
yle-dl-20220704/README.fi
yle-dl-20220704/setup.cfg
yle-dl-20220704/tests/conftest.py
yle-dl-20220704/tests/integration/test_areena_it.py
yle-dl-20220704/tests/integration/test_areena_radio_it.py
yle-dl-20220704/tests/integration/test_arkisto_it.py
yle-dl-20220704/tests/integration/test_arkivet_it.py
yle-dl-20220704/tests/integration/test_uutiset_it.py
yle-dl-20220704/tests/test_backend.py
yle-dl-20220704/tests/test_downloader.py
yle-dl-20220704/tests/test_flavor_filters.py
yle-dl-20220704/tests/test_timestamp.py
yle-dl-20220704/tests/test_title_formatter.py
yle-dl-20220704/tests/utils.py
yle-dl-20220704/yle_dl.egg-info/SOURCES.txt
yle-dl-20220704/yle_dl.egg-info/dependency_links.txt
yle-dl-20220704/yle_dl.egg-info/entry_points.txt
yle-dl-20220704/yle_dl.egg-info/requires.txt
yle-dl-20220704/yle_dl.egg-info/top_level.txt
yle-dl-20220704/yledl/__init__.py
yle-dl-20220704/yledl/__main__.py
yle-dl-20220704/yledl/backends.py
yle-dl-20220704/yledl/downloader.py
yle-dl-20220704/yledl/errors.py
yle-dl-20220704/yledl/exitcodes.py
yle-dl-20220704/yledl/extractors.py
yle-dl-20220704/yledl/ffprobe.py
yle-dl-20220704/yledl/geolocation.py
yle-dl-20220704/yledl/http.py
yle-dl-20220704/yledl/io.py
yle-dl-20220704/yledl/kaltura.py
yle-dl-20220704/yledl/localization.py
yle-dl-20220704/yledl/streamfilters.py
yle-dl-20220704/yledl/streamflavor.py
yle-dl-20220704/yledl/streamprobe.py
yle-dl-20220704/yledl/subtitles.py
yle-dl-20220704/yledl/timestamp.py
yle-dl-20220704/yledl/titleformatter.py
yle-dl-20220704/yledl/utils.py
yle-dl-20220704/yledl/version.py

------
Not going to comment on any python, since im not an expert.
Hope this helps

Comment 2 Otto Liljalaakso 2022-08-04 07:48:03 UTC
(In reply to Salman Butt from comment #1)
> UNOFFICIAL REVIEW:
> ------------------

Thank you for the feedback!

> POSSIBLE ISSUES:
> 
> Diff spec file in url and in SRPM.
> May need to check re upload the specfile

Yes, something was wrong.
I updated the description and bumped release.
Latest specfile and rpm are here:
Spec URL: http://oturpe.kapsi.fi/fedora/rpms/yle-dl/yle-dl.spec
SRPM URL: http://oturpe.kapsi.fi/fedora/rpms/yle-dl/yle-dl-20220704-2.fc37.src.rpm

> Requires:       /usr/bin/ffmpeg
> Will possibly be need to be changed to ffpmeg-free
>
> Recommends:     /usr/bin/wget
> Will possibly be need to be changed to wget 
> 
> However file/dir  dependency is valid:
> https://docs.fedoraproject.org/en-US/packaging-guidelines/
> #_file_and_directory_dependencies
> Will leave it to the official reviewer

Requiring ffmpeg-free would prevent using this package together with the full features ffmpeg from RPM Fusion.
Compatibility with RPM Fusion is the main reason for depending on the executable file;
both ffmpeg and ffmpeg-free provide that binary, so either can satisfy this Require.

Also, yle-dl uses ffmpeg by invoking the ffmpeg executable,
so in that sense the file dependency is the most correct way to express the dependency.

The case of wget is similar, except the RPM Fusion argument does not apply.
I put it as file dependency because the case is similar to ffmpeg,
and because of the 'most correct way' argument.
Changing to "Required: wget" would work here,
so I can change to that if requested.

> Licnese Unknown or generated
> --------------------
> yle-dl-20220704/COPYING
> yle-dl-20220704/MANIFEST.in
> yle-dl-20220704/README.fi
> yle-dl-20220704/setup.cfg
> yle-dl-20220704/tests/conftest.py
> yle-dl-20220704/tests/integration/test_areena_it.py
> yle-dl-20220704/tests/integration/test_areena_radio_it.py
> yle-dl-20220704/tests/integration/test_arkisto_it.py
> yle-dl-20220704/tests/integration/test_arkivet_it.py
> yle-dl-20220704/tests/integration/test_uutiset_it.py
> yle-dl-20220704/tests/test_backend.py
> yle-dl-20220704/tests/test_downloader.py
> yle-dl-20220704/tests/test_flavor_filters.py
> yle-dl-20220704/tests/test_timestamp.py
> yle-dl-20220704/tests/test_title_formatter.py
> yle-dl-20220704/tests/utils.py
> yle-dl-20220704/yle_dl.egg-info/SOURCES.txt
> yle-dl-20220704/yle_dl.egg-info/dependency_links.txt
> yle-dl-20220704/yle_dl.egg-info/entry_points.txt
> yle-dl-20220704/yle_dl.egg-info/requires.txt
> yle-dl-20220704/yle_dl.egg-info/top_level.txt
> yle-dl-20220704/yledl/__init__.py
> yle-dl-20220704/yledl/__main__.py
> yle-dl-20220704/yledl/backends.py
> yle-dl-20220704/yledl/downloader.py
> yle-dl-20220704/yledl/errors.py
> yle-dl-20220704/yledl/exitcodes.py
> yle-dl-20220704/yledl/extractors.py
> yle-dl-20220704/yledl/ffprobe.py
> yle-dl-20220704/yledl/geolocation.py
> yle-dl-20220704/yledl/http.py
> yle-dl-20220704/yledl/io.py
> yle-dl-20220704/yledl/kaltura.py
> yle-dl-20220704/yledl/localization.py
> yle-dl-20220704/yledl/streamfilters.py
> yle-dl-20220704/yledl/streamflavor.py
> yle-dl-20220704/yledl/streamprobe.py
> yle-dl-20220704/yledl/subtitles.py
> yle-dl-20220704/yledl/timestamp.py
> yle-dl-20220704/yledl/titleformatter.py
> yle-dl-20220704/yledl/utils.py
> yle-dl-20220704/yledl/version.py

I have suggested some license clarification upstream:

https://github.com/aajanki/yle-dl/pull/326

I did not touch the case of many source files no having a license header.
Apparently, upstream thinking is that they explain the project license in top level files such as README.md,
and the main source file yledl/yledl.py,
and the rest are considered to be implictly under the project license.
I will add a comment to my pull request,
asking upstream to consider adding license headers to every source file.

Comment 3 Otto Liljalaakso 2022-08-04 07:52:10 UTC
> I will add a comment to my pull request,
> asking upstream to consider adding license headers to every source file.

https://github.com/aajanki/yle-dl/pull/326#issuecomment-1204895431

Comment 4 Otto Liljalaakso 2022-08-06 06:42:38 UTC
(In reply to Otto Liljalaakso from comment #3)
> > I will add a comment to my pull request,
> > asking upstream to consider adding license headers to every source file.
> 
> https://github.com/aajanki/yle-dl/pull/326#issuecomment-1204895431

Upstream has added the headers:

https://github.com/aajanki/yle-dl/commit/402d3b27863aa80a67f5d71566eacc85ad9d7945

Comment 5 Ville-Pekka Vainio 2022-08-06 18:06:19 UTC
I could officially review this one, I use yle-dl myself. Salman has already done great work with the review!

Otto, what kind of functionality gets disabled if the user uses ffmpeg-free?

Comment 6 Otto Liljalaakso 2022-08-06 19:11:10 UTC
(In reply to Ville-Pekka Vainio from comment #5)
> I could officially review this one, I use yle-dl myself. Salman has already
> done great work with the review!

Great!

> Otto, what kind of functionality gets disabled if the user uses ffmpeg-free?

The only missing features in ffmpeg-free are patented codecs,
except for H.264, which is supported through Cisco's H.264
as is also done elsewhere in Fedora, in Firefox etc.
If Yle encodes some content using other patented codecs,
it may be that it cannot be downloaded.
I checked encoding of some files I downloaded from Yle Areena,
everything I checked was H.264.
So to my understanding, everything will work even with ffmpeg-free.
But, I only tested downloading a couple of videos from Yle Areena,
so testing has not been thorough.

Comment 7 Ville-Pekka Vainio 2022-08-08 19:14:39 UTC
What are your plans with regard to the licence headers?

The package is already looking good. I'd prefer if the wget Requires was just "wget". I think it might speed up dependency resolution a bit, but I'm not sure.
Other than that I'd be ready to accept the package.


Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated



===== MUST items =====

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: License field in the package spec file matches the actual license.
     Note: Maintainer has gotten upstream to add license headers to each
     source file.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[ ]: Requires correct, justified where necessary.
     Note: I'd prefer the wget Requires to be a package requirement,
     not a file.
[x]: Spec file is legible and written in American English.
[x]: Package is not known to require an ExcludeArch tag.
[x]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least
     one supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
[x]: If (and only if) the source package includes the text of the
     license(s) in its own file, then that file, containing the text of the
     license(s) for the package is included in %license.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Dist tag is present.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package must not depend on deprecated() packages.
[x]: Package use %makeinstall only when make install DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package does not use a name that already exists.
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as
     provided in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

Python:
[x]: Package meets the Packaging Guidelines::Python
[x]: Package contains BR: python2-devel or python3-devel
[x]: Packages MUST NOT have dependencies (either build-time or runtime) on
     packages named with the unversioned python- prefix unless no properly
     versioned package exists. Dependencies on Python packages instead MUST
     use names beginning with python2- or python3- as appropriate.
[x]: Python packages must not contain %{pythonX_site(lib|arch)}/* in %files

===== SHOULD items =====

Generic:
[x]: Final provides and requires are sane (see attachments).
[?]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[?]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed
     files.
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
[x]: Spec file according to URL is the same as in SRPM.


Source checksums
----------------
https://files.pythonhosted.org/packages/source/y/yle-dl/yle-dl-20220704.tar.gz :
  CHECKSUM(SHA256) this package     : b81af57852aa0004c9a0ffdc82e27189bcecee1ffaeb1ca90b6d3822c8a328e0
  CHECKSUM(SHA256) upstream package : b81af57852aa0004c9a0ffdc82e27189bcecee1ffaeb1ca90b6d3822c8a328e0


Requires
--------
yle-dl (rpmlib, GLIBC filtered):
    /usr/bin/ffmpeg
    /usr/bin/python3
    python(abi)
    python3.11dist(attrs)
    python3.11dist(configargparse)
    python3.11dist(lxml)
    python3.11dist(requests)



Provides
--------
yle-dl:
    python3.11dist(yle-dl)
    python3dist(yle-dl)
    yle-dl



Generated by fedora-review 0.8.0 (e988316) last change: 2022-04-07
Command line :/usr/bin/fedora-review -b 2115066
Buildroot used: fedora-rawhide-x86_64
Active plugins: Shell-api, Generic, Python
Disabled plugins: C/C++, PHP, Java, SugarActivity, R, Perl, fonts, Haskell, Ocaml
Disabled flags: EPEL6, EPEL7, DISTTAG, BATCH, EXARCH

Comment 8 Otto Liljalaakso 2022-08-09 08:48:05 UTC
(In reply to Ville-Pekka Vainio from comment #7)
> What are your plans with regard to the licence headers?

I was thinking of just waiting until there is a new release,
and let them trickle to Fedora on version bump.
In my opinion licensing is clear even without them,
and there is no rule in Fedora requiring them to be present,
so I do not see this as an urgent matter.
However, if you prefer, I can add upstream commit as a patch.
 
> The package is already looking good. I'd prefer if the wget Requires was
> just "wget". I think it might speed up dependency resolution a bit, but I'm
> not sure.

Changed.

> Other than that I'd be ready to accept the package.

Specfile: http://oturpe.kapsi.fi/fedora/rpms/yle-dl/20220704/3/yle-dl.spec
SRPM: http://oturpe.kapsi.fi/fedora/rpms/yle-dl/20220704/3/yle-dl-20220704-3.fc37.src.rpm

I am also planning to switch the package to rpmautospec once imported to dist-git.
FESCo recently  notified that they are planning to recommend it for all packages.
That notification has prompted me to try it again,
but I had already started with packaging yle-dl.
Switching in middle of review felt quite messy,
and also, apparently, dist-git import tooling also has a bug with --no-initial-commit,
which makes is difficult to directly import rpmautospec packages.

Comment 9 Ville-Pekka Vainio 2022-08-09 18:49:46 UTC
The package looks good to me, I've now approved it. If you want a co-maintainer, I'll volunteer, because I use the package anyway.

Salman, thanks again for your review! If you're looking for a sponsor, I hope you'll find one. I have never applied to be a sponsor, because I often don't have the time or energy to follow Fedora development closely. You clearly have a good understanding of Fedora packaging :) And thank you Otto for the packaging!

Comment 10 Otto Liljalaakso 2022-08-10 05:12:10 UTC
Thank you for the reviews!
I have requests the dist-git repository.

I will add you as a co-maintainer when the repo is available.
FESCo policy recommends that anyhow [1],
and this package in particular requires quick updates,
because changes on Yle side may break things.
Feel free to apply version bumps and such with direct commits.
Larger changes to packaging we can discuss through pull requests and Bugzilla issues.

[1]: https://docs.fedoraproject.org/en-US/fesco/Policy_for_encouraging_comaintainers_of_packages/

Comment 11 Gwyn Ciesla 2022-08-10 13:07:28 UTC
(fedscm-admin):  The Pagure repository was created at https://src.fedoraproject.org/rpms/yle-dl

Comment 12 Otto Liljalaakso 2022-08-11 06:31:48 UTC
* Repository created
* Built for Rawhide
* Branches for f35, f36, f37 requested
* Added to Koschei
* Added to release-monitoring.org

Release numbers are odd,
either rpmautospec is acting out, or I do not know how to use it
(it requires surprising amount of manual twiddling,
considering it has 'auto' in its name.)
My plan is to wait for new version,
things should look correct from there on.

Comment 13 Otto Liljalaakso 2022-09-24 18:09:21 UTC
Forgot to close the issue earlier.
Package is in the repositories, so I do it now.


Note You need to log in before you can comment on or make changes to this bug.