Bug 1676914 - Review Request: oraculum - Backend and API for Fedora QA Dashboard
Summary: Review Request: oraculum - Backend and API for Fedora QA Dashboard
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-13 15:19 UTC by Lukas Brabec
Modified: 2019-02-15 13:39 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-02-15 13:39:02 UTC
Type: Bug
Embargoed:
mhroncok: fedora-review+


Attachments (Terms of Use)

Comment 1 Miro Hrončok 2019-02-13 15:32:14 UTC
First things:

 - %{?python_enable_dependency_generator} + manual requires - is the generator not working, or?

 - %{__python3} setup.py build/install - use %py3_build and %py3_install please

 - cp in %install - please add -p to preserve timestamps

 - $RPM_BUILD_ROOT -> %{buildroot}

 - %{python3_sitelib}/oraculum and %{python3_sitelib}/*.egg-info -> add trailing slashes to make sure package FTBFS if it is no longer a directory

 - %{_httpd_modconfdir}/oraculum.conf - config noreplace? also, who owns the directory?

Comment 2 František Zatloukal 2019-02-13 15:54:16 UTC
(In reply to Miro Hrončok from comment #1)
>  - %{?python_enable_dependency_generator} + manual requires - is the
> generator not working, or?

It seems it does not work:

This is rpm build with requires:
$ rpm -qpR "build/0.0.1-3.fc29/oraculum-0.0.1-3.fc29.noarch.rpm"
/usr/bin/python3
config(oraculum) = 0.0.1-3.fc29
python(abi) = 3.7
python3-alembic
python3-bugzilla
python3-dateutil
python3-flask
python3-flask-cache
python3-flask-cors
python3-flask-login
python3-flask-sqlalchemy
python3-flask-wtf
python3-icalendar
python3-lxml
python3-mod_wsgi
python3-pygments
python3-pytz
python3-requests
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PartialHardlinkSets) <= 4.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1

And this one only with python_enable_dependency_generator:
$ rpm -qpR "build/0.0.1-3.fc29/oraculum-0.0.1-3.fc29.noarch.rpm"
/usr/bin/python3
config(oraculum) = 0.0.1-3.fc29
python(abi) = 3.7
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PartialHardlinkSets) <= 4.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1

Am I missing something or should I report it?

Comment 3 Miro Hrončok 2019-02-13 16:03:21 UTC
No upstream dependency information for the generator to work: https://pagure.io/fedora-qa/oraculum/blob/master/f/setup.py

I guess you are the upstream, so please fix that :)

Comment 4 František Zatloukal 2019-02-13 20:45:54 UTC
Okay... my bad, I was assuming it's going to look into requirements.txt :)

However, there is an issue with Flask-Caching ( https://pypi.org/project/Flask-Caching/ ) - package in Fedora is named differently - python-flask-cache . I'll create PR against that to at least provide Flask-Caching. I think we cannot use dependency generator until it's solved.

Everything else should be addressed apart from %{_httpd_modconfdir}/oraculum.conf - config noreplace? also, who owns the directory? ... we'll discuss if we actually want noreplace... and directory should be owned by httpd package.

SPEC: https://pagure.io/fedora-qa/oraculum/blob/c15db6289e12bf62e32fc80128e98bd8ba019916/f/oraculum.spec

Comment 5 Miro Hrončok 2019-02-14 00:09:45 UTC
https://pypi.org/project/Flask-Cache/ and https://pypi.org/project/Flask-Caching/ are 2 different packages.

The first is imported as flask_cache, the second as flask_caching.

python-flask-cache is Flask-Cache, if you want to depend on Flask-Caching, you need to package that one first.

Your code has:

  # This mess is required, because flask_cache is 'obsolete' and not working
  #  in PiP, and is replaced with flask_caching, but in Fedora, the flask_caching
  #  implementation is just used in flask_cache package instead....
  try:
      from flask_caching import Cache
  except ModuleNotFoundError:
      from flask_cache import Cache

That comment is a lie. python-flask-cache in Fedora is Flask-Cache, sources point to https://pypi.python.org/packages/source/F/Flask-Cache/Flask-Cache-0.13.1.tar.gz and there are no patches that change it Flask-Caching.

----------

> we'll discuss if we actually want noreplace.

Either you put it in /etc as %config(noreplace), or you put it in /usr without %config.

Files in /etc are meant to be edited by users (roots) and you shall not override user changes.

Files in /usr are not meant to be edited by users and you are OK overriding such changes.

%config without noreplace would need a thorough justification:

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_configuration_files


> and directory should be owned by httpd package.

In that case, you'll need to require httpd.
If you want to avoid hard requirement on httpd but still keep the file, you need to co-own the directory.

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_file_and_directory_ownership

Comment 6 Josef Skladanka 2019-02-14 09:34:00 UTC
(In reply to Miro Hrončok from comment #5)
 
> That comment is a lie. python-flask-cache in Fedora is Flask-Cache, sources
> point to
> https://pypi.python.org/packages/source/F/Flask-Cache/Flask-Cache-0.13.1.tar.
> gz and there are no patches that change it Flask-Caching.

I would not go as far as "lie" (and I'm choosing not to take this as an attack, but rather an error caused by English not being your first language, and probably some hasty reading of the comment itself), but you are right that the comment is probably incomplete/not specific enough.

Nowhere in the comment, I say the python-flask-cache package in fedora uses flask-caching sources, I'm rather pointing out, that instead of obsoleting flask-cache, and packaging flask-caching, the Fedora-package uses the `from flask_cache import` implementation present in flask-caching, instead of the `from flask.ext.cache import` line present in the git repo/tar.gz release of flask-cache.
The fact is, that in order for the code to work, when installing deps from PIP, flask-caching needs to be used, and when used in "fedora environment" flask-cache needs to be used (because flask-caching is not packaged, and flask-cache is basically hot-fixed to work with the new-style flask imports by sed in specfile).

If it will make you happier, I can, of course, change the comment to encompass all the details, but it does not change the facts of the matter at hand.

Comment 7 Miro Hrončok 2019-02-14 10:07:26 UTC
Mea culpa. It was no attack. I simply meant "is not true". Reminded me "the cake is a lie" and that made me use that line as a silly reference. Sorry about that.

"but in Fedora, the flask_caching implementation is just used in flask_cache package instead"

This is the part that I read as "Fedora's flask_cache package has the code (implementation) from flask_caching upstream" reading it as "the Fedora package makes obscure things, pretending to be what it is not". If I read that wrong because an error caused by English not being my first language, Ḯm sorry twice. if this is plausible implementation, please change the comment to say what it actually should mean: "We are compatible with both, but one is broken on PyPI and the second is not available in Fedora."

But that is your upstream, I hold no power over that and do whatever you prefer to do.

As for the current problem we are facing here, I'd suggest the following (try in order):

 1) get flask-cache fixed upstream if you can and only use flask_cache everywhere
 2) package the working flask-caching into Fedora (eventually even retiring flask-cache if the upstream is dead)
 3) add setup(install_requires=['flask-caching', ...]) in upstream, sed/patch it downstream to flask-cache to accommodate for this unfortunate situation
 4) don't add upstream dependency metadata at all, use manual requires (status quo) but also explicitly disable dependency generator with a comment

What not to do:

 Don't try to provide flask-caching from flask-cache ("I'll create PR against that to at least provide Flask-Caching."). That would be a real lie.

Comment 8 František Zatloukal 2019-02-14 14:21:04 UTC
All should be addressed, hope I didn't miss anything.

I have added sed workaround for Flask-Caching, I'll package the new module to Fedora and then remove that workaround.

SPEC: https://pagure.io/fedora-qa/oraculum/blob/c55bc7065842b54da2824971621d1103143ffa6e/f/oraculum.spec

(In reply to Miro Hrončok from comment #7)
>  Don't try to provide flask-caching from flask-cache ("I'll create PR
> against that to at least provide Flask-Caching."). That would be a real lie.

I won't, I promise :D

Comment 9 Miro Hrončok 2019-02-14 14:26:55 UTC
Are you changing the release tarball as we speak? Was there an actual release?

Comment 10 František Zatloukal 2019-02-14 15:29:23 UTC
Yeah, I've just tagged the release, but it seems there is something wrong with pagure when I try to download the released tar: https://pagure.io/fedora-qa/oraculum/archive/0.0.1/oraculum-0.0.1.tar.gz

Updated SPEC: https://pagure.io/fedora-qa/oraculum/blob/89daf307421405a4ef5d3e8e87e00614b56d7624/f/oraculum.spec

Comment 11 Miro Hrončok 2019-02-14 15:41:47 UTC
Could you please give me a prescribed form of spec + srpm links so I can run Fedora-Review?

Comment 12 Miro Hrončok 2019-02-14 15:47:57 UTC
That would be:

Spec URL: <spec URL here, direct file, no HTML preview>
SRPM URL: <srpm URL here, direct file, no HTML preview>

Thanks.

Comment 14 Miro Hrončok 2019-02-14 16:34:42 UTC
Thanks. I'll run it now.

----

The httpd requirement / dir ownership still remains unsolved.

----

Also, would you mind checking this one thing please? If you run oraculum as root, is %{_sysconfdir}/oraculum/__pycache__/settings.*.pyc created or not?

Comment 15 Miro Hrončok 2019-02-14 16:36:24 UTC
qadevel.cloud.fedoraproject.org has bad HTTPS, I cannot download the Source0.

Comment 16 František Zatloukal 2019-02-14 16:41:08 UTC
(In reply to Miro Hrončok from comment #14)
> The httpd requirement / dir ownership still remains unsolved.

Actually, httpd gets installed now, because it's dependency of "python3.7dist(mod-wsgi)". Is that enough?


(In reply to Miro Hrončok from comment #14)
> Also, would you mind checking this one thing please? If you run oraculum as
> root, is %{_sysconfdir}/oraculum/__pycache__/settings.*.pyc created or not?

It is not there.

Thanks!

Comment 17 František Zatloukal 2019-02-14 16:50:34 UTC
(In reply to Miro Hrončok from comment #15)
> qadevel.cloud.fedoraproject.org has bad HTTPS, I cannot download the Source0.

Really sorry about that, forgot to build it with one commit that fixed the Source0 url. Anyway, pagure seems to be broken when one tries to download any tar.gz of any tag... happens even with other projects.

Spec URL: https://copr-be.cloud.fedoraproject.org/results/frantisekz/taskotron-stack-python3/fedora-29-x86_64/00858624-oraculum/oraculum.spec
SRPM URL: https://copr-be.cloud.fedoraproject.org/results/frantisekz/taskotron-stack-python3/fedora-29-x86_64/00858624-oraculum/oraculum-0.0.1-3.fc29.src.rpm

Comment 18 Miro Hrončok 2019-02-14 17:10:20 UTC
(In reply to František Zatloukal from comment #16)
> (In reply to Miro Hrončok from comment #14)
> > The httpd requirement / dir ownership still remains unsolved.
> 
> Actually, httpd gets installed now, because it's dependency of
> "python3.7dist(mod-wsgi)". Is that enough?

Yes.

(In reply to František Zatloukal from comment #17)
> Anyway, pagure seems to be broken when one tries to download
> any tar.gz of any tag... happens even with other projects.

Have you reported that?

Comment 19 Miro Hrončok 2019-02-14 17:28:15 UTC
Checking: oraculum-0.0.1-3.fc30.noarch.rpm
          oraculum-0.0.1-3.fc30.src.rpm
oraculum.noarch: W: spelling-error Summary(en_US) Backend -> Backed, Back end, Back-end
oraculum.noarch: W: spelling-error %description -l en_US Backend -> Backed, Back end, Back-end
oraculum.noarch: W: no-manual-page-for-binary oraculum
oraculum.src: W: spelling-error Summary(en_US) Backend -> Backed, Back end, Back-end
oraculum.src: W: spelling-error %description -l en_US Backend -> Backed, Back end, Back-end
oraculum.src: W: invalid-url Source0: https://pagure.io/fedora-qa/oraculum/archive/0.0.1/oraculum-0.0.1.tar.gz HTTP Error 400: BAD REQUEST
2 packages and 0 specfiles checked; 0 errors, 6 warnings.


     Note: Directories without known owners: /etc/httpd, /etc/httpd/conf.d (should be OK if httpd is always pulled)

I cannot do source checksums now, but otherwise this will get APPROVED.

Comment 20 Miro Hrončok 2019-02-14 17:28:55 UTC
(if this is time sensitive, let me know and I'll approve without the checksums).

Comment 21 František Zatloukal 2019-02-14 17:46:56 UTC
(In reply to Miro Hrončok from comment #20)
> (if this is time sensitive, let me know and I'll approve without the
> checksums).

No, nothing critical, but thanks!

Spec URL: https://copr-be.cloud.fedoraproject.org/results/frantisekz/taskotron-stack-python3/fedora-29-x86_64/00858645-oraculum/oraculum.spec
SRPM URL: https://copr-be.cloud.fedoraproject.org/results/frantisekz/taskotron-stack-python3/fedora-29-x86_64/00858645-oraculum/oraculum-0.0.1-3.fc29.src.rpm


And yeah, I've reported that on #fedora-noc, created two bugs for infra and I am going to create one more for upstream pagure :)

Comment 22 Miro Hrončok 2019-02-14 18:07:56 UTC
Source checksums
----------------
https://releases.pagure.org/fedora-qa/oraculum/oraculum-0.0.1.tar.gz :
  CHECKSUM(SHA256) this package     : 0e6b58bfc554c62bfe9b1d3b9afbec2eb49cc4318ec2fc3ee9e816b363667bfd
  CHECKSUM(SHA256) upstream package : 3e85acebc70be353360fbf149af06c8c79d5a93f1ff80ac6af7e3810587bc151
diff -r also reports differences

Comment 24 Miro Hrončok 2019-02-15 09:23:55 UTC
Source checksums
----------------
https://releases.pagure.org/fedora-qa/oraculum/oraculum-0.0.2.tar.gz :
  CHECKSUM(SHA256) this package     : c0244fe490f99122726158362bb41cf347cc29b9796b6692067c73c298b1e077
  CHECKSUM(SHA256) upstream package : c0244fe490f99122726158362bb41cf347cc29b9796b6692067c73c298b1e077


Package APPROVED.

Thanks everybody and sorry again for the undesired language.

Comment 25 Igor Raits 2019-02-15 11:55:17 UTC
(fedscm-admin):  The Pagure repository was created at https://src.fedoraproject.org/rpms/oraculum

Comment 26 František Zatloukal 2019-02-15 13:39:02 UTC
(In reply to Miro Hrončok from comment #24)
> Source checksums
> ----------------
> https://releases.pagure.org/fedora-qa/oraculum/oraculum-0.0.2.tar.gz :
>   CHECKSUM(SHA256) this package     :
> c0244fe490f99122726158362bb41cf347cc29b9796b6692067c73c298b1e077
>   CHECKSUM(SHA256) upstream package :
> c0244fe490f99122726158362bb41cf347cc29b9796b6692067c73c298b1e077
> 
> 
> Package APPROVED.
> 
> Thanks everybody and sorry again for the undesired language.

Thanks!!!


Note You need to log in before you can comment on or make changes to this bug.