Bug 1727936 - [RFE] Convert python-nitrate to python3
Summary: [RFE] Convert python-nitrate to python3
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: python-nitrate
Version: 31
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Martin Zelený
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1558007 1738912 1745466 1756959 1768526
TreeView+ depends on / blocked
 
Reported: 2019-07-08 15:20 UTC by Martin Zelený
Modified: 2021-01-06 09:10 UTC (History)
9 users (show)

Fixed In Version: python-nitrate-1.5-4.fc31
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-24 20:21:01 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Martin Zelený 2019-07-08 15:20:25 UTC
Work on upstream python-nitrate project [1] to make the python3 compatible code.

[1] https://github.com/psss/python-nitrate

Comment 3 Lumír Balhar 2019-08-13 19:42:43 UTC
Let me know if you need any help with porting the stack to Python 3. Today, I've installed 1minute tip on Fedora 30 and from the list of new packages in my system, it's obvious that we need to move this forward.

Comment 6 Lumír Balhar 2019-09-25 13:19:29 UTC
The process to test python-nitrate in a virtual environment seems to work so I can help you with porting. Note that I needed to install libpq-devel which provides pg_config utility. I'm going to start from your branch where a lot is done.

Comment 8 Lumír Balhar 2019-09-26 04:34:03 UTC
By the way, the reason we cannot just alter PYTHONPATH and use nitrate module without installation is that the package folder name is `source` instead of `nitrate` and it's renamed only during the installation. The same reason prevents us from doing `pip install --editable`.

It would be good to rename the folder.

Comment 9 Lumír Balhar 2019-10-01 09:36:35 UTC
I've finished the initial phase of porting and the basic test described in comment#5 is working with both Python 2 and 3.

Pull request is here: https://github.com/ZelenyMartin/python-nitrate/pull/2

What next should I test and port?

Comment 10 Lumír Balhar 2019-10-01 09:43:36 UTC
By the way, the package has been retired: https://bugzilla.redhat.com/show_bug.cgi?id=1754904

Comment 11 Martin Zelený 2019-10-01 15:26:20 UTC
Merged Lumír's work into my fork. Obtaining TCMS permissions for him in progress. Current issues in tests (Python3):

  File "/home/mzeleny/.virtualenvs/python-nitrate/lib/python3.7/site-packages/nitrate/mutable.py", line 1445, in _fetch
    testplan, testcase = _idify(self.id)
ValueError: too many values to unpack (expected 2)

and

  File "/home/mzeleny/.virtualenvs/python-nitrate/lib/python3.7/site-packages/nitrate/containers.py", line 782, in __iter__
    self._items, key=lambda x: self._object.sortkey(x)):
TypeError: '<' not supported between instances of 'NoneType' and 'int'

and

  File "/home/mzeleny/.virtualenvs/python-nitrate/lib/python3.7/site-packages/nitrate/base.py", line 371, in __eq__
    self.__class__.__name__, other.__class__.__name__))
nitrate.xmlrpc_driver.NitrateError: Cannot compare 'User' with 'str'

Work in progress...

Comment 12 Lumír Balhar 2019-10-10 12:55:50 UTC
With the latest PR I've just open (https://github.com/ZelenyMartin/python-nitrate/pull/3), I have only these errors:

Traceback (most recent call last):
  File "source/tests.py", line 1387, in test_bugging3
    self.assertTrue(self.bug not in testcase.bugs)
AssertionError: False is not true

Traceback (most recent call last):
  File "source/tests.py", line 1217, in test_sortkey_update
    caseplan.sortkey = sortkey
  File "/home/lbalhar/.virtualenvs/nitrate/lib/python3.7/site-packages/nitrate/base.py", line 83, in setter
    self._fetch()
  File "/home/lbalhar/.virtualenvs/nitrate/lib/python3.7/site-packages/nitrate/mutable.py", line 1445, in _fetch
    testplan, testcase = _idify(self.id)
ValueError: too many values to unpack (expected 2)

Traceback (most recent call last):
  File "source/tests.py", line 1768, in test_add_remove
    self.assertTrue(testcase in testrun.testcases)
AssertionError: False is not true

The latest one happens with Python 2 as well. I cannot track the problem because the structure of classes is very complicated.

Let me know if you need any more help.

Comment 13 Lumír Balhar 2019-10-30 05:07:53 UTC
Any news here? We should finish it asap to bring this package back to Fedora.

Comment 14 Martin Zelený 2019-10-30 16:34:57 UTC
Solved one issue by commit [1]
Current state: one assert fails in test

  File "source/tests.py", line 1768, in test_add_remove
    self.assertTrue(testcase in testrun.testcases)
AssertionError: False is not true

[1] https://github.com/ZelenyMartin/python-nitrate/commit/3a603b9fff48827c93358463b7ab212346e2b5e8

Comment 15 Ben Cotton 2019-10-31 18:43:01 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 16 Martin Zelený 2019-11-01 08:58:09 UTC
Current state is that test suite is without error and is able to run both under python3 and python2. I made PR to original repo [1] and I'm waiting for Petr Šplichal to check it and possibly merge it.

[1] https://github.com/psss/python-nitrate/pull/10

Comment 17 Karel Volný 2019-11-04 09:45:23 UTC
(In reply to Martin Zelený from comment #16)
> Current state is that test suite is without error and is able to run both
> under python3 and python2. I made PR to original repo ...

well ... while at it, could you please also update the specfile and do a scratchbuild, pretty please with sugar on top? :-)

Comment 18 Martin Zelený 2019-11-04 16:32:16 UTC
This is my first attempt for scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=38760304

Comment 19 Lukáš Zachar 2019-11-04 20:58:07 UTC
Hm, I found an issue :/

$ cat reproducer.py
import nitrate
r = nitrate.TestRun(122250)
for a in r:
   print(a)

On python2 ... prints
On python3 ...
Traceback (most recent call last):
  File "x.py", line 3, in <module>
    for a in r: 
  File "/usr/local/lib/python3.7/site-packages/nitrate-1.4.1-py3.7.egg/nitrate/mutable.py", line 528, in __iter__
  File "/usr/local/lib/python3.7/site-packages/nitrate-1.4.1-py3.7.egg/nitrate/containers.py", line 946, in __iter__
TypeError: '<' not supported between instances of 'NoneType' and 'int'


I've installed from Martin's github by `sudo python3 setup.py install`


Can you reproduce, @Martin?

Comment 20 Lukáš Zachar 2019-11-04 21:12:16 UTC
I guess it's from missing (None) sortkey in few caseruns as shown by 
r = nitrate.TestRun(122250)
for i in r.caseruns._items:
    print(i.sortkey)

ipdb3 on reproducer.py shows  this frame:
> /usr/local/lib/python3.7/site-packages/nitrate-1.4.1-py3.7.egg/nitrate/containers.py(946)__iter__()
    945         """ Iterate over all included case runs ordered by sortkey """
--> 946         for caserun in sorted(self._items, key=lambda x: x.sortkey):
    947             yield caserun

Comment 21 Lukáš Zachar 2019-11-04 21:16:35 UTC
For the record there is change in behaviour:


$ python2 -c 'print(None < 1)'
True

$ python3 -c 'print(None < 1)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: '<' not supported between instances of 'NoneType' and 'int'

Comment 22 Lumír Balhar 2019-11-05 05:13:33 UTC
One of these issues has been already solved in: https://github.com/ZelenyMartin/python-nitrate/commit/480ee7d1e498bcf6b863d1a80bd580d08da3dc7a

It's just necessary to apply the same fix for more classes.

Comment 23 Karel Volný 2019-11-05 07:55:39 UTC
(In reply to Martin Zelený from comment #18)
> This is my first attempt for scratch build:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=38760304

thanks, but I prefer F31 ;-)
https://koji.fedoraproject.org/koji/taskinfo?taskID=38773438

it installs, and I can at least import the qe module ... however, any real usage ends in tracebacks :-/

Comment 24 Martin Zelený 2019-11-05 13:49:26 UTC
I made another fix with Lumir's help. Check the new scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=38777416

Comment 25 Lukáš Zachar 2019-11-05 16:18:43 UTC
Thanks, I've tried https://github.com/ZelenyMartin/python-nitrate/commit/50794665300014056ddb8f9919c628c46bacf871 and it works.
Good job!

Comment 26 Petr Šplíchal 2019-11-14 23:53:50 UTC
Fedora 31 update is ready for testing:
https://bodhi.fedoraproject.org/updates/FEDORA-2019-ef25129f44

Comment 27 Petr Šplíchal 2019-11-15 07:58:46 UTC
There's a possible problem with the update: Currently users cannot
install both python2-nitrate and python3-nitrate because of the
conflicting script and man pages:

    Error: Transaction check error:
      file /usr/bin/nitrate from install of python3-nitrate-1.5-2.fc31.noarch conflicts with file from package python2-nitrate-1.4-3.fc31.noarch
      file /usr/share/man/man1/python-nitrate.1.gz from install of python3-nitrate-1.5-2.fc31.noarch conflicts with file from package python2-nitrate-1.4-3.fc31.noarch

There are several solutions for this:

1) Keep it as it is (don't support both versions)
2) Drop nitrate script and man pages from python2-nitrate
3) Create a new subpackage 'nitrate' and move conflicting files there

What do you think would be the best way?

Comment 28 Lukáš Zachar 2019-11-15 08:04:03 UTC
IMO Point 3) - move conflicts to nitrate

Comment 29 Lumír Balhar 2019-11-15 10:34:33 UTC
Take a look on Python RPM Porting guide: https://python-rpm-porting.readthedocs.io/en/latest/#what-type-of-software-are-you-packaging

If the application does the same for Python 2 and 3, it should be included only in python3- subpackage, and python2- subpackage will still provide only importable module.

I guess that this is the case of nitrate.

Comment 30 Petr Šplíchal 2019-11-21 10:37:50 UTC
Thanks for the feedback. I've updated the spec file accordingly:
https://github.com/psss/python-nitrate/commit/cd00d31
Going to submit updates for f30, f31 and f32.

Comment 31 Fedora Update System 2019-11-21 10:48:10 UTC
FEDORA-2019-5a454922a5 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-5a454922a5

Comment 32 Petr Šplíchal 2019-11-21 10:51:53 UTC
Fresh rpms are also available in copr:
https://copr.fedorainfracloud.org/coprs/psss/python-nitrate/

Comment 33 Petr Šplíchal 2019-11-21 10:53:15 UTC
Hm, bug reload button changed the state to NEW. Returning back.

Comment 34 Fedora Update System 2019-11-21 15:12:19 UTC
FEDORA-2019-894e80ab6e has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-894e80ab6e

Comment 35 Fedora Update System 2019-11-22 02:31:43 UTC
python-nitrate-1.5-4.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-894e80ab6e

Comment 36 Fedora Update System 2019-11-30 00:55:57 UTC
python-nitrate-1.5-4.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Comment 37 Ales Zelinka 2020-04-16 13:17:47 UTC
I'm getting tracebacks in the cache module:

> Error in atexit._run_exitfuncs:
> Traceback (most recent call last):
>   File "/usr/lib/python3.7/site-packages/nitrate/cache.py", line 400, in exit
>     self.save()
>   File "/usr/lib/python3.7/site-packages/nitrate/cache.py", line 258, in save
>     self.expire()
>   File "/usr/lib/python3.7/site-packages/nitrate/cache.py", line 444, in expire
>     for id, current_object in current_class._cache.iteritems():
> AttributeError: 'dict' object has no attribute 'iteritems'


2to3 also reports deprecated usage of itervalues in the same module.


> $ rpm -qf /usr/lib/python3.7/site-packages/nitrate/cache.py
> python3-nitrate-1.5-4.fc31.noarch

Comment 38 Petr Šplíchal 2020-10-01 09:47:40 UTC
Should be fixed in https://github.com/psss/python-nitrate/commit/1872217

Comment 39 Ben Cotton 2020-11-03 15:19:40 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 40 Ben Cotton 2020-11-24 20:21:01 UTC
Fedora 31 changed to end-of-life (EOL) status on 2020-11-24. Fedora 31 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.