Bug 2132082 - pep8 job fails due to python2 dependency [NEEDINFO]
Summary: pep8 job fails due to python2 dependency
Keywords:
Status: ON_DEV
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-designate
Version: 17.0 (Wallaby)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Nate Johnston
QA Contact: Toni Freger
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-10-04 15:03 UTC by Szymon Datko
Modified: 2023-08-03 16:55 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:
ifrangs: needinfo? (njohnston)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1992825 0 None None None 2022-10-13 19:42:41 UTC
OpenStack gerrit 860964 0 None NEW Restrict virtualenv to avoid suds-jurko dep issue 2023-08-03 16:55:05 UTC
Red Hat Issue Tracker OSP-19305 0 None None None 2022-10-11 14:51:19 UTC

Description Szymon Datko 2022-10-04 15:03:48 UTC
Hello,

we observe the following issue when running pep8 job in Component CI for OSP 17 and 17.1

```
2022-09-24 13:17:50.503711 | container | Collecting suds-jurko>=0.6
2022-09-24 13:17:50.516858 | container |   Downloading suds-jurko-0.6.zip (255 kB)
2022-09-24 13:17:50.529055 | container |      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 255.8/255.8 KB 24.9 MB/s eta 0:00:00
2022-09-24 13:17:50.547561 | container |   Preparing metadata (setup.py): started
2022-09-24 13:17:51.357323 | container |   Preparing metadata (setup.py): finished with status 'error'
2022-09-24 13:17:51.362618 | container |   error: subprocess-exited-with-error
2022-09-24 13:17:51.362648 | container |
2022-09-24 13:17:51.362658 | container |   × python setup.py egg_info did not run successfully.
2022-09-24 13:17:51.362684 | container |   │ exit code: 1
2022-09-24 13:17:51.362694 | container |   ╰─> [1 lines of output]
2022-09-24 13:17:51.362700 | container |       error in suds-jurko setup command: use_2to3 is invalid.
2022-09-24 13:17:51.362708 | container |       [end of output]
2022-09-24 13:17:51.362715 | container |
2022-09-24 13:17:51.362724 | container |   note: This error originates from a subprocess, and is likely not a problem with pip.
2022-09-24 13:17:51.364608 | container | error: metadata-generation-failed
2022-09-24 13:17:51.364621 | container |
2022-09-24 13:17:51.364627 | container | × Encountered error while generating package metadata.
2022-09-24 13:17:51.364633 | container | ╰─> See above for output.
2022-09-24 13:17:51.364638 | container |
2022-09-24 13:17:51.364644 | container | note: This is an issue with the package mentioned above, not pip.
2022-09-24 13:17:51.364650 | container | hint: See above for details.
2022-09-24 13:17:51.368653 | container | WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
2022-09-24 13:17:51.368663 | container | You should consider upgrading via the '/root/src/code.engineering.redhat.com/openstack-designate/.tox/pep8/bin/python -m pip install --upgrade pip' command.
2022-09-24 13:17:51.538175 | container | ERROR: invocation failed (exit code 1)
2022-09-24 13:17:51.538287 | container | pep8 finish: developpkg /root/src/code.engineering.redhat.com/openstack-designate after 7.73 seconds
2022-09-24 13:17:51.538478 | container | ___________________________________ summary ____________________________________
2022-09-24 13:17:51.538695 | container | ERROR:   pep8: InvocationError for command /root/src/code.engineering.redhat.com/openstack-designate/.tox/pep8/bin/pip install --exists-action w -e . (exited with code 1)
2022-09-24 13:17:51.935631 | container | ERROR
2022-09-24 13:17:51.935807 | container | {
2022-09-24 13:17:51.935838 | container |   "delta": "0:00:40.784034",
2022-09-24 13:17:51.935861 | container |   "end": "2022-09-24 13:17:51.573975",
2022-09-24 13:17:51.935882 | container |   "msg": "non-zero return code",
2022-09-24 13:17:51.935901 | container |   "rc": 1,
2022-09-24 13:17:51.935920 | container |   "start": "2022-09-24 13:17:10.789941"
2022-09-24 13:17:51.935940 | container | }
```

The issue appears because the openstack-designate project specifies dependency that relies on suds-jurko package that is not maintained for almost a decade now: https://pypi.org/project/suds-jurko/#history

It was not an issue originally, however with recent versions of setuptools the 2to3 wrapper is gone and the Python package does not build properly (hence the error observed).

There was already an upstream attempt to resolve this:
– https://review.opendev.org/c/openstack/designate/+/813151
but in the end (for master) it was conducted differently
– https://review.opendev.org/c/openstack/designate/+/813380
and the dependent part of code is gone from the project.

There is no upstream fix for wallaby yet.

My proposal is to pin the virtualenv package that would restrict setuptools to right version that still supports the 2to3 wrapper, like was already done in the same case for other projects, e.g.
– https://code.engineering.redhat.com/gerrit/c/openstack-heat-agents/+/317658/2/tox.ini

Yours, Szymon

Comment 1 Michael Johnson 2022-10-04 15:39:30 UTC
Your test jobs must honor the upper-constraints.txt for Wallaby which provide the correct version of setuptools.

Comment 2 Szymon Datko 2022-10-04 15:45:57 UTC
It does; the problem is, tox is stupid and when creating virtualenv, it selects wrong version first and then when installing deps for project it uses the wrong version of setuptools.

This is due to how tox.ini for the given project is written.

Hence the proposed patch that was introduced in multiple projects, even upstream, eg. https://review.opendev.org/c/openstack/oslo.vmware/+/850479

Comment 3 Greg Rakauskas 2022-10-11 14:42:51 UTC
Hi,

The VANS squad looked at this BZ and has determined that this is a misconfigured test job.

Best,
--Greg R.

Comment 4 Szymon Datko 2022-10-11 22:01:14 UTC
There is the following upstream change proposal that will fix the issue: https://review.opendev.org/c/openstack/designate/+/860964

For now we proceed with a in-job workaround to make the job functional and voting: https://github.com/RedHatCRE/znoyder/pull/101
Tested here: https://code.engineering.redhat.com/gerrit/c/openstack-designate/+/430266

Comment 5 Nate Johnston 2022-10-13 19:42:42 UTC
Adding link to upstream bug.


Note You need to log in before you can comment on or make changes to this bug.