We observe issues with running pep8 jobs for Python-ProliantUtils as part of Component CI for OSP 17.0 and OSP 17.1. Error log: ``` 2022-08-27 15:31:59.085424 | container | pep8 installed: attrs==22.1.0,certifi==2022.6.15,charset-normalizer==2.1.1,debtcollector==2.5.0,fasteners==0.17.3,flake8==2.5.5,hacking==0.12.0,idna==3.3,iso8601==1.0.2,jsonschema==4.7.2,mccabe==0.2.1,msgpack==1.0.4,netaddr==0.8.0,netifaces==0.11.0,oslo.concurrency==5.0.0,oslo.config==9.0.0,oslo.i18n==5.1.0,oslo.serialization==5.0.0,oslo.utils==6.0.1,packaging==21.3,pbr==5.10.0,pep8==1.5.7,ply==3.11,-e git+file:///dev/null@5bc756917d396cef185054c8aa635a8af3759826#egg=proliantutils,pyasn1==0.4.8,pycryptodomex==3.15.0,pyflakes==0.8.1,pyparsing==3.0.9,pyrsistent==0.18.1,pysmi==0.3.4,pysnmp==4.4.12,python-dateutil==2.8.2,pytz==2022.2.1,PyYAML==6.0,requests==2.28.1,retrying==1.3.3,rfc3986==1.5.0,six==1.16.0,stevedore==4.0.0,sushy==4.3.0,urllib3==1.26.11,wrapt==1.14.1 2022-08-27 15:31:59.085872 | container | pep8 start: run-test-pre 2022-08-27 15:31:59.085990 | container | pep8 run-test-pre: PYTHONHASHSEED='1682216202' 2022-08-27 15:31:59.086028 | container | pep8 finish: run-test-pre after 0.00 seconds 2022-08-27 15:31:59.086132 | container | pep8 start: run-test 2022-08-27 15:31:59.086153 | container | pep8 run-test: commands[0] | flake8 proliantutils 2022-08-27 15:31:59.086231 | container | setting PATH=/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/bin:/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2022-08-27 15:31:59.088339 | container | [407] /root/src/code.engineering.redhat.com/python-proliantutils$ /root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/bin/flake8 proliantutils 2022-08-27 15:31:59.123560 | container | /root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pep8.py:102: FutureWarning: Possible nested set at position 1 2022-08-27 15:31:59.123606 | container | EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]') 2022-08-27 15:31:59.596131 | container | Traceback (most recent call last): 2022-08-27 15:31:59.596170 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/bin/flake8", line 8, in <module> 2022-08-27 15:31:59.596240 | container | sys.exit(main()) 2022-08-27 15:31:59.596249 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/flake8/main.py", line 36, in main 2022-08-27 15:31:59.596313 | container | report = flake8_style.check_files() 2022-08-27 15:31:59.596321 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/flake8/engine.py", line 181, in check_files 2022-08-27 15:31:59.596381 | container | return self._retry_serial(self._styleguide.check_files, paths=paths) 2022-08-27 15:31:59.596390 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/flake8/engine.py", line 172, in _retry_serial 2022-08-27 15:31:59.596432 | container | return func(*args, **kwargs) 2022-08-27 15:31:59.596449 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pep8.py", line 1670, in check_files 2022-08-27 15:31:59.596709 | container | self.input_dir(path) 2022-08-27 15:31:59.596721 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pep8.py", line 1706, in input_dir 2022-08-27 15:31:59.596983 | container | runner(os.path.join(root, filename)) 2022-08-27 15:31:59.596996 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/flake8/engine.py", line 126, in input_file 2022-08-27 15:31:59.597048 | container | return fchecker.check_all(expected=expected, line_offset=line_offset) 2022-08-27 15:31:59.597061 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pep8.py", line 1412, in check_all 2022-08-27 15:31:59.597287 | container | self.check_ast() 2022-08-27 15:31:59.597295 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pep8.py", line 1358, in check_ast 2022-08-27 15:31:59.597496 | container | checker = cls(tree, self.filename) 2022-08-27 15:31:59.597506 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/flake8/_pyflakes.py", line 61, in __init__ 2022-08-27 15:31:59.597551 | container | super(FlakesChecker, self).__init__(tree, filename, 2022-08-27 15:31:59.597559 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pyflakes/checker.py", line 294, in __init__ 2022-08-27 15:31:59.597620 | container | self.handleChildren(tree) 2022-08-27 15:31:59.597630 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pyflakes/checker.py", line 547, in handleChildren 2022-08-27 15:31:59.597722 | container | self.handleNode(node, tree) 2022-08-27 15:31:59.597730 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pyflakes/checker.py", line 589, in handleNode 2022-08-27 15:31:59.597816 | container | handler(node) 2022-08-27 15:31:59.597823 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pyflakes/checker.py", line 547, in handleChildren 2022-08-27 15:31:59.597910 | container | self.handleNode(node, tree) 2022-08-27 15:31:59.597917 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pyflakes/checker.py", line 588, in handleNode 2022-08-27 15:31:59.598014 | container | handler = self.getNodeHandler(node.__class__) 2022-08-27 15:31:59.598034 | container | File "/root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/lib/python3.9/site-packages/pyflakes/checker.py", line 458, in getNodeHandler 2022-08-27 15:31:59.598137 | container | self._nodeHandlers[node_class] = handler = getattr(self, nodeType) 2022-08-27 15:31:59.598145 | container | AttributeError: 'FlakesChecker' object has no attribute 'CONSTANT' 2022-08-27 15:31:59.654064 | container | ERROR: InvocationError for command /root/src/code.engineering.redhat.com/python-proliantutils/.tox/pep8/bin/flake8 proliantutils (exited with code 1) 2022-08-27 15:31:59.654109 | container | pep8 finish: run-test after 0.56 seconds 2022-08-27 15:31:59.654117 | container | pep8 start: run-test-post 2022-08-27 15:31:59.654124 | container | pep8 finish: run-test-post after 0.00 seconds 2022-08-27 15:31:59.654130 | container | ___________________________________ summary ____________________________________ 2022-08-27 15:31:59.654136 | container | ERROR: pep8: commands failed 2022-08-27 15:32:00.182502 | container | ERROR 2022-08-27 15:32:00.182757 | container | { 2022-08-27 15:32:00.182811 | container | "delta": "0:00:02.584868", 2022-08-27 15:32:00.182852 | container | "end": "2022-08-27 15:31:59.687051", 2022-08-27 15:32:00.182898 | container | "msg": "non-zero return code", 2022-08-27 15:32:00.182923 | container | "rc": 1, 2022-08-27 15:32:00.182945 | container | "start": "2022-08-27 15:31:57.102183" 2022-08-27 15:32:00.182965 | container | } ``` Details: https://sf.hosted.upshift.rdu2.redhat.com/logs/weekly/code.engineering.redhat.com/python-proliantutils/rhos-17.0-trunk-patches/osp-tox-pep8/8eb8b6d/ From what I found, the problem is this project depends on too old hacking library, which forces installing 5-year old deps (flake8==2.5.5,hacking==0.12.0) that do not support Python 3.9 apparently (https://code.engineering.redhat.com/gerrit/plugins/gitiles/python-proliantutils/+/refs/heads/rhos-17.0-trunk-patches/tox.ini#19). In upstream the problematic lines do not exist: https://opendev.org/x/proliantutils/src/branch/stable/wallaby/tox.ini Apparently the code was fixed in the last couple of years by commits such as: https://opendev.org/x/proliantutils/commit/64540ba31614e3108ea90188c01e2e7158438b74 Can we update the code base to make the tests working? Note the same problem applies for OSP 17.1 branch.
It seems we're going to need to work with the upstream maintainers of proliantutils in order to correct the python dependency issues.
There is an additional problem for proliantutils on stable/xena, there is a dependency conflict for sushy==4.3.0 vs sushy<4: (stable/xena) proliantutils]$ tox -epy39 py39 create: /home/steveb/dev/localstack/proliantutils/.tox/py39 py39 installdeps: -chttps://releases.openstack.org/constraints/upper/master, -r/home/steveb/dev/localstack/proliantutils/requirements.txt, -r/home/steveb/dev/localstack/proliantutils/test-requirements.txt ERROR: invocation failed (exit code 1), logfile: /home/steveb/dev/localstack/proliantutils/.tox/py39/log/py39-1.log ================================================================================================================== log start ================================================================================================================== Collecting pbr>=2.0.0 Using cached pbr-5.10.0-py2.py3-none-any.whl (112 kB) Collecting six>=1.9.0 Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting oslo.concurrency>=3.8.0 Downloading oslo.concurrency-5.0.1-py3-none-any.whl (48 kB) Collecting oslo.serialization>=1.10.0 Using cached oslo.serialization-5.0.0-py3-none-any.whl (25 kB) Collecting oslo.utils>=3.20.0 Using cached oslo.utils-6.0.1-py3-none-any.whl (100 kB) Collecting jsonschema>=2.6.0 Using cached jsonschema-4.14.0-py3-none-any.whl (82 kB) Collecting requests!=2.12.2,!=2.13.0,>=2.10.0 Using cached requests-2.28.1-py3-none-any.whl (62 kB) Collecting retrying!=1.3.0,>=1.2.3 Using cached retrying-1.3.3-py3-none-any.whl Collecting pysnmp<5.0.0,>=4.2.3 Using cached pysnmp-4.4.12-py2.py3-none-any.whl (296 kB) ERROR: Cannot install sushy<4.0.0 and >=3.11.0 because these package versions have conflicting dependencies. The conflict is caused by: The user requested sushy<4.0.0 and >=3.11.0 The user requested (constraint) sushy===4.3.0 To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies WARNING: You are using pip version 21.3.1; however, version 22.2.2 is available. You should consider upgrading via the '/home/steveb/dev/localstack/proliantutils/.tox/py39/bin/python -m pip install --upgrade pip' command. =================================================================================================================== log end =================================================================================================================== ERROR: could not install deps [-chttps://releases.openstack.org/constraints/upper/master, -r/home/steveb/dev/localstack/proliantutils/requirements.txt, -r/home/steveb/dev/localstack/proliantutils/test-requirements.txt]; v = InvocationError('/home/steveb/dev/localstack/proliantutils/.tox/py39/bin/python -m pip install -chttps://releases.openstack.org/constraints/upper/master -r/home/steveb/dev/localstack/proliantutils/requirements.txt -r/home/steveb/dev/localstack/proliantutils/test-requirements.txt', 1) ___________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________ ERROR: py39: could not install deps [-chttps://releases.openstack.org/constraints/upper/master, -r/home/steveb/dev/localstack/proliantutils/requirements.txt, -r/home/steveb/dev/localstack/proliantutils/test-requirements.txt]; v = InvocationError('/home/steveb/dev/localstack/proliantutils/.tox/py39/bin/python -m pip install -chttps://releases.openstack.org/constraints/upper/master -r/home/steveb/dev/localstack/proliantutils/requirements.txt -r/home/steveb/dev/localstack/proliantutils/test-requirements.txt', 1)