Bug 1808674 - ansible fails to build with Python 3.9: ansible-test units: error: argument --python: invalid choice: '3.9'
Summary: ansible fails to build with Python 3.9: ansible-test units: error: argument -...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: ansible
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Charalampos Stratakis
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1841619 (view as bug list)
Depends On:
Blocks: PYTHON39 F33FTBFS
TreeView+ depends on / blocked
 
Reported: 2020-02-29 10:42 UTC by Miro Hrončok
Modified: 2020-05-30 07:03 UTC (History)
7 users (show)

Fixed In Version: ansible-2.9.9-3.fc33
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-30 07:03:32 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2020-02-29 10:42:17 UTC
ansible fails to build with Python 3.9.0a4.

+ make PYTHON=/usr/bin/python3 tests-py3
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
bin/ansible-test units -v --python 3.9 
usage: ansible-test units [-h] [-e] [-v] [--color [COLOR]] [--debug]
                          [--truncate COLUMNS] [--redact] [--no-redact]
                          [--include TARGET] [--exclude TARGET]
                          [--require TARGET] [--coverage]
                          [--coverage-label COVERAGE_LABEL] [--coverage-check]
                          [--changed] [--untracked] [--ignore-committed]
                          [--ignore-staged] [--ignore-unstaged]
                          [--requirements] [--python-interpreter PATH]
                          [--local] [--venv] [--tox] [--tox-sitepackages]
                          [--docker [IMAGE]] [--remote PLATFORM]
                          [--remote-stage STAGE] [--remote-provider PROVIDER]
                          [--remote-aws-region REGION]
                          [--remote-terminate WHEN] [--python VERSION]
                          [--collect-only] [--num-workers NUM_WORKERS]
                          [--docker-no-pull] [--docker-keep-git]
                          [--docker-seccomp SC]
                          [TARGET ...]
ansible-test units: error: argument --python: invalid choice: '3.9' (choose from '2.6', '2.7', '3.5', '3.6', '3.7', '3.8', 'default')
make: *** [Makefile:150: tests-py3] Error 2

Seems like ansible test suite hardcodes a list of Python version :(


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01260187-ansible/

For all our attempts to build ansible with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/ansible/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-02-29 10:46:05 UTC
I guess this is the problem:

https://github.com/ansible/ansible/blob/ce0fc4005c7f95d78a719c35faa3975b66ff5272/test/lib/ansible_test/_internal/util.py#L114

SUPPORTED_PYTHON_VERSIONS = (
    '2.6',
    '2.7',
    '3.5',
    '3.6',
    '3.7',
    '3.8',
)

Comment 2 Toshio Kuratomi 2020-03-02 18:38:57 UTC
From upstream:

 <@mattclay> abadger1999: ansible-test in devel now has support for Python 3.9 -- however, the requirements haven't been updated yet for pylint, so it won't work with our container or requirements files. It may work with newer versions, but I haven't tested that yet.

Looks like there were several PRs to get it to work.  I found these three which look relevant:

* https://github.com/ansible/ansible/pull/67891
* https://github.com/ansible/ansible/pull/67907
* https://github.com/ansible/ansible/pull/67908

Comment 3 Miro Hrončok 2020-05-25 13:01:54 UTC
This comment is mass posted to all bugs blocking the Python 3.9 tracker, sorry if it is not 100 % relevant. When in doubt, please ask.


The Python 3.9 rebuild is in progress in a Koji side tag.

If you fix this bug, please don't rebuild the package in regular rawhide, but do it in the side tag with:

    $ fedpkg build --target=f33-python

The rebuild is progressing slowly and it is possible this package won't have all the required build dependencies yet. If that's the case, please just leave the fix committed and pushed and we will eventually rebuild it for you.

You are not asked to go and try rebuild all the missing dependencies yourself. If you know there is a bootstrap loop in the dependencies, let me know and we can untangle it together.

If you want to test your fix or reproduce the failure, you can still use the Copr repo mentioned in the initial comment of this bug: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Comment 4 Miro Hrončok 2020-05-29 07:11:48 UTC
Python 3.9 update: The f33-python side tag is currently being merged.

New builds in f33-python are no longer possible, but python3 is not yet updated to Python 3.9 in rawhide. You can check when Python is Python 3.9 with:

    $ koji wait-repo f33-build --build python3.9-3.9.0~b1-3.fc3

And build the packages normally after that.

Comment 5 Miro Hrončok 2020-05-29 16:59:02 UTC
*** Bug 1841619 has been marked as a duplicate of this bug. ***

Comment 6 Miro Hrončok 2020-05-29 16:59:55 UTC
I'm rising the severity here, because without ansible, the Fedora CI is broken.

Comment 8 Kevin Fenzi 2020-05-29 19:29:57 UTC
I am in the middle of a datacenter move, so I will likely not have cycles for this for a while. 

Hopefully Toshio or one of my other co-maintainers can look into it.

Comment 9 Charalampos Stratakis 2020-05-29 23:30:05 UTC
Will provide a fixed build.

There are some more failures with pytest 5 though, so I'll pin the dependency to version 4.

Comment 10 Igor Raits 2020-05-30 06:30:35 UTC
Hello,

Please note that this comment was generated automatically. If you feel that this output has mistakes, please contact me via email (ignatenkobrain).

All subpackages of a package agaisnt which this bug was filled are now installable or removed from Fedora 33.

Thanks for taking care of it!


Note You need to log in before you can comment on or make changes to this bug.