Bug 1781891

Summary: python-pip-shims: FTBFS in Fedora 32 (rawhide)
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: python-pip-shimsAssignee: Patrik Kopkan <pkopkan>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: medium    
Version: rawhideCC: pkopkan, pviktori
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: https://koschei.fedoraproject.org/package/python-pip-shims
Whiteboard:
Fixed In Version: 0.3.2-4.fc32 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-02-07 12:39:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1750908, 1748018, 1785415    

Description Miro Hrončok 2019-12-10 20:28:33 UTC
Description of problem:
Package python-pip-shims fails to build from source in Fedora rawhide.

Version-Release number of selected component (if applicable):
0.3.2-1.fc32

Steps to Reproduce:
koji build --scratch f32 python-pip-shims-0.3.2-1.fc32.src.rpm


+ PYTHONPATH=/builddir/build/BUILDROOT/python-pip-shims-0.3.2-1.fc32.noarch/usr/lib/python3.8/site-packages
+ /usr/bin/python3 -m pytest -k 'not test_resolution and not test_wheelbuilder'
============================= test session starts ==============================
platform linux -- Python 3.8.0, pytest-4.6.6, py-1.8.0, pluggy-0.12.0
rootdir: /builddir/build/BUILD/pip-shims-0.3.2, inifile: setup.cfg, testpaths: tests/
collected 33 items / 2 deselected / 31 selected
tests/test_instances.py ........................F...F..                  [100%]
=================================== FAILURES ===================================
______________________________ test_abstract_dist ______________________________
    def test_abstract_dist():
        ireq = InstallRequirement.from_editable(
            "git+https://github.com/requests/requests.git@2.19.1#egg=requests"
        )
>       abs_dist = make_abstract_dist(ireq)
E       TypeError: 'NoneType' object is not callable
tests/test_instances.py:297: TypeError
_______________________________ test_vcs_support _______________________________
    def test_vcs_support():
>       vcs = VcsSupport()
E       TypeError: 'NoneType' object is not callable
tests/test_instances.py:320: TypeError
=========================== short test summary info ============================
FAILED tests/test_instances.py::test_abstract_dist - TypeError: 'NoneType' ob...
FAILED tests/test_instances.py::test_vcs_support - TypeError: 'NoneType' obje...
============== 2 failed, 29 passed, 2 deselected in 0.44 seconds ===============

Additional info:
This package is tracked by Koschei. See:
https://koschei.fedoraproject.org/package/python-pip-shims
(In fact, I have only added it there when i realize it doesn't build, the information on updated packages is not helpful (yet).)

This blocks the rebuild of python-pip-shims to gain proper Python bytecode, as described in bz1748018.

Comment 1 Patrik Kopkan 2019-12-11 11:52:48 UTC
updating package helps partially it fails on different tests, but tox -e py38-pip1923 runs fine so I 'll probably need to update some dependency. And check in copr if doesn't break pipenv that will take a while.


mockbuild:

+ PYTHONPATH=/builddir/build/BUILDROOT/python-pip-shims-0.4.0-2.fc32.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/python3 -m pytest -k 'not test_resolution and not test_wheelbuilder'
============================= test session starts ==============================
platform linux -- Python 3.8.0, pytest-4.6.6, py-1.8.0, pluggy-0.13.0
rootdir: /builddir/build/BUILD/pip-shims-0.4.0, inifile: setup.cfg, testpaths: tests/
collected 37 items / 2 deselected / 35 selected                                

tests/test_instances.py ..............................FF...              [100%]

=================================== FAILURES ===================================
____________________________ test_get_packagefinder ____________________________

    def test_get_packagefinder():
        install_cmd = InstallCommand()
        finder = get_package_finder(
            install_cmd, python_versions=("27", "35", "36", "37", "38"), implementation="cp"
        )
        ireq = InstallRequirement.from_line("requests>=2.18")
        if install_req_from_line:
            ireq2 = install_req_from_line("requests>=2.18")
            assert str(ireq) == str(ireq2)
        requests_candidates = finder.find_all_candidates(ireq.name)
        candidates = sorted(
            [
                c
                for c in requests_candidates
                if c.version
                in ireq.specifier.filter(
                    (candidate.version for candidate in requests_candidates)
                )
            ],
            key=lambda c: c.version,
        )
>       best_version = candidates[-1]
E       IndexError: list index out of range

tests/test_instances.py:562: IndexError
------------------------------ Captured log call -------------------------------
WARNING  pip._vendor.urllib3.connectionpool:connectionpool.py:664 Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f0c77df3c40>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/requests/
WARNING  pip._vendor.urllib3.connectionpool:connectionpool.py:664 Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f0c77e8f9d0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/requests/
WARNING  pip._vendor.urllib3.connectionpool:connectionpool.py:664 Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f0c77e697f0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/requests/
WARNING  pip._vendor.urllib3.connectionpool:connectionpool.py:664 Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f0c77e697c0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/requests/
WARNING  pip._vendor.urllib3.connectionpool:connectionpool.py:664 Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f0c77e694f0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/requests/
_________________________________ test_resolve _________________________________

    def test_resolve():
        install_cmd = InstallCommand()
        ireq = InstallRequirement.from_line("requests>=2.18")
>       result = resolve(ireq, install_command=install_cmd)

tests/test_instances.py:570: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../BUILDROOT/python-pip-shims-0.4.0-2.fc32.x86_64/usr/lib/python3.8/site-packages/pip_shims/backports.py:1180: in resolve
    resolver.resolve(reqset)  # type: ignore
/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py:196: in resolve
    self._resolve_one(requirement_set, req)
/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py:361: in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py:307: in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(
/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py:134: in prepare_linked_requirement
    req.populate_link(finder, upgrade_allowed, require_hashes)
/usr/lib/python3.8/site-packages/pip/_internal/req/req_install.py:211: in populate_link
    self.link = finder.find_requirement(self, upgrade)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pip._internal.index.PackageFinder object at 0x7f0c77d72a30>
req = <InstallRequirement object: requests>=2.18 editable=False>
upgrade = False

    def find_requirement(self, req, upgrade):
        # type: (InstallRequirement, bool) -> Optional[Link]
        """Try to find a Link matching req
    
        Expects req, an InstallRequirement and upgrade, a boolean
        Returns a Link if found,
        Raises DistributionNotFound or BestVersionAlreadyInstalled otherwise
        """
        hashes = req.hashes(trust_internet=False)
        candidates = self.find_candidates(
            req.name, specifier=req.specifier, hashes=hashes,
        )
        best_candidate = candidates.get_best()
    
        installed_version = None    # type: Optional[_BaseVersion]
        if req.satisfied_by is not None:
            installed_version = parse_version(req.satisfied_by.version)
    
        def _format_versions(cand_iter):
            # This repeated parse_version and str() conversion is needed to
            # handle different vendoring sources from pip and pkg_resources.
            # If we stop using the pkg_resources provided specifier and start
            # using our own, we can drop the cast to str().
            return ", ".join(sorted(
                {str(c.version) for c in cand_iter},
                key=parse_version,
            )) or "none"
    
        if installed_version is None and best_candidate is None:
            logger.critical(
                'Could not find a version that satisfies the requirement %s '
                '(from versions: %s)',
                req,
                _format_versions(candidates.iter_all()),
            )
    
>           raise DistributionNotFound(
                'No matching distribution found for %s' % req
            )
E           pip._internal.exceptions.DistributionNotFound: No matching distribution found for requests>=2.18

/usr/lib/python3.8/site-packages/pip/_internal/index.py:1227: DistributionNotFound


I think that test_resolve should be skipped.

Comment 2 Patrik Kopkan 2019-12-19 11:55:27 UTC
got it working https://copr.fedorainfracloud.org/coprs/pkopkan/python-vistir/build/1129958/ but pipenv fails to build https://copr-be.cloud.fedoraproject.org/results/pkopkan/python-vistir/fedora-rawhide-x86_64/01129965-pipenv/build.log.gz

between versions of pip-shims function 'do_import' got removed and I didn't find any function similar to it.

Comment 3 Miro Hrončok 2019-12-19 12:02:36 UTC
Could you try to fix the package without updating it?

Comment 4 Patrik Kopkan 2019-12-19 12:14:12 UTC
I can think of either skipping tests or test the library against the bundled pipenv pip.

Comment 5 Miro Hrončok 2020-01-20 12:51:55 UTC
This now blocks the Python 3.9 rebuild.

Comment 6 Petr Viktorin (pviktori) 2020-01-29 11:32:18 UTC
Patrik, can you do this in any reasonable time? Or should someone else take this?

Comment 7 Patrik Kopkan 2020-02-03 15:46:14 UTC
(In reply to Petr Viktorin from comment #6)
> Patrik, can you do this in any reasonable time? Or should someone else take
> this?

If reasonable timeframe is two weeks then probably no.

Comment 8 Patrik Kopkan 2020-02-07 12:39:04 UTC
Upstream will adapt api of newer pip-shim's version. So the most effective/lazy sollution is to skip broken tests until that happen.

Build: https://koji.fedoraproject.org/koji/taskinfo?taskID=41422607