RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1834529 - python-s3transfer: bad dependency metadata due to bundling futures, jmespath, botocore
Summary: python-s3transfer: bad dependency metadata due to bundling futures, jmespath,...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: python-s3transfer
Version: 7.8
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Oyvind Albrigtsen
QA Contact: Brandon Perkins
URL:
Whiteboard:
Depends On:
Blocks: 1834528
TreeView+ depends on / blocked
 
Reported: 2020-05-11 21:00 UTC by Felix Schwarz
Modified: 2021-11-11 07:27 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-11 07:27:10 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Felix Schwarz 2020-05-11 21:00:48 UTC 
python-s3transfer 0.1.13-1.el7.0.1 in RHEL 7 contains bad Python-level metadata. It requires Python modules which are bundled.

Why is this a problem?

I can not run the test suite for "python-certbot-dns-route53" in EPEL 7. The certbot plugin depends on python-boto3. When I do "python setup.py test" Python tries to download all missing packages (which fails in koji for obvious reasons). I'm not yet sure if this also a runtime problem.


What should be done?

Remove all bundled packages from the Python-level requirements.


Details:

# cat /usr/lib/python2.7/site-packages/s3transfer-0.1.13-py2.7.egg-info/requires.txt
botocore>=1.3.0,<2.0.0
futures>=2.2.0,<4.0.0

[:python_version=="2.6" or python_version=="2.7"]
futures>=2.2.0,<4.0.0

The package bundles futures, botocore so these entries should be removed as Python-level dependencies (in setup.cfg).

The bundled botocore needs to be fixed (jmespath):

$ cat /usr/lib/fence-agents/bundled/botocore-1.8.35-py2.7.egg-info/requires.txt
jmespath>=0.7.1,<1.0.0
python-dateutil>=1.4,<3.0.0
docutils>=0.10

[:python_version=="2.6"]
ordereddict==1.1
simplejson==3.3.0
Comment 2 Felix Schwarz 2020-05-12 06:58:06 UTC 
Btw: also the "futures" requirement looks fishy to me. It should be necessary only for Python 2 but it is listed twice. This is no problem for the present rpm which only supports Python 2 but should be fixed if you want/need to support Python 3 at some point.
Comment 3 Brad Warren 2020-08-12 17:06:13 UTC 
This problem breaks the python2-certbot-dns-route53 package.

Certbot uses setuptools to find and load plugins such as python2-certbot-dns-route53 and setuptools verifies that all of the plugin's dependencies are satisified before loading the plugin. As a result, if you install python2-certbot-dns-route53 and run almost any Certbot command such as `certbot plugins`, Certbot will crash with output like:

An unexpected error occurred:
DistributionNotFound: futures>=2.2.0,<4.0.0
Comment 4 Felix Schwarz 2020-08-13 08:30:46 UTC 
ugh, that's worse than I expected (though logical in hindsight).

Oyvind, Brandon: Can I help you fixing this somehow?
Comment 5 Oyvind Albrigtsen 2020-08-21 14:11:54 UTC 
Where do you get that version from?
$ brew buildinfo python-s3transfer-0.1.13-1.el7.0.1                                                                                                         [11:14:12AM]
No such build: python-s3transfer-0.1.13-1.el7.0.1

This library is only supported for use with the resource-agents and fence-agents-* packages, but if you can make a patch to fix the requires.txt files post-build for the package we can fix it that way.
Comment 6 Felix Schwarz 2020-08-21 14:31:02 UTC 
The package is shipped in CentOS 7 in its "base" repo.

# yum info python-s3transfer
Name        : python-s3transfer
Arch        : noarch
Version     : 0.1.13
Release     : 1.el7.0.1
Size        : 2.9 M
Repo        : base/7/x86_64
Summary     : An Amazon S3 Transfer Manager
URL         : https://github.com/boto/s3transfer
License     : ASL 2.0
Description : S3transfer is a Python library for managing Amazon S3 transfers.


> if you can make a patch to fix the requires.txt files post-build for the package we can fix it that way.

Of course I can try to whip up a patch. Just so I don't go in the wrong direction: The easiest way would be to amend "bundled-futures-botocore-jmespath.patch" (https://git.centos.org/rpms/python-s3transfer/blob/c7/f/SOURCES/bundled-futures-botocore-jmespath.patch) so it removes the dependencies directly in setup.py *before* building the actual package. Would that work for you?
Comment 7 Oyvind Albrigtsen 2020-08-21 14:54:31 UTC 
I think it'd be better to do it in a separate patch post-build, so the build will still fail properly if we at some point upgrade any of the libs to a version that is incompatible with any of the other libs.
Comment 8 Jay Boyd 2020-09-13 14:57:53 UTC 
Is there a workaround?  4 months later it's still an issue
Comment 12 RHEL Program Management 2021-11-11 07:27:10 UTC 
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
Note You need to log in before you can comment on or make changes to this bug.