Bug 1494640

Summary: Ansible 2.3.2 conflict with python27-jmespath
Product: [Fedora] Fedora EPEL Reporter: jr.shank
Component: ansibleAssignee: Kevin Fenzi <kevin>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: el6CC: a.badger, adam.lewandowski, athmanem, carl, cgkades, jr.shank, kevin, maxim, orion, pcahyna, tibbs
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-03-29 14:46:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description jr.shank 2017-09-22 17:39:38 UTC
Description of problem:


Version-Release number of selected component (if applicable): 2.3.2


How reproducible: always


Steps to Reproduce:
1. Install python27-jmespath via YUM
2. Install ansible via YUM

Actual results: YUM reports "Processing Conflict: python27-jmespath-0.9.2-1.12.amzn1.noarch conflicts python-jmespath <= 0.5.0-1.1.amzn1"

Expected results: Ansible installs.

Additional info:

I encountered this on Amazon Linux 2017.03. Installing Ansible from EPEL worked fine with 2.3.1, but the 2.3.2 package appears to require a specific version of the python-jmespath which conflicts with python27-jmespath.

Ansible installed via PIP appears to work fine with python27-jmespath installed from EPEL.

Comment 1 jr.shank 2017-09-22 17:41:36 UTC
Lowering the priority since this can be worked around by installing Ansible via PIP.

Comment 2 Carl George 2017-09-22 18:10:44 UTC
python27-jmespath isn't from EPEL, it's from the Amazon Linux repo.  Amazon Linux isn't 100% compatible with RHEL or EPEL.  That said, I was able to get ansible installed in a clean docker container.

http://pastebin.centos.org/276426/

So I think if you just uninstall python27-jmespath it will work for you.

Comment 3 jr.shank 2017-09-22 18:17:44 UTC
Indeed uninstalling python27-jmespath allows ansible to be installed. Unfortunately it results in packages that rely on python27-jmespath to be uninstalled as well. One such package is aws-cli.

I understand that EPEL and the Amazon Linux repo are not 100% compatible. Is there anything that can be done with the ansible 2.3.2 package dependencies to restore compatibility in this case? There was no such incompatibility in the ansible 2.3.1 package, but that was apparently removed from the EPEL repository recently.

Comment 4 Carl George 2017-09-22 18:28:53 UTC
Amazon's attempt at "compatibility" is to have their python26-* packages provides the stock names.

    # repoquery --provides python26-jmespath
    python-jmespath = 0.9.2-1.12.amzn1
    python26-jmespath = 0.9.2-1.12.amzn1

That still works for stock RHEL packages, but many EPEL packages are getting renamed to python2-$name (providing python-$name) to match Fedora policy.  Thus, other EPEL packages are getting updated to require the python2-$name, as you noticed with ansible.  There are even some stock RHEL packages that are beginning to provide the python2-$name (see python-devel and python-setuptools)

I think it's fair to say that the goal is to make Amazon Linux compatible with RHEL and EPEL, not the other way around.  I would recommend asking Amazon to update their python26-jmespath package to provide python2-jmespath.

Alternatively (additionally?), if the ansible maintainer (not me) wishes, they could make this change to the ansible spec file:

    -Requires: python2-jmespath
    +Requires: python%{?fedora:2}-jmespath

Comment 5 Kevin Fenzi 2017-09-22 21:49:45 UTC
It sounds to me like amazon linux is pretty divergent here... 

We could do the requires Carl mentions there I suppose, but not sure how worth it it will be... as probibly ansible is going to drop out of epel6 before long anyhow (upstream has switched to needing python-cryptography for vault, and thats not available in rhel6).

Comment 6 Toshio Kuratomi 2017-09-22 22:22:07 UTC
For vault,  note that unless someone has broken this, we still fallback to pycrypto (precisely for running on rhel6).  I don't know if anyone's looked at whether python-cryptography could be built into epel6 either?

Comment 7 jr.shank 2017-09-22 22:57:33 UTC
I've created a request with AWS Support to update their python26-jmespath package to provide python2-jmespath. I'll update this bug when I get a response from them.

It would be greatly appreciated if the ansible maintainer could tweak the spec file as described by Carl.

Comment 8 Kevin Fenzi 2017-09-24 19:42:01 UTC
(In reply to Toshio Kuratomi from comment #6)
> For vault,  note that unless someone has broken this, we still fallback to
> pycrypto (precisely for running on rhel6).  I don't know if anyone's looked
> at whether python-cryptography could be built into epel6 either?

Huh, well, vault fails here with a:

Traceback (most recent call last):
  File "/usr/bin/ansible-vault", line 85, in <module>
    mycli = getattr(__import__("ansible.cli.%s" % sub, fromlist=[myclass]), myclass)
  File "/usr/lib/python2.6/site-packages/ansible/cli/__init__.py", line 45, in <module>
    from ansible.vars.manager import VariableManager
  File "/usr/lib/python2.6/site-packages/ansible/vars/manager.py", line 32, in <module>
    from jinja2.exceptions import UndefinedError
ImportError: No module named jinja2.exceptions

Can sort that out with you out of bug when you have time... 

(In reply to jr.shank from comment #7)
> I've created a request with AWS Support to update their python26-jmespath
> package to provide python2-jmespath. I'll update this bug when I get a
> response from them.
> 
> It would be greatly appreciated if the ansible maintainer could tweak the
> spec file as described by Carl.

I can, as soon as we sort out the vault issue...

Comment 9 Brett Yoakum 2017-09-25 23:47:20 UTC
I was able to work around the issue using the RPMs provided by Ansible. The python-jmespath package is not in the spec file that they are using, so I'm not sure why the one in EPEL has that requirement. The root issue is, I believe, the unneeded requirement for jmespath. Although the AWS package will always conflict since their 2.7 package has a file provided by the 2.6 package (jp.py)

http://releases.ansible.com/ansible/rpm/release/epel-6-x86_64/

Comment 10 Kevin Fenzi 2017-09-26 00:01:52 UTC
I added that in the last update. See: https://bugzilla.redhat.com/show_bug.cgi?id=1465586 basically its needed for the json_query filter. 

It's difficult for me to see how best to work around this without access to the amazon repos (are they public anywhere?) I was working on the assumption that their python27-jmespath didn't provide python-jmespath, but I am not sure thats the case, can someone attach: 

repoquery --whatprovides python-jmespath --qf '%{name}\n'

from one of the affected amazon linux instances?

Comment 11 Brett Yoakum 2017-09-26 00:12:40 UTC
I see that in the change log, however it is not needed for Ansible to function as there is a try catch block protecting it. It can safely be removed without breaking Ansible itself, and the error message it provides will instruct the user to install the correct package.

...
try:
    import jmespath
    HAS_LIB = True
except ImportError:
    HAS_LIB = False
...
if not HAS_LIB:
        raise AnsibleError('You need to install "jmespath" prior to running '
                           'json_query filter')

I'll work on providing you the info you are looking for. But essentially their python27 package and the one provided by EPEL conflict on /usr/bin/jp.py

Comment 12 Brett Yoakum 2017-09-26 00:16:09 UTC
On an AWS instance:

Without EPEL enabled 
$ repoquery --whatprovides python-jmespath --qf '%{name}\n'
python26-jmespath

python26-jmespath


With EPEL enabled
repoquery --whatprovides python-jmespath --qf '%{name}\n'
python26-jmespath

python26-jmespath

python2-jmespath

Comment 13 Kevin Fenzi 2017-09-26 18:16:41 UTC
Hum, so I am not sure how the suggestion in comment#4 is going to help here. 

The problem is the conflict of the python27-jmespath with the python-jmespath. (Both amazon packages). If you need python27-jmespath then you cannot have the other one(s) installed or they conflict. 

For now, I will just remove the jmespath requirement on epel6.

Comment 14 Carl George 2017-09-26 20:14:45 UTC
Kevin, there are Amazon Linux images on docker.io that I've found useful to explore issues like this.

    docker run -it --rm docker.io/amazonlinux:2017.03

Comment 15 Fedora Update System 2017-09-27 17:50:50 UTC
ansible-2.4.0.0-2.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-c64e477a67

Comment 16 Fedora Update System 2017-09-29 00:20:39 UTC
ansible-2.4.0.0-2.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-c64e477a67

Comment 17 Orion Poplawski 2019-03-29 14:46:35 UTC
This appears to have been resolved.