Bug 1902849 - osp13-osp16.1 ffu fails on clusters previously upgraded from osp8, osp10
Summary: osp13-osp16.1 ffu fails on clusters previously upgraded from osp8, osp10
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z16
: 13.0 (Queens)
Assignee: Lukas Bezdicka
QA Contact: Jason Grosso
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-11-30 19:35 UTC by Jeremy
Modified: 2022-08-02 13:58 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-08-16 11:32:16 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1917913 0 None None None 2021-03-05 16:45:33 UTC
OpenStack gerrit 778987 0 None NEW Ensure old installation match new images and get python-docker. 2021-03-24 12:07:13 UTC
OpenStack gerrit 780589 0 None NEW [Queens] Ensure undercloud contains necessary extra packages. 2021-03-24 12:07:17 UTC
Red Hat Issue Tracker OSP-1475 0 None None None 2022-08-02 13:58:53 UTC
Red Hat Issue Tracker UPG-2383 0 None None None 2021-08-16 11:33:19 UTC

Description Jeremy 2020-11-30 19:35:36 UTC
Description of problem:
on step 19.2 of the upgrade documentation -- > https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/framework_for_upgrades_13_to_16.1/upgrading-an-overcloud-with-external-ceph-deployments_upgrading-overcloud-external-ceph#running-the-overcloud-upgrade-preparation-upgrading-overcloud-external-ceph

The upgrade command which is failing is -->  openstack overcloud upgrade run --stack STACK NAME --tags system_upgrade --limit xxxx

Failure message :


2020-11-24 12:55:22,143 p=100846 u=mistral n=ansible | TASK [tripleo-podman : Stop a container] ***************************************
2020-11-24 12:55:22,143 p=100846 u=mistral n=ansible | Tuesday 24 November 2020  12:55:22 -0500 (0:00:00.326)       0:00:32.456 ******
2020-11-24 12:55:22,805 p=100846 u=mistral n=ansible | failed: [xxxxx] (item=985976509ce8) => {"ansible_loop_var": "item", "changed": false, "item": "985976509ce8", "msg": "Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6)) on xxxxx's Python /usr/bin/python. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter, for example via `pip install docker` or `pip install docker-py` (Python 2.6). The error was: No module named docker"}
2020-11-24 12:55:23,159 p=100846 u=mistral n=ansible | failed: [xxxx] (item=06ce45f22ce4) => {"ansible_loop_var": "item", "changed": false, "item": "06ce45f22ce4", "msg": "Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6)) on xxxxx's Python /usr/bin/python. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter, for example via `pip install docker` or `pip install docker-py` (Python 2.6). The error was: No module named docker"}


Version-Release number of selected component (if applicable):
osp13

How reproducible:
unknown

Steps to Reproduce:
1. attempt ffu on systems that were upgraded from osp10 timeframe to osp13
2.
3.

Actual results:
update fails

Expected results:
update works.

Additional info:

I found that the package python-docker contains the module needed it seems. This rpm is in the osp13 image, however the rpm isn't on osp10 systems. So this problem occurs when older environments are upgraded. 


[root@undercloud-0 docker]# grep -ir "import docker" /usr/lib/python2.7/site-packages/docker
/usr/lib/python2.7/site-packages/docker/__init__.py:from .client import DockerClient, from_env
/usr/lib/python2.7/site-packages/docker/api/client.py:        >>> import docker
...
...
[root@undercloud-0 docker]# rpm -q --whatprovides /usr/lib/python2.7/site-packages/docker/__init__.py
python-docker-2.4.2-2.el7.noarch

Comment 2 Sofer Athlan-Guyot 2020-12-07 14:16:24 UTC
Hi,

one way is to make sure we get it installed during osp13 update. Let's add the necessary snippet there.

Comment 4 Sofer Athlan-Guyot 2021-03-05 16:45:34 UTC
Hi Jeremy,

sorry for the long delay. Here is an upstream patch, I'll just need to do sanity check on it, but according to your description that should do the trick.

Comment 6 Sofer Athlan-Guyot 2021-03-15 13:57:09 UTC
Adding it to the undercloud as well.

Comment 10 Vlada Grosu 2021-06-18 11:55:42 UTC
I've added a note to the Framework for Upgrades (13 to 16.1) guide to point to the KB solution to avoid this BZ:

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/framework_for_upgrades_13_to_16.1/index?lb_target=production#known-issues-that-might-block-an-upgrade

Thank you

Comment 13 Lukas Bezdicka 2021-08-16 11:32:16 UTC
The code side of fix would have to go into OSP13 and it still does not guarantee that users will actually update to this version. It's better to document the issue and just make sure users with so long living environments will apply the workaround.


Note You need to log in before you can comment on or make changes to this bug.