Bug 1691879 - Director deployed OCP 3.11 deployment fails while running /usr/bin/tripleo-deploy-openshift
Summary: Director deployed OCP 3.11 deployment fails while running /usr/bin/tripleo-de...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 15.0 (Stein)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: beta
: 15.0 (Stein)
Assignee: Martin André
QA Contact: Ofer Blaut
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-22 17:48 UTC by Marius Cornea
Modified: 2019-09-04 15:12 UTC (History)
5 users (show)

Fixed In Version: openstack-tripleo-common-10.7.1-0.20190606230405.3ff454a.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-29 16:32:22 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1821437 0 None None None 2019-03-23 07:34:45 UTC
OpenStack gerrit 645979 0 None None None 2019-03-23 07:34:45 UTC

Description Marius Cornea 2019-03-22 17:48:50 UTC
Description of problem:

Director deployed OCP 3.11 deployment fails while running /usr/bin/tripleo-deploy-openshift:

 [root@undercloud-0 stack]# cat /var/lib/mistral/openshift/openshift/playbook.log
--config-download-dir is deprecated, use --plan instead
Trying to pull 192.168.24.1:8787/openshift3/ose-ansible:v3.11...Getting image source signatures
Copying blob 2cb1196a3b27: 72.31 MiB / 72.31 MiB  5s
Copying blob c9c433594a59: 1.21 KiB / 1.21 KiB  5s
Copying blob b9bf6fa9627f: 128.46 MiB / 128.46 MiB  5s
Copying config 0498430e0cc8: 5.57 KiB / 5.57 KiB  0s
Writing manifest to image destination
Storing signatures
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftInfra_groups.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftInfra_hosts.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftInfra_openshift_glusterfs.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftMaster_groups.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftMaster_hosts.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftMaster_openshift_master.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftWorker_groups.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftWorker_hosts.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/OpenShiftWorker_openshift_glusterfs.yml': Permission denied
cp: cannot stat '/var/lib/mistral/openshift/openshift/inventory/groups.yml': Permission denied

 [root@undercloud-0 stack]# ls -lah /var/lib/mistral/openshift/openshift/inventory/
total 44K
drwxr-xr-x. 2 tripleo-admin root          4.0K Mar 22 13:31 .
drwxr-xr-x. 3 tripleo-admin root           117 Mar 22 13:31 ..
-rw-rw-r--. 1 tripleo-admin tripleo-admin  266 Mar 22 13:31 groups.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin  141 Mar 22 13:31 OpenShiftInfra_groups.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin 1.3K Mar 22 13:31 OpenShiftInfra_hosts.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin  334 Mar 22 13:30 OpenShiftInfra_openshift_glusterfs.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin  144 Mar 22 13:30 OpenShiftMaster_groups.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin 1.3K Mar 22 13:30 OpenShiftMaster_hosts.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin  166 Mar 22 13:31 OpenShiftMaster_openshift_master.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin  144 Mar 22 13:30 OpenShiftWorker_groups.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin 1.3K Mar 22 13:30 OpenShiftWorker_hosts.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin  441 Mar 22 13:30 OpenShiftWorker_openshift_glusterfs.yml

Version-Release number of selected component (if applicable):
15-trunk  -p RHOS_TRUNK-15.0-RHEL-8-20190320.n.1

How reproducible:
100%

Steps to Reproduce:
1. Deploy OSP15 undercloud
2. Deploy OCP 3.11 overcloud

Actual results:

Deployment fails while running sudo /usr/bin/tripleo-deploy-openshift --config-download-dir /var/lib/mistral/openshift --image 192.168.24.1:8787/openshift3/ose-ansible
:v3.11 2>&1 | tee /var/lib/mistral/openshift/openshift/playbook.log\n exit ${PIPESTATUS[0]}

Expected results:
No failures.

Additional info:

Comment 1 Martin André 2019-03-22 20:49:06 UTC
Looks like a selinux issue:

(undercloud) [stack@undercloud-0 ~]$ ls -l /var/lib/mistral/openshift/openshift
total 20
-rw-rw-r--. 1 tripleo-admin tripleo-admin  383 Mar 22 16:00 global_gluster_vars.yml
-rw-rw-r--. 1 tripleo-admin tripleo-admin 3206 Mar 22 16:01 global_vars.yml
drwxr-xr-x. 2 tripleo-admin root          4096 Mar 22 16:01 inventory
-rw-rw-r--. 1 tripleo-admin tripleo-admin 1190 Mar 22 16:01 playbook.log
-rw-rw-r--. 1 tripleo-admin tripleo-admin 1262 Mar 22 16:01 playbook.yml


(undercloud) [stack@undercloud-0 ~]$ sudo podman run --net=host -u 0 -v /var/lib/mistral/openshift:/var/lib/mistral/openshift -t 192.168.24.1:8787/openshift3/ose-ansible:v3.11 ls -l /var/lib/mistral/openshift/openshift/
ls: cannot access /var/lib/mistral/openshift/openshift/global_gluster_vars.yml: Permission denied
ls: cannot access /var/lib/mistral/openshift/openshift/global_vars.yml: Permission denied
ls: cannot access /var/lib/mistral/openshift/openshift/playbook.yml: Permission denied
total 8
-?????????? ? ?    ?       ?            ? global_gluster_vars.yml
-?????????? ? ?    ?       ?            ? global_vars.yml
drwxr-xr-x. 2 1002 root 4096 Mar 22 20:01 inventory
-rw-rw-r--. 1 1002 1003 1190 Mar 22 20:01 playbook.log
-?????????? ? ?    ?       ?            ? playbook.yml

Comment 2 Martin André 2019-03-23 07:19:11 UTC
We need to add 'z' option while mounting the /var/lib/mistral/openshift.

(undercloud) [stack@undercloud-0 ~]$ sudo podman run --net=host -u 0 -v /var/lib/mistral/openshift:/var/lib/mistral/openshift:z -t 192.168.24.1:8787/openshift3/ose-ansible:v3.11 ls -l /var/lib/mistral/openshift/openshift/
total 20
-rw-rw-r--. 1 1002 1003  383 Mar 22 20:00 global_gluster_vars.yml
-rw-rw-r--. 1 1002 1003 3206 Mar 22 20:01 global_vars.yml
drwxr-xr-x. 2 1002 root 4096 Mar 22 20:01 inventory
-rw-rw-r--. 1 1002 1003 1190 Mar 22 20:01 playbook.log
-rw-rw-r--. 1 1002 1003 1262 Mar 22 20:01 playbook.yml

Will provide a patch shortly.

Comment 3 Martin André 2019-03-23 07:34:46 UTC
Submitted patch upstream at https://review.openstack.org/645979.

Comment 4 Martin André 2019-05-04 09:26:38 UTC
This was merged about a month ago upstream and already made its way into the downstream tripleo-common packaging.

Comment 8 Marius Cornea 2019-07-29 16:32:22 UTC
Director deployed OCP is deprecated in OSP15.


Note You need to log in before you can comment on or make changes to this bug.