Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
First Last Prev Next    This bug is not in your last search results.
Bug 1480172 - OSP11 -> OSP12 upgrade: during upgrade the switch-from-non-containerized-to-containerized-ceph-daemons.yml provided by ceph-ansible fails because it requires user input
OSP11 -> OSP12 upgrade: during upgrade the switch-from-non-containerized-to-c...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
12.0 (Pike)
Unspecified Unspecified
urgent Severity urgent
: rc
: 12.0 (Pike)
Assigned To: Giulio Fidente
Yogev Rabl
: Triaged
Depends On:
Blocks:
  Show dependency tree / graph
 
Reported: 2017-08-10 06:23 EDT by Marius Cornea
Modified: 2018-02-05 14:12 EST (History)
20 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170821194253.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-12-13 16:51:30 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenStack gerrit 494246 None None None 2017-08-16 11:41 EDT
Github ceph/ceph-ansible/issues/1758 None None None 2017-08-10 07:38 EDT
Red Hat Product Errata RHEA-2017:3462 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-15 20:43:25 EST

  None (edit)
Description Marius Cornea 2017-08-10 06:23:35 EDT 
Description of problem:
OSP11 -> OSP12 upgrade: major-upgrade-composable-steps-docker fails with resources.WorkflowTasks_Step2_Execution: ERROR when upgrading ceph deployments with ceph-ansible.yaml

Version-Release number of selected component (if applicable):
ceph-ansible-2.3.0-0.1.rc5.el7cp.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170805163045.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy basic OSP11 environment with 1 ctrl, 1 compute, 1 ceph node:

timeout 100m openstack overcloud deploy \
--templates /usr/share/openstack-tripleo-heat-templates \
--libvirt-type kvm \
--ntp-server clock.redhat.com \
-e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \
-e /home/stack/virt/internal.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /home/stack/virt/hostnames.yml \
-e /home/stack/virt/debug.yaml \
-e /home/stack/virt/nodes_data.yaml 

2. Upgrade undercloud to OSP12

3. Run major-upgrade-composable-steps-docker upgrade step and switch to ceph-ansible:

timeout 100m openstack overcloud deploy \
--templates /usr/share/openstack-tripleo-heat-templates \
--libvirt-type kvm \
--ntp-server clock.redhat.com \
-e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
-e /home/stack/virt/internal.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /home/stack/virt/hostnames.yml \
-e /home/stack/virt/debug.yaml \
-e /home/stack/virt/nodes_data.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-composable-steps-docker.yaml \
-e /home/stack/docker-osp12.yaml \

Set the CephAnsiblePlaybook parameter to point to the switch-from-non-containerized-to-containerized-ceph-daemons.yml playbook:

CephAnsiblePlaybook: /home/stack/switch-from-non-containerized-to-containerized-ceph-daemons.yml

Actual results:
major-upgrade-composable-steps-docker fails

Expected results:
upgrade completes fine.

Additional info:

The ceph-ansible command:

', u'become': True, u'forks': 3, u'ssh_common_args': u'-o StrictHostKeyChecking=no', u'playbook': u'/home/stack/switch-from-non-containerized-to-containerized-ceph-daemons.yml'}']                                
 Unexpected error while running command.                                                                                                                                                                           
Command: ansible-playbook /tmp/ansible-mistral-actiono9ievc/playbook.yaml --user tripleo-admin --become --become-user root --extra-vars {\"monitor_secret\": \"***\", \"ceph_conf_overrides\": {\"global\": {\"osd_
pool_default_pg_num\": 32, \"osd_pool_default_size\": 1}}, \"fetch_directory\": \"/tmp/file-mistral-actionWNAiYg\", \"user_config\": true, \"ceph_docker_image_tag\": \"latest\", \"containerized_deployment\": tru
e, \"public_network\": \"172.17.3.0/24\", \"generate_fsid\": false, \"monitor_address_block\": \"172.17.3.0/24\", \"monitor_interface\": \"br_ex\", \"admin_secret\": \"***\", \"keys\": [{\"mon_cap\": \"allow r\"
, \"osd_cap\": \"allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=backups, allow rwx pool=vms, allow rwx pool=images, allow rwx pool=metrics\", \"acls\": [\"u:glance:r--\", \"u
:nova:r--\", \"u:cinder:r--\", \"u:gnocchi:r--\"], \"name\": \"client.openstack\", \"key\": \"AQAuZ4tZAAAAABAAurr1kyUdEM3jwPepTiOrSQ==\"}], \"openstack_keys\": [{\"mon_cap\": \"allow r\", \"osd_cap\": \"allow cl
ass-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=backups, allow rwx pool=vms, allow rwx pool=images, allow rwx pool=metrics\", \"acls\": [\"u:glance:r--\", \"u:nova:r--\", \"u:cinder:r
--\", \"u:gnocchi:r--\"], \"name\": \"client.openstack\", \"key\": \"AQAuZ4tZAAAAABAAurr1kyUdEM3jwPepTiOrSQ==\"}], \"journal_collocation\": true, \"ntp_service_enabled\": false, \"ceph_docker_image\": \"rhceph-2
-rhel7\", \"cluster_network\": \"172.17.4.0/24\", \"fsid\": \"cb721718-7d3b-11e7-ad93-5254008ebd12\", \"journal_size\": 512, \"openstack_config\": true, \"ceph_docker_registry\": \"192.168.24.1:8787/ceph\", \"ce
ph_stable\": true, \"devices\": [\"/dev/vdb\"], \"ceph_origin\": \"distro\", \"openstack_pools\": [{\"rule_name\": \"\", \"pg_num\": 32, \"name\": \"volumes\"}, {\"rule_name\": \"\", \"pg_num\": 32, \"name\": \"
backups\"}, {\"rule_name\": \"\", \"pg_num\": 32, \"name\": \"vms\"}, {\"rule_name\": \"\", \"pg_num\": 32, \"name\": \"images\"}, {\"rule_name\": \"\", \"pg_num\": 32, \"name\": \"metrics\"}], \"pools\": [], \"
docker\": true, \"ip_version\": \"ipv4\"} --forks 3 --ssh-common-args \"-o StrictHostKeyChecking=no\" --ssh-extra-args \"-o UserKnownHostsFile=/dev/null\" --inventory-file /tmp/ansible-mistral-actiono9ievc/inven
tory.yaml --private-key /tmp/ansible-mistral-actiono9ievc/ssh_private_key --skip-tags package-install,with_pkg                                                                                                     
Exit code: 4                                                                                                                                                                                                       
Stdout: u''                                                                                                                                                                                                        
Stderr: u\"[WARNING]: log file at /var/log/ansible.log is not writeable and we cannot create it, aborting\                                                                                                         
\                                                                                                                                                                                                                  
ERROR! playbooks must be a list of plays\                                                                                                                                                                          
\                                                                                                                                                                                                                  
The error appears to have been in '/tmp/ansible-mistral-actiono9ievc/playbook.yaml': line 1, column 1, but may\                                                                                                    
be elsewhere in the file depending on the exact syntax problem.\                                                                                                                                                   
\                                                                                                                                                                                                                  
The offending line appears to be:\                                                                                                                                                                                 
\                                                                                                                                                                                                                  
\                                                                                                                                                                                                                  
/home/stack/switch-from-non-containerized-to-containerized-ceph-daemons.yml\                                                                                                                                       
^ here\                                                                                                                                                                                                            
\""                                                                                                                                                                                                                


The inventory file:
inventory.yaml:
clients:                      
  hosts:                      
    192.168.24.9: {}          
mdss:                         
  hosts: {}                   
mons:                         
  hosts:                      
    192.168.24.17: {}         
osds:                         
  hosts:                      
    192.168.24.13: {}         
rgws:                         
  hosts: {}
Comment 1 Giulio Fidente 2017-08-10 06:53:51 EDT 
Content of the playbook.yaml file is as follows:

# more playbook.yaml 
/home/stack/switch-from-non-containerized-to-containerized-ceph-daemons.yml
Comment 2 Giulio Fidente 2017-08-10 06:58:27 EDT 
Looks like the playbook to run is misintepreted as a string to build the playbook from instead of copying the contents of the file into playbook.yaml
Comment 3 Giulio Fidente 2017-08-10 07:11:20 EDT 
I believe the issue is that /home/stack/switch-from-non-containerized-to-containerized-ceph-daemons.yml was not readable by mistral. 

If we set CephAnsiblePlaybook: to /usr/share/ceph-ansible/infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml the playbook should run (and probably fail later on something different).

Can you confirm the above relaunching the upgrade?
Comment 4 Marius Cornea 2017-08-10 07:36:12 EDT 
(In reply to Giulio Fidente from comment #3)
> I believe the issue is that
> /home/stack/switch-from-non-containerized-to-containerized-ceph-daemons.yml
> was not readable by mistral. 
> 
> If we set CephAnsiblePlaybook: to
> /usr/share/ceph-ansible/infrastructure-playbooks/switch-from-non-
> containerized-to-containerized-ceph-daemons.yml the playbook should run (and
> probably fail later on something different).
> 
> Can you confirm the above relaunching the upgrade?

So we changed CephAnsiblePlaybook to point to the one provided in /usr/share and now it failed as the playbook requires user input:

https://github.com/ceph/ceph-ansible/blob/master/infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml#L11-L15
Comment 7 Giulio Fidente 2017-08-11 02:55:58 EDT 
We can workaround the prompt passing its value via THT
Comment 12 Yogev Rabl 2017-11-15 22:35:05 EST 
verified on openstack-tripleo-heat-templates-7.0.3-6.el7ost.noarch
Comment 15 errata-xmlrpc 2017-12-13 16:51:30 EST 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.