Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1609896

Summary: UC deployment fails on BM (An exception occurred during task execution. To see the full traceback, use -vvv. The error was: error: (25, 'Inappropriate ioctl for device'))
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: ansibleAssignee: Sam Doran <sdoran>
Status: CLOSED UPSTREAM QA Contact: Gurenko Alex <agurenko>
Severity: high Docs Contact:
Priority: high    
Version: 14.0 (Rocky)CC: agurenko, dbecker, emacchi, jcoufal, jjoyce, jschluet, mburns, morazi, slinaber, tvignaud
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-24 15:35:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2018-07-30 18:27:15 UTC 
UC deployment fails on BM (An exception occurred during task execution. To see the full traceback, use -vvv. The error was: error: (25, 'Inappropriate ioctl for device'))


Environment:
heat-cfntools-1.2.6-5.el7.noarch
python-heat-agent-docker-cmd-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-role-redhat-subscription-1.0.1-1.el7ost.noarch
openstack-heat-agents-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
openstack-heat-common-12.0.0-0.20180604085325.7d878a8.el7ost.noarch
python-tripleoclient-heat-installer-10.3.1-0.20180716131225.300dfb3.el7ost.noarch
python-heat-agent-json-file-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
python-heat-agent-apply-config-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-pacemaker-1.0.4-0.20180220234310.0e4d7c0.el7ost.noarch
ansible-role-tripleo-modify-image-0.0.1-0.20180718224444.c2ef865.el7ost.noarch
python-heat-agent-ansible-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
openstack-heat-api-12.0.0-0.20180604085325.7d878a8.el7ost.noarch
python2-heatclient-1.16.0-0.20180619123220.f095a0f.el7ost.noarch
python-heat-agent-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
python-heat-agent-puppet-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-role-container-registry-0.0.1-0.20180718101745.9fcedac.el7ost.noarch
instack-undercloud-9.1.1-0.20180716115151.bc82d48.el7ost.noarch
openstack-heat-monolith-12.0.0-0.20180604085325.7d878a8.el7ost.noarch
puppet-heat-13.1.1-0.20180719053944.8ef3646.el7ost.noarch
python-heat-agent-hiera-1.6.1-0.20180709100740.fdd6a5f.el7ost.noarch
ansible-2.5.4-1.el7ae.noarch
ansible-tripleo-ipsec-8.1.1-0.20180405121919.325d233.el7ost.noarch
openstack-tripleo-heat-templates-9.0.0-0.20180720154239.959e1d7.el7ost.noarch
openstack-heat-engine-12.0.0-0.20180604085325.7d878a8.el7ost.noarch


Steps to reproduce:

Edit undercloud.conf to include:
[stack@undercloud75 ~]$ cat undercloud.conf 
[DEFAULT]
# Network interface on the Undercloud that will be handling the PXE
# boots and DHCP for Overcloud instances. (string value)
local_interface = eth1
local_ip = 192.168.0.1/24
#undercloud_public_host = 192.168.0.2
#undercloud_admin_host = 192.168.0.3
generate_service_certificate = false
container_images_file=/home/stack/containers-prepare-parameter.yaml
undercloud_ntp_servers=clock.redhat.com
docker_insecure_registries=docker-registry.engineering.redhat.com
# BZ#1608508
undercloud_enable_selinux=False
[ctlplane-subnet]
local_subnet = ctlplane-subnet
cidr = 192.168.0.0/24
dhcp_start = 192.168.0.5
dhcp_end = 192.168.0.24
gateway = 192.168.0.1
inspection_iprange = 192.168.0.100,192.168.0.120
masquerade = true


#Note the attempt to use nonSSL undercloud.

Running 'openstack undercloud install' results in:

RUNNING HANDLER [container-registry : restart docker] ****************************************************************************************************************************************
changed: [undercloud75]

RUNNING HANDLER [container-registry : Docker | reload systemd] *******************************************************************************************************************************
ok: [undercloud75]

RUNNING HANDLER [container-registry : Docker | reload docker] ********************************************************************************************************************************
changed: [undercloud75]

RUNNING HANDLER [container-registry : Docker | pause while Docker restarts] ******************************************************************************************************************
Pausing for 10 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
[container-registry : Docker | pause while Docker restarts]
Waiting for docker restart:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: error: (25, 'Inappropriate ioctl for device')
fatal: [undercloud75]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}

RUNNING HANDLER [container-registry : Docker | wait for docker] ******************************************************************************************************************************

PLAY RECAP ***********************************************************************************************************************************************************************************
undercloud75               : ok=141  changed=36   unreachable=0    failed=1

Install artifact is located at /home/stack/undercloud-install-20180730171454.tar.bzip2

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Deployment Failed!

ERROR: Heat log files: /var/log/heat-launcher/undercloud_deploy-ypFKGI

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Deployment failed.
'InstallUndercloud' object has no attribute 'heat_launch'




###############################

Re-running 'openstack undercloud install' proceeds.
Comment 2 Alexander Chuzhoy 2018-07-30 21:03:01 UTC 
The issue reproduces.
Comment 3 Emilien Macchi 2018-07-30 21:13:43 UTC 
So the trace is visible here:
http://paste.openstack.org/show/726884/

See https://github.com/ansible/ansible/commit/1d1595b990641b6a3446c6a94ddf992ef9cfac39 for context.
We probably need a backport in Ansible 2.5.
Comment 4 Sam Doran 2018-07-30 21:36:26 UTC 
This bug was introduced in Ansible 2.5.4. It is fixed in Ansible 2.5.7 and 2.6.2.

Upstream fix: https://github.com/ansible/ansible/pull/42217

Upstream backport for Ansible 2.5:

https://github.com/ansible/ansible/pull/42441

Upstream backport for Ansible 2.6:

https://github.com/ansible/ansible/pull/42440
Comment 6 Alexander Chuzhoy 2018-07-30 23:41:24 UTC 
The failure reproduces even with SSL enabled (default).
Comment 7 Alexander Chuzhoy 2018-07-30 23:44:15 UTC 
Switching back to assigned.
We need to have the right version available downstream to verify it works.
Comment 8 Sam Doran 2018-09-20 13:41:21 UTC 
This will always fail regardless of the SSL configuration since it is due to a bug in the pause module in Ansible when redirecting stdout to a file (via >, >>, or tee). The fix is to ensure Ansible 2.5.4-2.5.6 are not used.
Comment 9 Sam Doran 2018-09-21 20:13:15 UTC 
From looking at the list of packages on the puddle[1], Ansible 2.7 is present. This should no longer be happening.

[1]: http://download-node-02.eng.bos.redhat.com/rcm-guest/puddles/OpenStack/14.0-RHEL-7/latest/RH7-RHOS-14.0/x86_64/os/Packages/
Comment 11 Sam Doran 2018-09-24 15:35:55 UTC 
Confirmed this is now resolved.