Bug 1372916

Summary: zero byte ifcfg files and ssh keys after overcloud deployment in Ravello
Product: Red Hat OpenStack Reporter: Michael Jarrett <mjarrett>
Component: rhosp-directorAssignee: Angus Thomas <athomas>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Omri Hochman <ohochman>
Severity: urgent Docs Contact:
Priority: high    
Version: 8.0 (Liberty)CC: bfournie, dbecker, dsneddon, dtantsur, jcoufal, mburns, mjarrett, morazi, rhel-osp-director-maint
Target Milestone: ---Keywords: Triaged
Target Release: ---Flags: dsneddon: needinfo? (mjarrett)
bfournie: needinfo? (mjarrett)
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-19 20:26:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Michael Jarrett 2016-09-03 19:01:56 UTC
Description of problem: After deploying the overcloud, ssh keys and ifcfg files are zero btye when deploying in Ravello. This issue is not present with the ILT implementation. When this happens to compute1, if I remove the ifcfg-eth0 file and reboot, compute then receives the proper ifcfg-eth0 file. In the cloud-init log DataSourceConfigDriveNet points to /dev/vda1. This is the drive that is mounted and the data is copied to the controller when the image is transferred. When I mount /dev/vda1, the correct information for meta-data.json is there. The /etc/os-collect-config.conf has the proper meta-data url as well.



How reproducible:


Steps to Reproduce:
1. Deploy the overcloud 
 [stack@director ~]$ openstack overcloud deploy --templates ~/templates \
 --ntp-server 172.25.250.254 \
 --control-scale 1 --compute-scale 1 \
 --control-flavor control --compute-flavor compute \
 --neutron-tunnel-types vxlan --neutron-network-type vxlan \
 -e ~/templates/environments/network-isolation.yaml \
 -e ~/templates/network-environment.yaml
2. Attempt to SSH into the compute1 VM
3. Console into compute1 VM and CAT the ifcfg-eth0 file and the authorized_keys file.

Actual results:
Cannot SSH into the compute1 VM. After accessing the VM using the console, ifcfg-eth0 and authorized_keys are zero byte.

Expected results:
Access the compute1 VM using ssh. The ifcfg-eth0 and authorized_keys files are  populated correctly.

Additional info:

Compute1-Cloud-init.log
Aug 30 02:47:21 localhost cloud-init: Cloud-init v. 0.7.6 running 'init-local' at Tue, 30 Aug 2016 06:47:21 +0000. Up 11.59 seconds.
Aug 30 02:47:30 localhost cloud-init: Cloud-init v. 0.7.6 running 'init' at Tue, 30 Aug 2016 06:47:30 +0000. Up 21.10 seconds.
Aug 30 02:47:31 localhost cloud-init: 2016-08-30 02:47:31,248 - util.py[WARNING]: Route info failed: Unexpected error while running command.
Aug 30 02:47:31 localhost cloud-init: Command: ['netstat', '-rn']
Aug 30 02:47:31 localhost cloud-init: Exit code: 1
Aug 30 02:47:31 localhost cloud-init: Reason: -
Aug 30 02:47:31 localhost cloud-init: Stdout: 'Kernel IP routing table\nDestination     Gateway         Genmask         Flags   MSS Window  irtt Iface\n'
Aug 30 02:47:31 localhost cloud-init: Stderr: ''
Aug 30 02:47:31 localhost cloud-init: ci-info: +++++++++++++++++++++++Net device info+++++++++++++++++++++++
Aug 30 02:47:31 localhost cloud-init: ci-info: +--------+------+-----------+-----------+-------------------+
Aug 30 02:47:31 localhost cloud-init: ci-info: | Device |  Up  |  Address  |    Mask   |     Hw-Address    |
Aug 30 02:47:31 localhost cloud-init: ci-info: +--------+------+-----------+-----------+-------------------+
Aug 30 02:47:31 localhost cloud-init: ci-info: |  lo:   | True | 127.0.0.1 | 255.0.0.0 |         .         |
Aug 30 02:47:31 localhost cloud-init: ci-info: | eth0:  | True |     .     |     .     | 52:54:00:00:fa:0c |
Aug 30 02:47:31 localhost cloud-init: ci-info: +--------+------+-----------+-----------+-------------------+
Aug 30 02:47:31 localhost cloud-init: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Aug 30 02:47:38 localhost cloud-init: Cloud-init v. 0.7.6 running 'modules:config' at Tue, 30 Aug 2016 06:47:37 +0000. Up 28.23 seconds.
Aug 30 02:47:39 localhost cloud-init: Cloud-init v. 0.7.6 running 'modules:final' at Tue, 30 Aug 2016 06:47:39 +0000. Up 30.06 seconds.
Aug 30 02:47:40 localhost cloud-init: Cloud-init v. 0.7.6 finished at Tue, 30 Aug 2016 06:47:40 +0000. Datasource DataSourceConfigDriveNet [net,ver=2][source=/dev/vda1].  Up 30.52 seconds

Compute1-Cloud-init.log after removing /etc/sysconfig/network-scripts/ifcfg-eth0 and the rebooting

Aug 30 02:49:39 overcloud-novacompute-0 cloud-init: ci-info: +-------+-----------------+---------------+-----------------+-----------+-------+
Aug 30 02:49:39 overcloud-novacompute-0 cloud-init: ci-info: | Route |   Destination   |    Gateway    |     Genmask     | Interface | Flags |
Aug 30 02:49:39 overcloud-novacompute-0 cloud-init: ci-info: +-------+-----------------+---------------+-----------------+-----------+-------+
Aug 30 02:49:39 overcloud-novacompute-0 cloud-init: ci-info: |   0   |     0.0.0.0     | 172.25.250.10 |     0.0.0.0     |    eth0   |   UG  |
Aug 30 02:49:39 overcloud-novacompute-0 cloud-init: ci-info: |   1   | 169.254.169.254 | 172.25.250.10 | 255.255.255.255 |    eth0   |  UGH  |
Aug 30 02:49:39 overcloud-novacompute-0 cloud-init: ci-info: |   2   |   172.25.250.0  |    0.0.0.0    |  255.255.255.0  |    eth0   |   U   |
Aug 30 02:49:39 overcloud-novacompute-0 cloud-init: ci-info: +-------+-----------------+---------------+-----------------+-----------+-------+
Aug 30 02:49:45 overcloud-novacompute-0 cloud-init: Cloud-init v. 0.7.6 running 'modules:config' at Tue, 30 Aug 2016 06:49:45 +0000. Up 33.31 seconds.
Aug 30 02:49:46 overcloud-novacompute-0 cloud-init: Cloud-init v. 0.7.6 running 'modules:final' at Tue, 30 Aug 2016 06:49:46 +0000. Up 34.83 seconds.
Aug 30 02:49:47 overcloud-novacompute-0 cloud-init: Cloud-init v. 0.7.6 finished at Tue, 30 Aug 2016 06:49:47 +0000. Datasource DataSourceConfigDriveNet [net,ver=2][source=/dev/vda1].  Up 35.07 seconds
e(pstate=pstate)

Comment 2 Dan Sneddon 2016-10-14 17:40:07 UTC
Do you know if this bug is still present with OSP 9 or OSP 10?

Comment 4 Bob Fournier 2017-07-20 18:55:36 UTC
This looks similar to https://bugzilla.redhat.com/show_bug.cgi?id=1467496, although that was for OSP-10.  In 1467496 it was a neutron openvswitch bug that has been fixed which was causing the IP addresses (and ifcfg files) to not be populated on the compute, along with other deployment parameters.

Michael - can you indicate in which release you were seeing this problem? That will allow us to determine if its the same issue.

Comment 5 Bob Fournier 2017-09-03 15:21:42 UTC
Any update on requested info?

Comment 6 Bob Fournier 2017-09-19 20:26:04 UTC
Closing this out, its been almost a year since info has been requested and not supplied.