Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1133715

Summary: Rubygem-Staypuft: LiveCD on bare metal - Neutron+Vxlan deployment fails upon installing the networker - reports: Execution of '/usr/sbin/ifdown enp2s0f1' returned 1
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: rhel-osp-installer LiveCDAssignee: Jiri Stransky <jstransk>
Status: CLOSED WORKSFORME QA Contact: Alexander Chuzhoy <sasha>
Severity: high Docs Contact:
Priority: high    
Version: 5.0 (RHEL 7)CC: aberezin, ajeain, mburns, morazi, rhos-maint, sasha, sclewis, yeylon
Target Milestone: z2Keywords: Triaged
Target Release: Installer   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-28 14:14:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1141936, 1145301    
Bug Blocks:    
Attachments:
Description Flags
/var/log/messages file from the networker none

Description Alexander Chuzhoy 2014-08-25 23:58:47 UTC
Created attachment 930669 [details]
/var/log/messages file from the networker

Rubygem-Staypuft:  LiveCD on bare metal - Neutron+Vxlan deployment fails upon installing the networker  - reports: 	Execution of '/usr/sbin/ifdown enp2s0f1' returned 1

Environment: rhelosp-installer-live-6.5-20140820.1.iso

Steps to reproduce:

1. Boot a bare metal host from the live cd and install rhel-osp-installer.
2. Configure/run a Neutron+Vxlan networking type deployment with 2 computes.

Result:
The deployment fails upon installing the networker. 


Expected result:
The deployment should complete with no issues.


Note: 
1) 
I see no related errors in the /var/log/messages files - attaching anyway.
Resumed the deployment and it resumed with no issues.

2)
I noticed that the default route was configured through the NIC connected to the isolated network instead of the externally connected NIC.


The YAML for the host is below:
---
classes:
  quickstack::neutron::networker:
    amqp_host: 192.168.0.4
    amqp_password: 4bf041971f1e6d0748d17479faf209c0
    amqp_provider: rabbitmq
    amqp_username: openstack
    controller_priv_host: 192.168.0.4
    enable_tunneling: 'true'
    external_network_bridge: ''
    fixed_network_range: ${$quickstack::params::fixed_network_range}
    mysql_ca: /etc/ipa/ca.crt
    mysql_host: 192.168.0.4
    neutron_db_password: 2eb3b0d310fcae74a4c51483cd210c0f
    neutron_metadata_proxy_secret: 2352bbfc7efe225ac2089e46119883d7
    neutron_user_password: 2579606267b1f1149958b8e3be8c7235
    nova_db_password: 367c2a1c0c5d1d61eba58bdb2cd1a320
    nova_user_password: 331940ea7c596a99769acff8637f299c
    ovs_bridge_mappings:
    - physnet-external:br-ex
    ovs_bridge_uplinks:
    - br-ex:enp2s0f1
    ovs_l2_population: 'True'
    ovs_tunnel_iface: enp2s0f0
    ovs_tunnel_network: ''
    ovs_tunnel_types:
    - vxlan
    - gre
    ovs_vlan_ranges:
    - physnet-external
    ovs_vxlan_udp_port: '4789'
    ssl: 'false'
    tenant_network_type: vxlan
    tunnel_id_ranges: 1:1000
    verbose: 'true'
parameters:
  puppetmaster: livecd.example.com
  domainname: Default domain used for provisioning
  hostgroup: base_RedHat_7/Neutron-vxlan/Neutron Networker
  root_pw: $5$fm$EO4A.ybSB/ofUaZWkzNePd38XRpUQNXls8y1feWvIy3
  puppet_ca: livecd.example.com
  foreman_env: production
  owner_name: Admin User
  owner_email: root
  ip: 192.168.0.6
  mac: 04:7d:7b:62:7d:5a
  ntp-server: clock.redhat.com
  ui::cinder::driver_backend: lvm
  ui::cinder::eqlx_group_name: group-0
  ui::cinder::eqlx_pool: default
  ui::cinder::rbd_secret_uuid: 8d3e27f4-e2a8-4348-843c-b1487170fa54
  ui::cinder::san_login: grpadmin
  ui::deployment::amqp_provider: rabbitmq
  ui::deployment::layout_name: Controller / Compute
  ui::deployment::networking: neutron
  ui::deployment::platform: rhel7
  ui::glance::driver_backend: local
  ui::neutron::compute_tenant_interface: enp2s0f0
  ui::neutron::external_interface_name: enp2s0f1
  ui::neutron::networker_tenant_interface: enp2s0f0
  ui::neutron::network_segmentation: vxlan
  ui::neutron::use_external_interface: 'true'
  ui::nova::network_manager: FlatDHCPManager
  ui::passwords::admin: f8efa110d6c2d1536ab1c23b0c7e5cdd
  ui::passwords::amqp: 4bf041971f1e6d0748d17479faf209c0
  ui::passwords::amqp_nssdb: da381652199c7a4b96c2b37ed9f61ec5
  ui::passwords::ceilometer_metering_secret: 5d0247da453a056dfb742eb5f2e2405f
  ui::passwords::ceilometer_user: eb25899a3da8ae9137cc3c9162468e37
  ui::passwords::cinder_db: ddc8661b8d2ad8d99580c1e59d684c4e
  ui::passwords::cinder_user: 2b5b0f28431a341784437eaa80d5ceeb
  ui::passwords::glance_db: f22454a952078bbad384699b479b7202
  ui::passwords::glance_user: 0ffde0efafbf53afe1759ed5f86211e2
  ui::passwords::heat_auth_encrypt_key: 7f6a58121cc009a89fca2abc0b386eae
  ui::passwords::heat_cfn_user: 0e484820dd1359e0d4ac0a90aa6bc880
  ui::passwords::heat_db: c05df0f1c8d2df3b43484f0428d3da5f
  ui::passwords::heat_user: 5ca80faf835714eb2f69d8fe32539a8e
  ui::passwords::horizon_secret_key: 4c47b806308a5d1500d230e9fa4a0005
  ui::passwords::keystone_admin_token: fb0923478e11f38aad58a9c8cfc1d809
  ui::passwords::keystone_db: 157772349b6f0662de0bdf9836cb25b9
  ui::passwords::keystone_user: 4418275b6183c12b487abff040f0ab7b
  ui::passwords::mode: random
  ui::passwords::mysql_root: 307d4d1b87fab85fc5878aac4306e5f9
  ui::passwords::neutron_db: 2eb3b0d310fcae74a4c51483cd210c0f
  ui::passwords::neutron_metadata_proxy_secret: 2352bbfc7efe225ac2089e46119883d7
  ui::passwords::neutron_user: 2579606267b1f1149958b8e3be8c7235
  ui::passwords::nova_db: 367c2a1c0c5d1d61eba58bdb2cd1a320
  ui::passwords::nova_user: 331940ea7c596a99769acff8637f299c
  ui::passwords::swift_admin: 1e48a4af52d71c7eb02985a008c127f0
  ui::passwords::swift_shared_secret: 5eef4218a9d34549e57a645f032a576a
  ui::passwords::swift_user: 0606e240361dbfbed1cc8aef0b4ab63e
environment: production

Comment 1 Mike Burns 2014-08-26 13:23:24 UTC
Seems like this might be an environmental issue.  

 Determining IP information for enp2s0f1... failed.

Does this reproduce on a non-livecd installation?

Comment 2 Alexander Chuzhoy 2014-08-26 13:46:20 UTC
This might be an environmental issue.
Notes:
1. The subsequent runs of puppet resulted with no issues on this host.
2. We haven't seen anything like that on this setup with non-LiveCD installations.

Comment 3 Alexander Chuzhoy 2014-08-26 22:23:05 UTC
Tried the same deployment in the same setup and it has completed successfully with no issues.

Comment 4 Alexander Chuzhoy 2014-08-28 23:05:43 UTC
Reproduced running Neutron+GRE from LiveCD.

Comment 6 Alexander Chuzhoy 2014-09-25 17:24:28 UTC
Verified: rhelosp-installer-live-6.5-20140924.1.iso


The reported issue doesn't reproduce and the deployment completes successfully. The instances have network availability.

Comment 7 Alexander Chuzhoy 2014-09-25 20:16:27 UTC
Re-opening - reproduced with Neutron+Vlan deployment from LiveCD  on bare metal.

Comment 9 Alexander Chuzhoy 2014-09-29 18:03:37 UTC
Reproduced again with Neutron+Vlan deployment from LiveCD  on bare metal.

Comment 10 Arthur Berezin 2014-09-30 08:35:52 UTC
Does this happen only when using LiveCD?

Comment 11 Alexander Chuzhoy 2014-09-30 13:34:37 UTC
Didn't happen on VM setup.

Comment 12 Jiri Stransky 2014-10-10 16:28:53 UTC
Discussed with Sasha on irc, it seems to be intermittently occurring issue. There's a hint in the attached /var/log/messages what's probably the cause, this is the interesting bit:

http://pastebin.test.redhat.com/239361

Both enp2s0f0 and enp2s0f1 interfaces are set to be configured via DHCP, but configuration succeeded only for enp2s0f0. The interface enp2s0f1 didn't get a DHCP response.

We can investigate further when we hit the issue again.

Comment 13 Mike Burns 2014-10-28 14:14:30 UTC
given that this is not reproducing, closing this bug.  If it does reproduce, please provide an environment and re-open or re-file the BZ.

Thanks