Bug 1105479 - Failed to execute stage 'Misc configuration': 'GATEWAY'
Summary: Failed to execute stage 'Misc configuration': 'GATEWAY'
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-setup
Version: 3.4.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: 3.5.0
Assignee: Sandro Bonazzola
QA Contact: Nikolai Sednev
URL:
Whiteboard: integration
Depends On: 1180085
Blocks: rhev3.5beta 1156165
TreeView+ depends on / blocked
 
Reported: 2014-06-06 08:57 UTC by Artyom
Modified: 2015-02-11 20:40 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, deployment of the hosted engine failed if the 'GATEWAY' was specified in network configuration files and 'BOOTPROTO' was set to 'none' due to use of deprecated 'addNetwork' API in network setup. Now, the python vdsm.cli module is used in network setup and deployment of the hosted engine does not fail in this situation.
Clone Of:
Environment:
Last Closed: 2015-02-11 20:40:06 UTC
oVirt Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
hosted-engine-setup.log (265.17 KB, text/x-log)
2014-06-06 08:57 UTC, Artyom
no flags Details
log from deploy hosted-engine -> cacert.pem (139.76 KB, text/plain)
2014-11-25 09:08 UTC, Petr Kubica
no flags Details
ovirt hosted engine setup (245.65 KB, text/plain)
2015-01-08 09:34 UTC, Petr Matyáš
no flags Details
logs (42.87 KB, application/x-gzip)
2015-01-27 16:14 UTC, Nikolai Sednev
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0161 0 normal SHIPPED_LIVE ovirt-hosted-engine-setup bug fix and enhancement update 2015-12-07 21:35:11 UTC
oVirt gerrit 28595 0 None MERGED net_config: use vdscli and setupNetworks 2020-02-05 13:25:50 UTC
oVirt gerrit 32297 0 None MERGED net_config: use vdscli and setupNetworks 2020-02-05 13:25:50 UTC

Description Artyom 2014-06-06 08:57:39 UTC
Created attachment 902818 [details]
hosted-engine-setup.log

Description of problem:
In time of hosted-engine deployment process, you have possibility to choose network interface to setup bridge on it(ifcfg-ethX), but if ifcfg-ethX not have parameters GATEWAY and also BOOTPROTO=none, deployment with next traceback in setup log:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/otopi/context.py", line **FILTERED**42, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/network/bridge.py", line 238, in _misc
    'gateway=%s' % caps['cfg']['GATEWAY'],
KeyError: 'GATEWAY'

Version-Release number of selected component (if applicable):
ovirt-hosted-engine-setup-1.1.2-5.el6ev.noarch

How reproducible:
Always

Steps to Reproduce:
1. Edit one of network interface scripts(ifcfg-ethX), comment GATEWAY line(if you have one) and change BOTPROTO=none
2. Run hosted-engine --deploy process and choose ethX as network interface and continue deployment process
3.

Actual results:
hosted-engine --deploy failed with ERROR

Expected results:
Hosted-engine --deploy must success without any errors(not sure about it, because if interface have bootproto=none, rhevm bridge also can't receive ip) or at least check interface parameters before and print appropriate message something like 'EthX can't be used as interface for hosted-engine, because *reason*'

Additional info:

Comment 1 Sandro Bonazzola 2014-06-13 10:33:56 UTC
Waiting on Antoni's patch adding gateway info as commented in http://gerrit.ovirt.org/28595

Comment 2 Eyal Edri 2014-07-21 12:43:07 UTC
removing AutomationBlocker, since its not automation blocker.

Comment 3 Sandro Bonazzola 2014-07-21 12:45:53 UTC
Antoni, assigning this to you since you're the one working on the gateway thing.

Comment 4 Sandro Bonazzola 2014-09-02 12:30:53 UTC
Antoni, merged patch http://gerrit.ovirt.org/28595: do we need anything else here?

Comment 5 Antoni Segura Puimedon 2014-09-02 12:55:38 UTC
That should be all. We can move it to QE as far as I'm concerned

Comment 7 Petr Kubica 2014-11-21 10:37:54 UTC
Hi, 
There isn't any warning or error. Is it correct behavior ?
When I deployed hosted engine via ssh in screen, it disconnected me from server without any warning. 
Deploying contiuned on server withnout any error, but in the next step I must manually set default gateway to continue (e.g. install VM from pxe)

Petr

Comment 8 Antoni Segura Puimedon 2014-11-21 11:33:36 UTC
With the patch or without?

Comment 9 Petr Kubica 2014-11-25 09:05:48 UTC
I think, the patch is in rhevm repository, but it isn't.
I tested old version.

I installed:
ovirt-hosted-engine-ha-1.3.0-0.0.master.20141106150008.20141106150006.git058aaf1.el6.noarch
ovirt-hosted-engine-setup-1.3.0-0.0.master.20140902095911.git25962a3.el6.noarch
ovirt-engine-sdk-python-3.6.0.0-0.6.20141111.git2006d3f.el6.noarch

But when I tried run hosted-engine --deploy it give me an error with cert

# hosted-engine --deploy[ INFO  ] Stage: Initializing
          Continuing will configure this host for serving as hypervisor and create a VM where you have to install oVirt Engine afterwards.
          Are you sure you want to continue? (Yes, No)[Yes]: 
[ INFO  ] Generating a temporary VNC password.
[ INFO  ] Stage: Environment setup
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20141125095546-bfwhuo.log
          Version: otopi-1.3.0 (otopi-1.3.0-1.el6ev)
[ INFO  ] Hardware supports virtualization
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Generating VDSM certificates
[ ERROR ] Failed to execute stage 'Environment setup': [Errno 2] No such file or directory: '/etc/pki/vdsm/certs/cacert.pem'
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf'
[ INFO  ] Answer file '/etc/ovirt-hosted-engine/answers.conf' has been updated
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination

see the attached log

Comment 10 Petr Kubica 2014-11-25 09:08:37 UTC
Created attachment 961072 [details]
log from deploy hosted-engine -> cacert.pem

Comment 12 Petr Kubica 2014-12-11 15:39:50 UTC
I set on eth0 BOOTPROTO="none"

I found workaround for my issue with cert: "vdsm-tool configure --force" from
https://bugzilla.redhat.com/show_bug.cgi?id=1139020

Then I ran hosted-engine --deploy, at choose a nic, I choose eth0
Please indicate a nic to set ovirtmgmt bridge on: (eth1, usb0, eth0) [eth1]: eth0

but the connection get lost on 
[ INFO  ] Configuring the management bridge

Then I must set default gw via KVM. No error or no warnings during the deploy hosted engine.

ovirt-hosted-engine-ha-1.3.0-0.0.master.20141208140105.20141208140103.gitf73bdea.el6.noarch
ovirt-hosted-engine-setup-1.3.0-0.0.master.20140902095911.git25962a3.el6.noarch

Comment 13 Sandro Bonazzola 2014-12-15 07:44:27 UTC
(In reply to Petr Kubica from comment #12)
> I set on eth0 BOOTPROTO="none"
> 
> I found workaround for my issue with cert: "vdsm-tool configure --force" from
> https://bugzilla.redhat.com/show_bug.cgi?id=1139020
> 
> Then I ran hosted-engine --deploy, at choose a nic, I choose eth0
> Please indicate a nic to set ovirtmgmt bridge on: (eth1, usb0, eth0) [eth1]:
> eth0
> 
> but the connection get lost on 
> [ INFO  ] Configuring the management bridge
> 
> Then I must set default gw via KVM. No error or no warnings during the
> deploy hosted engine.
> 
> ovirt-hosted-engine-ha-1.3.0-0.0.master.20141208140105.20141208140103.
> gitf73bdea.el6.noarch
> ovirt-hosted-engine-setup-1.3.0-0.0.master.20140902095911.git25962a3.el6.
> noarch

Please don't mix ovirt rpms targeted to 3.6 with rhev packages targeted to 3.5 for verifying bugs on RHEV 3.5.
Moving back to ON_QA.
Please open an ovirt bug if you can reproduce above on a clean ovirt installation using packages from ovirt master..

Comment 14 Petr Matyáš 2015-01-08 09:34:08 UTC
Created attachment 977689 [details]
ovirt hosted engine setup

Failed with [ ERROR ] Failed to execute stage 'Misc configuration': Failed to setup networks {'rhevm': {'nic': 'eth0', 'netmask': '255.255.252.0', 'bootproto': 'none', 'ipaddr': '10.34.60.233'}}. Error code: "16" message: "Unexpected exception"

ovirt-hosted-engine-ha-1.2.4-5.el6ev.noarch
ovirt-hosted-engine-setup-1.2.1-8.el6ev.noarch

Comment 15 Sandro Bonazzola 2015-01-08 09:55:05 UTC
(In reply to Petr Matyáš from comment #14)
> Created attachment 977689 [details]
> ovirt hosted engine setup
> 
> Failed with [ ERROR ] Failed to execute stage 'Misc configuration': Failed
> to setup networks {'rhevm': {'nic': 'eth0', 'netmask': '255.255.252.0',
> 'bootproto': 'none', 'ipaddr': '10.34.60.233'}}. Error code: "16" message:
> "Unexpected exception"
> 
> ovirt-hosted-engine-ha-1.2.4-5.el6ev.noarch
> ovirt-hosted-engine-setup-1.2.1-8.el6ev.noarch

The above issue is unrelated to this bug. The unexpected exception come from VDSM api.
Please open a new bug on VDSM component about it and attach VDSM logs to that.

Comment 16 Nikolai Sednev 2015-01-25 14:03:03 UTC
Hi Sandro,
Can you clarify something regarding this bug please?
I'm a bit frustrated, how can possibly hosted-engine deployment succeed without gateway configured and available? 

During yum install of hosted engine, many mandatory components being sent to host, without configured gateway it's impossible, so gateway is mandatory, unless host not placed within the same subnet with it's repo server.

Regarding boot protocol I agree that it's not mandatory, as IP configuration might be set manually with DNS, IP and gateway parameters.

Comment 17 Nikolai Sednev 2015-01-27 16:13:07 UTC
	
2015-Jan-27, 13:33
Host blue-vdsc.qa.lab.tlv.redhat.com installation failed. Failed to configure management network on the host.

2015-Jan-27, 13:33
Failed to configure management network on host blue-vdsc.qa.lab.tlv.redhat.com due to setup networks failure.

rhevm-guest-agent-common-1.0.10-2.el6ev.noarch
rhevm-3.5.0-0.30.el6ev.noarch
vdsm-4.16.8.1-6.el6ev.x86_64
sanlock-2.8-1.el6.x86_64
libvirt-client-0.10.2-46.el6_6.2.x86_64
ovirt-hosted-engine-ha-1.2.4-5.el6ev.noarch
mom-0.4.1-4.el6ev.noarch
qemu-kvm-rhev-0.12.1.2-2.448.el6.x86_64
ovirt-hosted-engine-setup-1.2.1-9.el6ev.noarch

I've set ifcfg-eth0 settings of bootproto=none and there also were no configuration for default gateway, so I hadn't changed them.
I didn't configured ifcfg-eth0 to static IP, only configured IP parameters via cli as:
ifconfig 10.35.103.12 netmask 255.255.255.0 
route add default gw 10.35.103.254 eth0

So deployment continued to like this:

[ INFO  ] Stage: Setup validation                                                                                                                          
                                                                                                                                                           
          --== CONFIGURATION PREVIEW ==--                                                                                                                  
                                                                                                                                                           
          Bridge interface                   : eth0                                                                                                        
          Engine FQDN                        : nsednev-he-1.qa.lab.tlv.redhat.com                                                                          
          Bridge name                        : rhevm                                                                                                       
          SSH daemon port                    : 22                                                                                                          
          Firewall manager                   : iptables                                                                                                    
          Gateway address                    : 10.35.117.254                                                                                               
          Host name for web application      : blue-vdsc.qa.lab.tlv.redhat.com                                                                             
          Host ID                            : 6                                                                                                           
          Image alias                        : HE                                                                                                          
          Image size GB                      : 25                                                                                                          
          Storage connection                 : 10.35.160.108:/RHEV/nsednev_HE_3_5                                                                          
          Console type                       : vnc                                                                                                         
          Memory size MB                     : 4096                                                                                                        
          MAC address                        : 00:16:3E:7B:B8:53                                                                                           
          Boot type                          : disk                                                                                                        
          Number of CPUs                     : 2                                                                                                           
          CPU Type                           : model_Conroe                                                                                                
[ INFO  ] Stage: Transaction setup                                                                                                                         
[ INFO  ] Stage: Misc configuration                                                                                                                        
[ INFO  ] Stage: Package installation                                                                                                                      
[ INFO  ] Stage: Misc configuration                                                                                                                        
[ INFO  ] Configuring libvirt                                                                                                                              
[ INFO  ] Configuring VDSM                                                                                                                                 
[ INFO  ] Starting vdsmd                                                                                                                                   
[ INFO  ] Waiting for VDSM hardware info                                                                                                                   
[ INFO  ] Waiting for VDSM hardware info                                                                                                                   
[ INFO  ] Waiting for VDSM hardware info                                                                                                                   
[ INFO  ] Connected to Storage Domain                                                                                                                      
[ INFO  ] Configuring VM                                                                                                                                   
[ INFO  ] Updating hosted-engine configuration                                                                                                             
[ INFO  ] Stage: Transaction commit                                                                                                                        
[ INFO  ] Stage: Closing up                                                                                                                                
[ INFO  ] Waiting for the host to become operational in the engine. This may take several minutes...                                                       
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ INFO  ] Still waiting for VDSM host to become operational...                                                                                             
[ ERROR ] Timed out while waiting for host to start. Please check the logs.                                                                                
[ ERROR ] Unable to add blue-vdsc.qa.lab.tlv.redhat.com to the manager                                                                                     
[ INFO  ] Enabling and starting HA services                                                                                                                
          Hosted Engine successfully set up                                                                                                                
[ INFO  ] Stage: Clean up                                                                                                                                  
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150127173955.conf'                                                  
[ INFO  ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf'                                                                                   
[ INFO  ] Stage: Pre-termination                                                                                                                           
[ INFO  ] Stage: Termination                                                                                                                               
[root@blue-vdsc ~]#      


logs from host and engine attached.

Comment 18 Nikolai Sednev 2015-01-27 16:14:38 UTC
Created attachment 984744 [details]
logs

Comment 20 errata-xmlrpc 2015-02-11 20:40:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0161.html


Note You need to log in before you can comment on or make changes to this bug.