Bug 1518850

Created attachment 1360432 [details]
ovirt-hosted-engine-setup log

Description of problem:
hosted-engine --deploy failed on clean deployment over NFS


Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. hosted-engine --deploy

Actual results:
stuck on Misc Configuration stage on the engine.

Expected results:
deployment finish successfully

Additional info:
hosted-engine --deploy output:

[root@localhost ~]# hosted-engine --deploy
[ INFO  ] Stage: Initializing
[ INFO  ] Generating a temporary VNC password.
[ INFO  ] Stage: Environment setup
          During customization use CTRL-D to abort.
          Continuing will configure this host for serving as hypervisor and create a VM where you have to install the engine afterwards.
          Are you sure you want to continue? (Yes, No)[Yes]: 
          It has been detected that this program is executed through an SSH connection without using screen.
          Continuing with the installation may lead to broken installation if the network connection fails.
          It is highly recommended to abort the installation and run it inside a screen session using command "screen".
          Do you want to continue anyway? (Yes, No)[No]: yes
[ INFO  ] Hardware supports virtualization
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20171129102032-cob2nq.log
          Version: otopi-1.7.4_master (otopi-1.7.4-0.0.master.20171127123410.git0db8111.el7.centos)
[ INFO  ] Detecting available oVirt engine appliances
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Generating libvirt-spice certificates
[ INFO  ] Stage: Environment customization
         
          --== STORAGE CONFIGURATION ==--
         
          Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs3, nfs4)[nfs3]: 
          Please specify the full shared storage connection path to use (example: host:/path): 192.168.100.50:/var/nfs_folder
         
          --== HOST NETWORK CONFIGURATION ==--
         
          Please indicate a pingable gateway IP address [192.168.100.1]: 
          iptables was detected on your computer, do you wish setup to configure it? (Yes, No)[Yes]: 
          Please indicate a nic to set ovirtmgmt bridge on: (ens3) [ens3]: 
         
          --== VM CONFIGURATION ==--
         
          The following appliance have been found on your system:
          	[1] - The oVirt Engine Appliance image (OVA) - 4.2-20171127.1.el7.centos
          	[2] - Directly select an OVA file
          Please select an appliance (1, 2) [1]: 
[ INFO  ] Verifying its sha1sum
[ INFO  ] Checking OVF archive content (could take a few minutes depending on archive size)
[ INFO  ] Checking OVF XML content (could take a few minutes depending on archive size)
          Please specify the console type you would like to use to connect to the VM (vnc, spice) [vnc]: 
[ INFO  ] Detecting host timezone.
          Would you like to use cloud-init to customize the appliance on the first boot (Yes, No)[Yes]? 
          Would you like to generate on-fly a cloud-init ISO image (of no-cloud type)
          or do you have an existing one (Generate, Existing)[Generate]? 
          Please provide the FQDN you would like to use for the engine appliance.
          Note: This will be the FQDN of the engine VM you are now going to launch,
          it should not point to the base host or to any other existing machine.
          Engine VM FQDN: (leave it empty to skip):  []: she-engine.idodoroz.com
          Please provide the domain name you would like to use for the engine appliance.
          Engine VM domain: [idodoroz.com]
          Automatically execute engine-setup on the engine appliance on first boot (Yes, No)[Yes]? 
          Automatically restart the engine VM as a monitored service after engine-setup (Yes, No)[Yes]? 
          Enter root password that will be used for the engine appliance (leave it empty to skip): 
          Confirm appliance root password: 
          Enter ssh public key for the root user that will be used for the engine appliance (leave it empty to skip): 
[WARNING] Skipping appliance root ssh public key
          Do you want to enable ssh access for the root user (yes, no, without-password) [yes]: 
          Please specify the size of the VM disk in GB: [50]: 
          Please specify the memory size of the VM in MB (Defaults to maximum available): [4972]: 
          The following CPU types are supported by this host:
          	 - model_Broadwell: Intel Broadwell Family
          	 - model_Broadwell-noTSX: Intel Broadwell-noTSX Family
          	 - model_Haswell: Intel Haswell Family
          	 - model_Haswell-noTSX: Intel Haswell-noTSX Family
          	 - model_SandyBridge: Intel SandyBridge Family
          	 - model_Westmere: Intel Westmere Family
          	 - model_Nehalem: Intel Nehalem Family
          	 - model_Penryn: Intel Penryn Family
          	 - model_Conroe: Intel Conroe Family
          Please specify the CPU type to be used by the VM [model_Broadwell]: 
          Please specify the number of virtual CPUs for the VM (Defaults to appliance OVF value): [4]: 2
          You may specify a unicast MAC address for the VM or accept a randomly generated default [00:16:3e:7e:77:33]: 12:12:12:12:12:12
          How should the engine VM network be configured (DHCP, Static)[DHCP]? 
          Add lines for the appliance itself and for this host to /etc/hosts on the engine VM?
          Note: ensuring that this host could resolve the engine VM hostname is still up to you
          (Yes, No)[No]    
         
          --== HOSTED ENGINE CONFIGURATION ==--
         
          Enter engine admin password: 
          Confirm engine admin password: 
          Please provide the name of the SMTP server through which we will send notifications [localhost]: 
          Please provide the TCP port number of the SMTP server [25]: 
          Please provide the email address from which notifications will be sent [root@localhost]: 
          Please provide a comma-separated list of email addresses which will get notifications [root@localhost]: 
[ INFO  ] Stage: Setup validation
[WARNING] Cannot validate host name settings, reason: resolved host does not match any of the local addresses
         
          --== CONFIGURATION PREVIEW ==--
         
          Bridge interface                   : ens3
          Engine FQDN                        : she-engine.idodoroz.com
          Bridge name                        : ovirtmgmt
          Host address                       : host1.idodoroz.com
          SSH daemon port                    : 22
          Firewall manager                   : iptables
          Gateway address                    : 192.168.100.1
          Storage Domain type                : nfs3
          Image size GB                      : 50
          Host ID                            : 1
          Storage connection                 : 192.168.100.50:/var/nfs_folder
          Memory size MB                     : 4972
          Console type                       : vnc
          Number of CPUs                     : 2
          MAC address                        : 12:12:12:12:12:12
          OVF archive (for disk boot)        : /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-4.2-20171127.1.el7.centos.ova
          Appliance version                  : 4.2-20171127.1.el7.centos
          Restart engine VM after engine-setup: True
          Engine VM timezone                 : America/New_York
          CPU Type                           : model_Broadwell
         
          Please confirm installation settings (Yes, No)[Yes]: 
[ INFO  ] Stage: Transaction setup
[ INFO  ] Stage: Misc configuration
[ INFO  ] Stage: Package installation
[ INFO  ] Stage: Misc configuration
[ INFO  ] Configuring libvirt
[ INFO  ] Configuring VDSM
[ INFO  ] Starting vdsmd
[ INFO  ] Configuring the management bridge
[ INFO  ] Creating Storage Domain
[ INFO  ] Creating Storage Pool
[ INFO  ] Connecting Storage Pool
[ INFO  ] Verifying sanlock lockspace initialization
[ INFO  ] Creating Image for 'hosted-engine.lockspace' ...
[ INFO  ] Image for 'hosted-engine.lockspace' created successfully
[ INFO  ] Creating Image for 'hosted-engine.metadata' ...
[ INFO  ] Image for 'hosted-engine.metadata' created successfully
[ INFO  ] Creating VM Image
[ INFO  ] Extracting disk image from OVF archive (could take a few minutes depending on archive size)
[ INFO  ] Validating pre-allocated volume size
[ INFO  ] Uploading volume to data domain (could take a few minutes depending on archive size)
[ INFO  ] Image successfully imported from OVF
[ INFO  ] Destroying Storage Pool
[ INFO  ] Start monitoring domain
[ INFO  ] Configuring VM
[ INFO  ] Updating hosted-engine configuration
[ INFO  ] Stage: Transaction commit
[ INFO  ] Stage: Closing up
[ INFO  ] Creating VM
          You can now connect to the VM with the following command:
          	hosted-engine --console
          You can also graphically connect to the VM from your system with the following command:
          	remote-viewer vnc://host1.idodoroz.com:5900
          Use temporary password "3290JcOd" to connect to vnc console.
          Please ensure that your Guest OS is properly configured to support serial console according to your distro documentation.
          Follow http://www.ovirt.org/Serial_Console_Setup#I_need_to_access_the_console_the_old_way for more info.
          If you need to reboot the VM you will need to start it manually using the command:
          hosted-engine --vm-start
          You can then set a temporary password using the command:
          hosted-engine --add-console-password
[ INFO  ] Running engine-setup on the appliance
          |- [ INFO  ] Stage: Initializing
          |- [ INFO  ] Stage: Environment setup
          |-           Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/root/ovirt-engine-answers', '/root/heanswers.conf']
          |-           Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20171129152901-uh2kd4.log
          |-           Version: otopi-1.7.4_master (otopi-1.7.4-0.0.master.20171121144806.git3ef7ec5.el7.centos)
          |- [ INFO  ] Stage: Environment packages setup
          |- [ INFO  ] Stage: Programs detection
          |- [ INFO  ] Stage: Environment setup
          |- [ INFO  ] Stage: Environment customization
          |-          
          |-           --== PRODUCT OPTIONS ==--
          |-          
          |-           Configure Image I/O Proxy on this host? (Yes, No) [Yes]: 
          |-          
          |-           --== PACKAGES ==--
          |-          
          |-           Configure ovirt-provider-ovn (Yes, No) [Yes]: 
          |-          
          |-           --== NETWORK CONFIGURATION ==--
          |-          
          |- [ INFO  ] firewalld will be configured as firewall manager.
          |-          
          |-           --== DATABASE CONFIGURATION ==--
          |-          
          |-          
          |-           --== OVIRT ENGINE CONFIGURATION ==--
          |-          
          |-           Use default credentials (admin@internal) for ovirt-provider-ovn (Yes, No) [Yes]: 
          |-          
          |-           --== STORAGE CONFIGURATION ==--
          |-          
          |-          
          |-           --== PKI CONFIGURATION ==--
          |-          
          |-          
          |-           --== APACHE CONFIGURATION ==--
          |-          
          |-          
          |-           --== SYSTEM CONFIGURATION ==--
          |-          
          |-          
          |-           --== MISC CONFIGURATION ==--
          |-          
          |-           Please choose Data Warehouse sampling scale:
          |-           (1) Basic
          |-           (2) Full
          |-           (1, 2)[1]: 
          |-          
          |-           --== END OF CONFIGURATION ==--
          |-          
          |- [ INFO  ] Stage: Setup validation
          |- [WARNING] Less than 16384MB of memory is available
          |-          
          |-           --== CONFIGURATION PREVIEW ==--
          |-          
          |-           Application mode                        : both
          |-           Default SAN wipe after delete           : False
          |-           Firewall manager                        : firewalld
          |-           Update Firewall                         : True
          |-           Host FQDN                               : she-engine.idodoroz.com
          |-           Configure local Engine database         : True
          |-           Set application as default page         : True
          |-           Configure Apache SSL                    : True
          |-           Engine database secured connection      : False
          |-           Engine database user name               : engine
          |-           Engine database name                    : engine
          |-           Engine database host                    : localhost
          |-           Engine database port                    : 5432
          |-           Engine database host name validation    : False
          |-           Engine installation                     : True
          |-           PKI organization                        : idodoroz.com
          |-           Set up ovirt-provider-ovn               : True
          |-           Configure WebSocket Proxy               : True
          |-           DWH installation                        : True
          |-           DWH database secured connection         : False
          |-           DWH database host                       : localhost
          |-           DWH database user name                  : ovirt_engine_history
          |-           DWH database name                       : ovirt_engine_history
          |-           DWH database port                       : 5432
          |-           DWH database host name validation       : False
          |-           Configure local DWH database            : True
          |-           Configure Image I/O Proxy               : True
          |-           Configure VMConsole Proxy               : True
          |- [ INFO  ] Stage: Transaction setup
          |- [ INFO  ] Stopping engine service
          |- [ INFO  ] Stopping ovirt-fence-kdump-listener service
          |- [ INFO  ] Stopping dwh service
          |- [ INFO  ] Stopping Image I/O Proxy service
          |- [ INFO  ] Stopping vmconsole-proxy service
          |- [ INFO  ] Stopping websocket-proxy service
          |- [ INFO  ] Stage: Misc configuration
[ ERROR ] Engine setup got stuck on the appliance
[ ERROR ] Failed to execute stage 'Closing up': Engine setup is stalled on the appliance since 1800 seconds ago.
         Please check its log on the appliance.
         
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20171129105906.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue,fix and redeploy
          Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20171129102032-cob2nq.log