Bug 1400640

Summary: Hosted-engine deploy on a rebuilt server
Product: [oVirt] ovirt-host-deploy Reporter: Matheus Gomes <itsupport>
Component: GeneralAssignee: Sahina Bose <sabose>
Status: CLOSED DEFERRED QA Contact: Pavel Stehlik <pstehlik>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.5.2CC: bugs, itsupport, oourfali, ylavi
Target Milestone: ---Flags: rule-engine: ovirt-4.1+
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-06 14:32:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Gluster RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Ovirt Log none

Description Matheus Gomes 2016-12-01 16:59:27 UTC
Created attachment 1226909 [details]
Ovirt Log

Description of problem:

We had a hardware failure on one of our nodes. we have a Gluster Bricks: 2 x (2 + 1) = 6 setup.

host1
host2
arb1
host3
host4
arb2

We initially had all servers up and running as it should. HW failure happened and I've rebuild the server with same IP/FQDN to keep the same setup with Gluster and backup-volfile-servers IP's

After running hosted-engine --deploy:

[ ERROR ] Cannot automatically add the host to cluster Default: Cannot add Host. Host with the same address already exists.

The GUI has no control over this server, cannot put into maintenance mode or reinstall.

Documentation for this sort of issue is unavailable


Version-Release number of selected component (if applicable):

    Dep-Install pulseaudio-libs-6.0-7.el7.x86_64                           
@base
    Dep-Install pykickstart-1.99.66.6-1.el7.noarch                         @base
    Dep-Install pyparted-1:3.9-13.el7.x86_64                               @base
    Dep-Install python-IPy-0.75-6.el7.noarch                               @base
    Dep-Install python-backports-1.0-8.el7.x86_64                          @base
    Dep-Install python-backports-ssl_match_hostname-3.4.0.2-4.el7.noarch   @base
    Dep-Install python-blivet-1:0.61.15.37-1.el7.noarch                    @base
    Dep-Install python-cpopen-1.5-3.el7.x86_64                             @centos-ovirt40-release
    Dep-Install python-daemon-1.6-4.el7.noarch                             @centos-ovirt40-release
    Dep-Install python-dmidecode-3.10.13-11.el7.x86_64                     @base
    Dep-Install python-ethtool-0.8-5.el7.x86_64                            @base
    Dep-Install python-inotify-0.9.4-4.el7.noarch                          @base
    Dep-Install python-ioprocess-0.16.1-1.el7.noarch                       @centos-ovirt40-release
    Dep-Install python-libguestfs-1:1.28.1-1.55.el7.centos.4.x86_64        @updates
    Dep-Install python-lockfile-1:0.9.1-4.el7.centos.noarch                @extras
    Dep-Install python-lxml-3.2.1-4.el7.x86_64                             @base
    Dep-Install python-magic-5.11-31.el7.noarch                            @base
    Dep-Install python-netaddr-0.7.5-7.el7.noarch                          @base
    Dep-Install python-pthreading-0.1.3-3.el7.noarch                       @centos-ovirt40-release
    Dep-Install python-pyblock-0.53-6.el7.x86_64                           @base
    Dep-Install python-requests-2.6.0-1.el7_1.noarch                       @base
    Dep-Install python-suds-0.4.1-5.el7.noarch                             @base
    Dep-Install python-urllib3-1.10.2-2.el7_1.noarch                       @base
    Dep-Install python-webob-1.2.3-6.el7.noarch                            @base
    Dep-Install qemu-img-ev-10:2.3.0-31.el7.16.1.x86_64                    @ovirt-4.0
    Dep-Install qemu-kvm-common-ev-10:2.3.0-31.el7.16.1.x86_64             @ovirt-4.0
    Dep-Install qemu-kvm-ev-10:2.3.0-31.el7.16.1.x86_64                    @ovirt-4.0
    Dep-Install qemu-kvm-tools-ev-10:2.3.0-31.el7.16.1.x86_64              @ovirt-4.0
    Dep-Install radvd-1.9.2-9.el7.x86_64                                   @base
    Dep-Install safelease-1.0-7.el7.x86_64                                 @centos-ovirt40-release
    Dep-Install sanlock-3.2.4-2.el7_2.x86_64                               @updates
    Dep-Install sanlock-lib-3.2.4-2.el7_2.x86_64                           @updates
    Dep-Install sanlock-python-3.2.4-2.el7_2.x86_64                        @updates
    Install     screen-4.1.0-0.23.20120314git3c2946.el7_2.x86_64           @updates
    Dep-Install scrub-2.5.2-5.el7.x86_64                                   @base
    Dep-Install seabios-bin-1.7.5-11.el7.noarch                            @base
    Dep-Install seavgabios-bin-1.7.5-11.el7.noarch                         @base
    Dep-Install setools-libs-3.3.7-46.el7.x86_64                           @base
    Dep-Install sg3_utils-1.37-5.el7.x86_64                                @base
    Dep-Install sg3_utils-libs-1.37-5.el7.x86_64                           @base
    Dep-Install sgabios-bin-1:0.20110622svn-4.el7.noarch                   @base
    Dep-Install sgpio-1.2.0.10-13.el7.x86_64                               @base
    Dep-Install socat-1.7.2.4-1.el6.rf.x86_64                              @rpmforge
    Dep-Install sos-3.2-35.el7.centos.3.noarch                             @updates
    Dep-Install spice-server-0.12.4-15.el7_2.2.x86_64                      @updates
    Dep-Install supermin5-5.1.10-1.2.el7.x86_64                            @base
    Dep-Install syslinux-4.05-12.el7.x86_64                                @base
    Dep-Install syslinux-extlinux-4.05-12.el7.x86_64                       @base
    Dep-Install systemd-python-219-19.el7_2.13.x86_64                      @updates
    Dep-Install tree-1.6.0-10.el7.x86_64                                   @base
    Dep-Install unbound-libs-1.4.20-26.el7.x86_64                          @base
    Dep-Install usbredir-0.6-7.el7.x86_64                                  @base
    Install     vdsm-4.18.15.3-1.el7.centos.x86_64                         @ovirt-4.0
    Dep-Install vdsm-api-4.18.15.3-1.el7.centos.noarch                     @ovirt-4.0
    Dep-Install vdsm-cli-4.18.15.3-1.el7.centos.noarch                     @ovirt-4.0
    Install     vdsm-gluster-4.18.15.3-1.el7.centos.noarch                 @ovirt-4.0
    Dep-Install vdsm-hook-vmfex-dev-4.18.15.3-1.el7.centos.noarch          @ovirt-4.0
    Dep-Install vdsm-infra-4.18.15.3-1.el7.centos.noarch                   @ovirt-4.0
    Dep-Install vdsm-jsonrpc-4.18.15.3-1.el7.centos.noarch                 @ovirt-4.0
    Dep-Install vdsm-python-4.18.15.3-1.el7.centos.noarch                  @ovirt-4.0
    Dep-Install vdsm-xmlrpc-4.18.15.3-1.el7.centos.noarch                  @ovirt-4.0
    Dep-Install vdsm-yajsonrpc-4.18.15.3-1.el7.centos.noarch               @ovirt-4.0
    Dep-Install virt-v2v-1:1.28.1-1.55.el7.centos.4.x86_64                 @updates



How reproducible
Chances of hardware failure and re-installation is high, glad we got this right before it was in full production.

Steps to Reproduce:
1.Deploy Gluster+Ovirt on cluster
2.Lose a cluster node
3.Reinstall cluster node

Actual results:
[ ERROR ] Cannot automatically add the host to cluster Default: Cannot add Host. Host with the same address already exists.

Expected results:
To succeed


Additional info:

Comment 1 Sahina Bose 2016-12-27 05:50:18 UTC
"The GUI has no control over this server, cannot put into maintenance mode or reinstall."
In such cases of hardware failure, the node needs to be moved to maintenance, and reinstalled from GUI once hardware failure is fixed. Is there an error while moving this host to maintenance from GUI? Can you provide details?

Comment 2 Matheus Gomes 2017-02-06 14:32:34 UTC
The workaround for allowing a downed host to be replaced with same name and ip's is to manually delete the host from Database, only then will it allow to place the node under maintenance mode and be removed via GUI.