Description of problem:

There are situations which require the user to be able to configure a 
certain network validation test in a Hosted Engine deployment and/or 
a RHHI deployment instead to default to the only method today (ping 
default gw)

Security policies in organizations might refuse ICMP traffic on default 
gateway, this makes the validation during the deployment to fail and 
also will affect/low the score in the HA mechanism for the HE VM to be 
migrated to another HE Host in the cluster in case of a network failure

Version-Release number of selected component (if applicable):

Latest bits RHV 4.2.7

How reproducible:

100%

Steps to Reproduce:
1. Block ICMP traffic on default gateway
2. Deploy HostedEngine or RHHI


Actual results:

cockpit UI or `hosted-engine` CLI fails due to not being able to ping 
default gateway


Expected results:

Users who could not rely on ICMP traffic due to security constrains, 
should be able to configure different network test or to which IP the 
ping must be sent, an idea from Simone, instead of ping the def gw, one 
could configure DNS IPs for example or an array of well-know servers in 
the environment.


Additional info:

As a workaround, we could nat ICMP traffic or disable HA penalty due to
network issues, although, be aware that with both WAs, we are going to
ignore network issues. In that case, the HA logic to migrate the 
HostedEngine VM to another HE Host in the cluster might not work as 
expected


Nat ICMP traffic, this will be needed for initial deployment

# Create an iptables rule on the hypervisors that would redirect the icmp traffic back to localhost:
iptables -t nat -A OUTPUT -p icmp -d @gateway-ip@ -j DNAT --to-destination 127.0.0.1


Setting gateway penalty to 0, this won't help during initial deployment, 
but will be useful after it, to avoid wrong score calculation in the HA 
mechanism

# vi /etc/ovirt-hosted-engine-ha/agent.conf 
gateway-score-penalty=0