Bug 1152202

Summary: No route to rhel7 after successful host deploy
Product: Red Hat Enterprise Virtualization Manager Reporter: Ohad Basan <obasan>
Component: vdsmAssignee: Saggi Mizrahi <smizrahi>
Status: CLOSED NOTABUG QA Contact: Pavel Stehlik <pstehlik>
Severity: high Docs Contact:
Priority: high    
Version: 3.5.0CC: bazulay, dfediuck, ecohen, eedri, gklein, iheim, lpeer, lsurette, oourfali, pkliczew, yeylon
Target Milestone: ---Keywords: AutomationBlocker, AutomationTriaged, Triaged
Target Release: 3.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: infra
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-21 12:16:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1147536, 1164308, 1164311    

Description Ohad Basan 2014-10-13 14:55:15 UTC
Description of problem:
I have a setup of rhevm 3.5.
added a rhel7 host. host deploy succeeded and vdsm is up
selinux is set on permissive state. yet the engine can not seem to establish connection with the host despite the fact that the host is fully reachable.
I will attach all the relevant logs

2014-10-13 17:37:02,000 WARN  [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65]
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65]
        at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65]
        at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:]

Comment 3 Eyal Edri 2014-10-21 08:57:53 UTC
fails automation on el7 - adding to beta blocker.

Comment 4 Saggi Mizrahi 2014-10-21 12:16:41 UTC
We don't do routing. We just use the IP. If there is no route to host it's not because of anything related to jsonrpc.

Comment 5 Piotr Kliczewski 2014-10-21 12:45:50 UTC
Saggi is right that it is not related to jsonrpc. Together with Ohad we investigated the issue and after capturing the traffic we can see that the host response to any network frame sent to vdsm port is ICMP type 3 code 10 which is 
according to the iptables rule:

REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

My assumption is that we do not configure iptables properly during host deploy.

Comment 6 Piotr Kliczewski 2014-10-21 13:32:20 UTC
We have confirmation that host deploy did not configure iptables because it was told not to do it.