Bug 1582836

Summary: [RFE][TestOnly] - Certify Skydive with RHV 4.3 and the oVirt OVN provider
Product: Red Hat Enterprise Virtualization Manager Reporter: Yaniv Lavi <ylavi>
Component: ovirt-provider-ovnAssignee: Marcin Mirecki <mmirecki>
Status: CLOSED CURRENTRELEASE QA Contact: Dominik Holler <dholler>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.2.0CC: danken, dholler, mburman, myakove, nplanel, safchain
Target Milestone: ovirt-4.3.0Keywords: FutureFeature, TestOnly
Target Release: ---Flags: mburman: testing_plan_complete-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-08 15:11:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1568448, 1669567    
Bug Blocks: 994170, 1647203    

Description Yaniv Lavi 2018-05-27 08:59:04 UTC
Description of problem:
Skydive is provided with OSP for real-time network topology and protocols analyzer for complex SDN environments.
Certify Skydive with RHV and the oVirt OVN provider to provide the same capabilities on RHV.

Comment 1 Yaniv Lavi 2018-05-27 09:00:56 UTC
Ansible playbook that deploy analyzers and agents:
http://skydive.network/documentation/deployment#ansible

Comment 2 safchain 2018-05-28 13:22:11 UTC
Here a fix that address etcd server list generation issue when using skydive_listen_ip=0.0.0.0
https://github.com/skydive-project/skydive/pull/1057

Comment 3 Meni Yakove 2018-05-29 09:01:09 UTC
Still have the same issue:
May 29 11:57:10 network-ge-1.scl.lab.tlv.redhat.com skydive[615]: 2018-05-29T11:57:10.383+0300        ERROR        analyzer/server.go:222 NewServerFromConfig        network network-ge-1: Etcd server not ready: client: etcd cluster is unavailable or misconfigured;client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 10.35.161.250:12379: getsockopt: connection refused


/etc/skydive/skydive.yml
etcd:
  embedded: true
  listen: 0.0.0.0:12379
  peers:
    network-ge-1: http://10.35.161.250:12380
  servers:
  - http://10.35.161.250:12379


netstat:
tcp        0      0 127.0.0.1:12379         0.0.0.0:*               LISTEN      615/skydive         
tcp        0      0 127.0.0.1:12380         0.0.0.0:*               LISTEN      615/skydive


curl 10.35.161.250:12379
curl: (7) Failed connect to 10.35.161.250:12379; Connection refused

iptables is not running.

Comment 4 Meni Yakove 2018-05-29 09:01:35 UTC
Still have the same issue:
May 29 11:57:10 network-ge-1.scl.lab.tlv.redhat.com skydive[615]: 2018-05-29T11:57:10.383+0300        ERROR        analyzer/server.go:222 NewServerFromConfig        network network-ge-1: Etcd server not ready: client: etcd cluster is unavailable or misconfigured;client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 10.35.161.250:12379: getsockopt: connection refused


/etc/skydive/skydive.yml
etcd:
  embedded: true
  listen: 0.0.0.0:12379
  peers:
    network-ge-1: http://10.35.161.250:12380
  servers:
  - http://10.35.161.250:12379


netstat:
tcp        0      0 127.0.0.1:12379         0.0.0.0:*               LISTEN      615/skydive         
tcp        0      0 127.0.0.1:12380         0.0.0.0:*               LISTEN      615/skydive


curl 10.35.161.250:12379
curl: (7) Failed connect to 10.35.161.250:12379; Connection refused

iptables is not running.

Comment 5 safchain 2018-05-29 09:47:54 UTC
can you give the skydive version used:
$ skydive version

As the "listen" parameter is set to "0.0.0.0:12379" and the peers is set too, you should have in your netstat result 2 entries per IP present in you host. 

Is selinux enabled and blocking the port binding ?

Comment 6 Meni Yakove 2018-05-29 11:01:11 UTC
For the version I took this PR  https://github.com/skydive-project/skydive/pull/1057

git fetch origin pull/1057/head:etcd_fix

After disable selinux I can open the webUI.

How can I make it works with selinux?

Comment 7 Meni Yakove 2018-05-29 11:01:12 UTC
For the version I took this PR  https://github.com/skydive-project/skydive/pull/1057

git fetch origin pull/1057/head:etcd_fix

After disable selinux I can open the webUI.

How can I make it works with selinux?

Comment 8 Nicolas PLANEL 2018-05-30 05:28:42 UTC
@Meni : could you share 
 1/ the skydive configuration /etc/skydive/skydive.yml
 2/ grep skydive /var/log/audit/audit.log



I don't known what's differ from your setup, but here my working test :

1/ I did install a centos with selinux enabled by default
2/ deploy with ansible from the PR 1057
3/ edit skydive configuration file /etc/skydive/skydive.yml
   o set 0.0.0.0:8082   for analyzer.listen
   o set 0.0.0.0:12379  for etcd.listen


curl 192.168.0.7:12379
curl 192.168.0.7:8082

working find with selinux enabled (enforcing)


[root@centos7-1804 centos]# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 192.168.0.7:12380       0.0.0.0:*               LISTEN      12357/skydive       
tcp        0      0 127.0.0.1:12380         0.0.0.0:*               LISTEN      12357/skydive       
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      505/rpcbind         
tcp        0      0 127.0.0.1:8081          0.0.0.0:*               LISTEN      10637/skydive       
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1386/sshd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      999/master          
tcp        0      0 192.168.0.7:12379       0.0.0.0:*               LISTEN      12357/skydive       
tcp        0      0 127.0.0.1:12379         0.0.0.0:*               LISTEN      12357/skydive       
tcp6       0      0 ::1:12380               :::*                    LISTEN      12357/skydive       
tcp6       0      0 :::111                  :::*                    LISTEN      505/rpcbind         
tcp6       0      0 :::8082                 :::*                    LISTEN      12357/skydive       
tcp6       0      0 :::22                   :::*                    LISTEN      1386/sshd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      999/master          
tcp6       0      0 ::1:12379               :::*                    LISTEN      12357/skydive       
[root@centos7-1804 centos]# sestatus 
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      31

Comment 9 Meni Yakove 2018-05-30 07:47:29 UTC
I don't know what changed but now it's working with selinux on, so for now, we are good.

Comment 10 Nicolas PLANEL 2018-06-04 02:54:49 UTC
SELinux in Skydive, you call follow the PR here :
https://github.com/skydive-project/skydive/pull/1068

Comment 12 Dan Kenigsberg 2018-12-01 19:27:28 UTC
while verifying this, please try verifying

Bug 994170 - [RFE] [SkyDive] Ability to test network connectivity to logical networks on a host

Comment 14 Dominik Holler 2019-01-25 17:32:23 UTC
Final verification with public OSP14 repos is still missing.