Bug 1350883 - vdscli.connect's heuristic ends up reading the local server address from vdsm config, where it finds the default ipv6-local address of "::".
vdscli.connect's heuristic ends up reading the local server address from vdsm...
Status: CLOSED CURRENTRELEASE
Product: vdsm
Classification: oVirt
Component: General (Show other bugs)
4.18.4
Unspecified Unspecified
high Severity medium (vote)
: ovirt-4.0.2
: 4.18.9
Assigned To: Edward Haas
Michael Burman
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-06-28 11:16 EDT by HWSD
Modified: 2016-08-12 10:25 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-08-12 10:25:29 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.0.z+
rule-engine: exception+
rule-engine: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 60020 master MERGED vdscli: Replace the default '::' host with hostname 2016-07-26 01:45 EDT
oVirt gerrit 61363 ovirt-4.0 MERGED vdscli: Replace the default '::' host with hostname 2016-07-26 06:08 EDT

  None (edit)
Description HWSD 2016-06-28 11:16:41 EDT
After upgrade from oVirt 3.6 to 4.0 ha broker is not able to connect to storage backend.

vdscli.connect's heuristic ends up reading the local
server address from vdsm config, where it finds the default ipv6-local
address of "::" but IPV6 is disabled.

More details on ovirt-users ml:

http://lists.ovirt.org/pipermail/users/2016-June/040578.html
Comment 1 HWSD 2016-06-28 11:51:35 EDT
[root@ovirt01 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
;vdsmdummy;             8000.000000000000       no
DMZ             8000.40167e3684f1       no              bond0.3
                                                        vnet1
                                                        vnet2
LAN_HAW         8000.40167e3684f1       no              bond0.1
ovirtmgmt               8000.00a1b08b2298       no              enp6s1
                                                        vnet0

[root@ovirt01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
3: enp6s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovirtmgmt state UP qlen 1000
    link/ether 00:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
4: enp4s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
5: enp5s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
6: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 00:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.50/24 brd 192.168.1.255 scope global ovirtmgmt
       valid_lft forever preferred_lft forever
7: bond0.1@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master LAN_HAW state UP
    link/ether 40:**:**:**:**:** f1 brd ff:ff:ff:ff:ff:ff
8: LAN_HAW: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
9: bond0.3@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master DMZ state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
10: DMZ: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
11: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether ca:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
12: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovirtmgmt state UNKNOWN qlen 500
    link/ether fe:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
13: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master DMZ state UNKNOWN qlen 500
    link/ether fe:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
14: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master DMZ state UNKNOWN qlen 500
    link/ether fe:**:**:**:**:** brd ff:ff:ff:ff:ff:ff


[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/bonds/bond0
{
    "nics": [
        "enp4s0",
        "enp5s0"
    ],
    "options": "mode=2 miimon=100"


[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/nets/DMZ
{
    "bonding": "bond0",
    "vlan": "3",
    "STP": "no",
    "bridged": "true",
    "mtu": "1500"
}

[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/nets/LAN_HAW
{
    "bonding": "bond0",
    "vlan": "1",
    "STP": "no",
    "bridged": "true",
    "mtu": "1500"
}

[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/nets/ovirtmgmt
{
    "nic": "enp6s1",
    "netmask": "255.255.255.0",
    "bootproto": "none",
    "ipaddr": "192.168.1.50",
    "gateway": "192.168.1.254"
}
Comment 2 Edward Haas 2016-06-29 06:45:17 EDT
The loopback interface has IPv6 disabled.
The client uses the address mentioned in vdsm.conf (by default the '::'), but no ipv6 addresses are available on the host, therefore it will fail to establish a connection.

Enabling IPv6 on the lo iface should resolve the issue.
Comment 3 Dan Kenigsberg 2016-06-29 07:36:03 EDT
HWSD, have you done something in particular to disable inet6 on the loopback device? What's your /proc/sys/net/ipv6/conf/default/disable_ipv6 ?
Comment 4 HWSD 2016-06-29 07:56:20 EDT
I don't remember if I did something to disable IPV6 during host setup (last year...).

Enabling inet6 ONLY on lo interface don't solve.
I have to enable inet6 on lo and ovirtmgmt.

[root@ovirt01 ~]# cat /proc/sys/net/ipv6/conf/default/disable_ipv6
1
Comment 5 Edward Haas 2016-06-29 08:05:02 EDT
Have you restarted the vdsmd service? The server needs to listen on IPv6.
Comment 6 HWSD 2016-06-29 08:18:30 EDT
I restarted vdsmd and the host.... I have to manually enable ipv6 on lo and ovirtmgmt.
oVirt manage the network configuration.

I have a crontab hack:

[root@ovirt01 etc]# crontab -l
@reboot echo 'echo 0 > /proc/sys/net/ipv6/conf/lo/disable_ipv6' | /usr/bin/at now+1 minutes
@reboot echo 'echo 0 > /proc/sys/net/ipv6/conf/ovirtmgmt/disable_ipv6' | /usr/bin/at now+1 minutes
@reboot echo '/usr/sbin/route add default gw 192.168.1.254'  | /usr/bin/at now+1 minutes


When atd execute the command all start to work. Whitout any restart.
Comment 7 Michael Burman 2016-07-28 06:31:20 EDT
Verified on - vdsm-4.18.9-1.el7ev.x86_64

Note You need to log in before you can comment on or make changes to this bug.