Bug 1350883 - vdscli.connect's heuristic ends up reading the local server address from vdsm config, where it finds the default ipv6-local address of "::".
Summary: vdscli.connect's heuristic ends up reading the local server address from vdsm...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: General
Version: 4.18.4
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ovirt-4.0.2
: 4.18.9
Assignee: Edward Haas
QA Contact: Michael Burman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-28 15:16 UTC by HWSD
Modified: 2016-08-12 14:25 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-08-12 14:25:29 UTC
oVirt Team: Network
Embargoed:
rule-engine: ovirt-4.0.z+
rule-engine: exception+
rule-engine: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1358530 0 high CLOSED jsonrpcclient fails connecting with the default parameters if the hostname is not resolvable 2022-08-15 13:23:04 UTC
oVirt gerrit 60020 0 master MERGED vdscli: Replace the default '::' host with hostname 2016-07-26 05:45:57 UTC
oVirt gerrit 61363 0 ovirt-4.0 MERGED vdscli: Replace the default '::' host with hostname 2016-07-26 10:08:04 UTC

Internal Links: 1358530

Description HWSD 2016-06-28 15:16:41 UTC
After upgrade from oVirt 3.6 to 4.0 ha broker is not able to connect to storage backend.

vdscli.connect's heuristic ends up reading the local
server address from vdsm config, where it finds the default ipv6-local
address of "::" but IPV6 is disabled.

More details on ovirt-users ml:

http://lists.ovirt.org/pipermail/users/2016-June/040578.html

Comment 1 HWSD 2016-06-28 15:51:35 UTC
[root@ovirt01 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
;vdsmdummy;             8000.000000000000       no
DMZ             8000.40167e3684f1       no              bond0.3
                                                        vnet1
                                                        vnet2
LAN_HAW         8000.40167e3684f1       no              bond0.1
ovirtmgmt               8000.00a1b08b2298       no              enp6s1
                                                        vnet0

[root@ovirt01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
3: enp6s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovirtmgmt state UP qlen 1000
    link/ether 00:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
4: enp4s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
5: enp5s0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
6: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 00:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.50/24 brd 192.168.1.255 scope global ovirtmgmt
       valid_lft forever preferred_lft forever
7: bond0.1@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master LAN_HAW state UP
    link/ether 40:**:**:**:**:** f1 brd ff:ff:ff:ff:ff:ff
8: LAN_HAW: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
9: bond0.3@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master DMZ state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
10: DMZ: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 40:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
11: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether ca:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
12: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovirtmgmt state UNKNOWN qlen 500
    link/ether fe:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
13: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master DMZ state UNKNOWN qlen 500
    link/ether fe:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
14: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master DMZ state UNKNOWN qlen 500
    link/ether fe:**:**:**:**:** brd ff:ff:ff:ff:ff:ff


[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/bonds/bond0
{
    "nics": [
        "enp4s0",
        "enp5s0"
    ],
    "options": "mode=2 miimon=100"


[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/nets/DMZ
{
    "bonding": "bond0",
    "vlan": "3",
    "STP": "no",
    "bridged": "true",
    "mtu": "1500"
}

[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/nets/LAN_HAW
{
    "bonding": "bond0",
    "vlan": "1",
    "STP": "no",
    "bridged": "true",
    "mtu": "1500"
}

[root@ovirt01 ~]# cat /var/lib/vdsm/persistence/netconf/nets/ovirtmgmt
{
    "nic": "enp6s1",
    "netmask": "255.255.255.0",
    "bootproto": "none",
    "ipaddr": "192.168.1.50",
    "gateway": "192.168.1.254"
}

Comment 2 Edward Haas 2016-06-29 10:45:17 UTC
The loopback interface has IPv6 disabled.
The client uses the address mentioned in vdsm.conf (by default the '::'), but no ipv6 addresses are available on the host, therefore it will fail to establish a connection.

Enabling IPv6 on the lo iface should resolve the issue.

Comment 3 Dan Kenigsberg 2016-06-29 11:36:03 UTC
HWSD, have you done something in particular to disable inet6 on the loopback device? What's your /proc/sys/net/ipv6/conf/default/disable_ipv6 ?

Comment 4 HWSD 2016-06-29 11:56:20 UTC
I don't remember if I did something to disable IPV6 during host setup (last year...).

Enabling inet6 ONLY on lo interface don't solve.
I have to enable inet6 on lo and ovirtmgmt.

[root@ovirt01 ~]# cat /proc/sys/net/ipv6/conf/default/disable_ipv6
1

Comment 5 Edward Haas 2016-06-29 12:05:02 UTC
Have you restarted the vdsmd service? The server needs to listen on IPv6.

Comment 6 HWSD 2016-06-29 12:18:30 UTC
I restarted vdsmd and the host.... I have to manually enable ipv6 on lo and ovirtmgmt.
oVirt manage the network configuration.

I have a crontab hack:

[root@ovirt01 etc]# crontab -l
@reboot echo 'echo 0 > /proc/sys/net/ipv6/conf/lo/disable_ipv6' | /usr/bin/at now+1 minutes
@reboot echo 'echo 0 > /proc/sys/net/ipv6/conf/ovirtmgmt/disable_ipv6' | /usr/bin/at now+1 minutes
@reboot echo '/usr/sbin/route add default gw 192.168.1.254'  | /usr/bin/at now+1 minutes


When atd execute the command all start to work. Whitout any restart.

Comment 7 Michael Burman 2016-07-28 10:31:20 UTC
Verified on - vdsm-4.18.9-1.el7ev.x86_64


Note You need to log in before you can comment on or make changes to this bug.