Bug 1477587 - Upgrade incompatibility: Can no longer mix IPv4/IPv6 in virtual_ipaddress when using IPv6 VRRP instance
Upgrade incompatibility: Can no longer mix IPv4/IPv6 in virtual_ipaddress whe...
Status: NEW
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: keepalived (Show other bugs)
7.4
x86_64 Linux
unspecified Severity medium
: rc
: ---
Assigned To: Ryan O'Hara
Brandon Perkins
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-08-02 08:26 EDT by Robert Scheck
Modified: 2017-08-04 02:25 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Robert Scheck 2017-08-02 08:26:58 EDT
Description of problem:
Since updating from RHEL 7.3 to 7.4 (thus keepalived 1.2.x to 1.3.x), the
following configuration does no longer work:

-- snipp --
  native_ipv6
  unicast_src_ip 2001:db8::1
  unicast_peer {
    2001:db8::2
  }
  virtual_ipaddress {
    192.0.2.1/30 dev bond1.1000
    fe80::1/64 dev bond1.1000
    2001:db8:0:1000::1/64 dev bond1.1000
    192.0.2.250/29 dev bond0
    2001:db8:0:4003::2/64 dev bond0
  }
-- snapp --

Above leads to the following log output:

Aug  2 14:23:19 tux1 Keepalived_vrrp[7664]: (VRRP_INSTANCE): address family must match VRRP instance [192.0.2.1/30] - ignoring
Aug  2 14:23:19 tux1 Keepalived_vrrp[7664]: (VRRP_INSTANCE): address family must match VRRP instance [192.0.2.250/29] - ignoring

And this again leads to the fact that no IPv4 addresses are set up to bond0,
while this worked properly with keepalived 1.2.x before.

The important point is that our keepalived-internal communication happens
only via IPv6, while keepalived shall maintain IPv4 and IPv6 floating IPs;
to get this working "native_ipv6" was required for keepalived 1.2.x, but it
does not make any difference for 1.3.x if that option is (still) set or not.

Version-Release number of selected component (if applicable):
keepalived-1.3.5-1.el7.x86_64

How reproducible:
Everytime, see above.

Actual results:
virtual_ipaddress no longer accepts IPv4 addresses for IPv6 VRRP instance.

Expected results:
Working stuff with keepalived 1.3.x, like it was in 1.2.x.

Additional info:
This feels like a incompatibility/regression and should IMHO not happen
within a stable RHEL release.
Comment 2 Robert Scheck 2017-08-02 09:32:46 EDT
Cross-filed ticket 01903060 on the Red Hat customer portal.
Comment 3 Robert Scheck 2017-08-02 16:33:08 EDT
Configuration below (minus enable_script_security) works with keepalived
1.2.x, but not with 1.3.x (leads to errors mentioned before), thus all the
IPv4 addresses are not set up (obfuscated 192.0.2.1/30, 192.0.2.250/29):

--- snipp ---
global_defs {
    router_id tux1
    enable_script_security  # Keepalived yells about scripts?!
#    script_user root root  # Keepalived yells anyway?! RHBZ#1477563
    vrrp_iptables  # Empty to avoid iptables rules
#    vrrp_ipset  # Empty to avoid ipsets; does not work, RHBZ#1477572
#    vrrp_version 2  # tux2 still believes this is VRRP 3, RHBZ#1477552
}

vrrp_sync_group VRRP_GROUP {
    group {
        VRRP_INSTANCE
    }
    notify_master "/etc/conntrackd/primary-backup.sh primary"
    notify_backup "/etc/conntrackd/primary-backup.sh backup"
    notify_fault "/etc/conntrackd/primary-backup.sh fault"
}

vrrp_instance VRRP_INSTANCE {
    interface em2
    state BACKUP
    virtual_router_id 51
    priority 150
    track_interface {
        bond0
        bond1
    }
    native_ipv6  # keepalived 1.2.x hates IPv6 unicast_* w/o this option?!
    unicast_src_ip 2001:db8::1
    unicast_peer {
        2001:db8::2
    }
    virtual_ipaddress {
        192.0.2.1/30 dev bond1.1000
        fe80::1/64 dev bond1.1000
        2001:db8:0:1000::1/64 dev bond1.1000
        192.0.2.250/29 dev bond0
        2001:db8:0:4003::2/64 dev bond0
    }
    virtual_routes {
        blackhole 192.0.2.0/24
        blackhole 2001:db8::/32
    }
    advert_int 1
    nopreempt
    garp_master_delay 0
    dont_track_primary
}
--- snapp ---
Comment 4 Robert Scheck 2017-08-03 11:42:33 EDT
Bug #1477552 comment #8 lead to findings that also apply here:

https://github.com/acassen/keepalived/commit/485847cd30503c1ec2370713c2593a2216f19bb1#diff-bb37771a5dd629fb6332c05768e92a95R1606

keepalived-1.2.13-9.el7_3.x86_64 (RHEL 7.3) allowed this:

--- snipp ---
vrrp_instance VRRP_INSTANCE {
    # …
    native_ipv6
    unicast_src_ip 2001:db8::1
    unicast_peer {
        2001:db8::2
    }
    virtual_ipaddress {
        192.0.2.1/30 dev bond1.1000
        fe80::1/64 dev bond1.1000
        2001:db8:0:1000::1/64 dev bond1.1000
        192.0.2.250/29 dev bond0
        2001:db8:0:4003::2/64 dev bond0
    }
}
--- snipp ---

Using keepalived-1.3.5-1.el7.x86_64 (RHEL 7.4), the following happens and
applies:

- Keepalived in 1.3.5-1.el7 no longer allows mixing IPv4 and IPv6 addresses
  in virtual_ipaddress section
- In this specific case the inter-keepalived-communication is IPv6, thus
  IPv4 addresses can't be put into virtual_ipaddress section

It is NOT possible to make the configuration above working with keepalived
1.2.x AND 1.3.x, because of further differences mentioned in bug #1477552.

Any upgrade, when having IPv6 for inter-keepalived-communication, requires
a configuration change when upgrading from keepalived 1.2.x to 1.3.x. It is
not possible to run a 1.2.x and 1.3.x mixed keepalived cluster when having
IPv6 for inter-keepalived-communication.

Above configuration needs to be rewritten for keepalived-1.3.5-1.el7.x86_64 
(RHEL 7.4) like this:

--- snipp ---
vrrp_sync_group vrrp_group {
    # …
    group {
        vrrp_ipv4
        vrrp_ipv6
    }
}

vrrp_instance vrrp_ipv4 {
    # …
    unicast_src_ip 192.0.2.5
    unicast_peer {
        192.0.2.6
    }
    virtual_ipaddress {
        192.0.2.1/30 dev bond1.1000
        192.0.2.250/29 dev bond0
    }
}

vrrp_instance vrrp_ipv6 {
    # …
    unicast_src_ip 2001:db8::1
    unicast_peer {
        2001:db8::2
    }
    virtual_ipaddress {
        fe80::1/64 dev bond1.1000
        2001:db8:0:1000::1/64 dev bond1.1000
        2001:db8:0:4003::2/64 dev bond0
    }
}
--- snipp ---

Please update the RHEL 7.4 release notes to reflect these findings to help
other customers about this keepalived upstream incompatibility when upgrading
from RHEL 7.3.
Comment 5 Ryan O'Hara 2017-08-03 12:20:42 EDT
Just curious, in the configuration shown where you have separate VRRP instances (vrrp_ipv4 and vrrp_ipv6), do either have vrrp_version or native_ipv6 set?
Comment 6 Robert Scheck 2017-08-03 12:25:01 EDT
(In reply to Ryan O'Hara from comment #5)
> Just curious, in the configuration shown where you have separate VRRP
> instances (vrrp_ipv4 and vrrp_ipv6), do either have vrrp_version or
> native_ipv6 set?

No and no. And to be more verbose:

- Keyword "native_ipv6" is ignored according to keepalived 1.3.x source
- VRRP instance vrrp_ipv4 uses VRRP 2 (tested/verified)
- VRRP instance vrrp_ipv6 uses VRRP 3 (tested/verified)
- When setting "vrrp_version 3", both instances are using VRRP 3 (also
  tested/verified)

Note You need to log in before you can comment on or make changes to this bug.