Bug 1232408 - Keepalived version 1.2.17 breaks multiple vrrp_instance configs
Summary: Keepalived version 1.2.17 breaks multiple vrrp_instance configs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: keepalived
Version: 21
Hardware: Unspecified
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Ryan O'Hara
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-16 16:57 UTC by Dan Prince
Modified: 2015-07-14 15:43 UTC (History)
4 users (show)

Fixed In Version: keepalived-1.2.18-1.fc22
Clone Of:
Environment:
Last Closed: 2015-07-14 15:42:03 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Dan Prince 2015-06-16 16:57:35 UTC
Description of problem:

After the recent keepalived 1.2.17 package update virtual IPs are no longer available. Specifically a config file which contains multiple vrrp_instance declarations ends up running only one of them:

vrrp_instance 51 {
  virtual_router_id 51

  advert_int 1

  priority  101
  state     MASTER

  interface br-ex

  virtual_ipaddress {

      192.0.2.19 dev br-ex
  }

  track_script {
  haproxy
  }
}

vrrp_instance 52 {
  virtual_router_id 52

  advert_int 1

  priority  101
  state     MASTER

  interface br-ex

  virtual_ipaddress {
      192.0.2.18 dev br-ex
  }

  track_script {
  haproxy
  }

}

After running this configuration only the second vrrp_instance is up (192.0.2.18). The first IP address is unaviable.


Version-Release number of selected component (if applicable):

keepalived-1.2.17-2.fc21.x86_64

How reproducible:

Always.


Steps to Reproduce:
1. Install keepalived-1.2.17-2.fc21.x86_64
2. Create a config file with more than one vrrp_instance (possibly using the same device).
3. Try to ping both IPs. One will fail.


Additional info:

This update broke the upstream TripleO CI system which is based on Fedora 21 (stable). We've temporarily reverted to use the keepalived 1.2.16 package.

https://review.openstack.org/#/c/191536/

Simply reverting to the previous keepalived package fixes all functional issues with keepalived.

Comment 1 Dan Prince 2015-06-16 16:58:37 UTC
Related upstream TripleO bug where we initially noticed the keepalived regression:

https://bugs.launchpad.net/tripleo/+bug/1464719

Comment 2 Ryan O'Hara 2015-06-16 17:44:48 UTC
Could you please attach the complete keepalived.conf file? Thanks.

Comment 3 Dan Prince 2015-06-16 18:18:21 UTC
Sure. Here is a more complete keepalived.conf example:

# This file is managed by Puppet. DO NOT EDIT.
global_defs {
  notification_email {
    root@localdomain
  }
  notification_email_from keepalived@localdomain
  smtp_server localhost
  smtp_connect_timeout 30
  router_id ov-evncrqmcfls-0-kgow3lvx7iky-controller-hitezil32t2z
}

static_ipaddress {

}

vrrp_script haproxy {
  script   "systemctl status haproxy.service"
  interval 2
  weight   2
}
vrrp_instance 51 {
  virtual_router_id 51
  advert_int 1
  priority  101
  state     MASTER
  interface br-ex
  virtual_ipaddress {
      192.0.2.5 dev br-ex
  }
  track_script {
    haproxy
  }
}
vrrp_instance 52 {
  virtual_router_id 52
  advert_int 1
  priority  101
  state     MASTER
  interface br-ex
  virtual_ipaddress {
      192.0.2.4 dev br-ex
  }

  track_script {
    haproxy
  }
}

Comment 4 Ryan O'Hara 2015-06-16 18:51:26 UTC
I've recreated this problem on RHEL7.1 using keepalived 1.2.17. The track script is not causing any problems. In fact, this can be reproduced on a single node (ie. no other keepalived nodes participating in VRRP). Also note that dumping the config reveals that keepalived did parse both VRRP instances, but there is no attempt to add the VIP from the second instance. I will continue to investigate.

Comment 5 Ryan O'Hara 2015-06-16 19:35:51 UTC
Seems this problem is directly related to the fact that both VRRP instances are using the same interface. If I use different interfaces in each VRRP instance, it works as expected. Appears that a VRRP "thread" is not being created for one of the VRRP instances when the interfaces are the same in both.

Comment 6 Ryan O'Hara 2015-06-17 16:32:07 UTC
This is the patch that causes the regression:

https://github.com/acassen/keepalived/commit/76f9ce36ae389983b48a83a933182739786908c9

Still trying to understand why this causes problems.

Comment 7 Ryan O'Hara 2015-06-17 18:28:05 UTC
Scratch build for F21 available here:

https://kojipkgs.fedoraproject.org//work/tasks/123/10110123/keepalived-1.2.17-3.fc21.x86_64.rpm

Comment 8 Fedora Update System 2015-06-17 21:32:06 UTC
keepalived-1.2.17-3.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/keepalived-1.2.17-3.fc22

Comment 9 Fedora Update System 2015-06-17 21:33:16 UTC
keepalived-1.2.17-3.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/keepalived-1.2.17-3.fc21

Comment 10 Fedora Update System 2015-06-21 00:02:33 UTC
Package keepalived-1.2.17-3.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing keepalived-1.2.17-3.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-10228/keepalived-1.2.17-3.fc21
then log in and leave karma (feedback).

Comment 11 Fedora Update System 2015-06-23 13:56:17 UTC
keepalived-1.2.17-4.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/keepalived-1.2.17-4.fc21

Comment 12 Fedora Update System 2015-06-23 13:57:28 UTC
keepalived-1.2.17-4.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/keepalived-1.2.17-4.fc22

Comment 13 Fedora Update System 2015-07-01 15:52:27 UTC
keepalived-1.2.18-1.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/keepalived-1.2.18-1.fc22

Comment 14 Fedora Update System 2015-07-01 15:52:38 UTC
keepalived-1.2.18-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/keepalived-1.2.18-1.fc21

Comment 15 Fedora Update System 2015-07-14 15:42:03 UTC
keepalived-1.2.18-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2015-07-14 15:43:57 UTC
keepalived-1.2.18-1.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.