Bug 1207730 - Continuous IPv6 router solicitation loop
Summary: Continuous IPv6 router solicitation loop
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: NetworkManager
Version: 7.1
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: ---
Assignee: Dan Williams
QA Contact: Desktop QE
URL:
Whiteboard:
Depends On: 1044757
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-31 15:06 UTC by Petr Sklenar
Modified: 2015-11-19 11:01 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Router Advertisements without DNS configuration can no longer cause Router Solicitations to be sent in quick succession.
Clone Of: 1044757
Environment:
Last Closed: 2015-11-19 11:01:18 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
GNOME Bugzilla 748085 None None None 2019-07-15 20:22:26 UTC
Red Hat Product Errata RHSA-2015:2315 normal SHIPPED_LIVE Moderate: NetworkManager security, bug fix, and enhancement update 2015-11-19 10:06:58 UTC

Description Petr Sklenar 2015-03-31 15:06:26 UTC
+++ This bug was initially created as a clone of Bug #1044757 +++

NM can get into a state where it continuously solicits the router for an advertisement, which gets delivered, which causes another solicitation, repeat forever.

--- Additional comment from Dan Williams on 2013-12-18 23:46:12 CET ---

Tore, can you attach your radvd.conf, or otherwise inform me what the expected DNS server lifetime is from a wireshark capture?  There may be something wrong with NM's handling of the RDNSS and DNSSL lifetimes and timeouts that is causing the continuous solicitation.

--- Additional comment from Tom Hughes on 2013-12-19 01:54:50 CET ---

I can confirm seeing this, and that removing RDNSS and DNSSL settings stops it. The radvd.conf was:

interface br0
{
	AdvSendAdvert on;

	prefix 2001:8b0:bd:1::/64
	{
		AdvOnLink on;
		AdvAutonomous on;
	};

	RDNSS 2001:8b0:bd:1:204:e2ff:fe22:eba9 2001:8b0:bd:1:2b0:c2ff:fe02:4cf3 2001:8b0:bd:1:be5f:f4ff:fe45:81fd {
	};

	DNSSL compton.nu {
	};
};

Wireshark reports 600 as the advertised lifetime of the DNS details.

--- Additional comment from Tore Anderson on 2013-12-19 14:24:13 CET ---

Attached is /var/log/messages with NM debugging output enabled from when I've reproduced the bug. I'm using NetworkManager-0.9.9.0-20.git20131003.fc20.x86_64 on wired ethernet.

The log starts as I'm enabling networking at 10:06:30. It successfully connects and activates IPv6 using SLAAC and information-only DHCPv6, and for a while it is smooth sailing. There are some errors during that time, however:

[rdisc/nm-lndp-rdisc.c:194] send_rs(): (enp2s0): cannot send router solicitation: -99.
[platform/nm-linux-platform.c:1127] add_object(): Netlink error: Invalid address for specified address family
:: inet6 dev 2 scope nowhere
created boot-time+ <99>t<D8>C^? last-updated boot-time+ <99>t<D8>C^?

I don't know if these have any relevance on this bug, just thought it prudent to point them out.

At 11:12:01 the real fun starts. At this point it starts sending RSes and receives RAs in a close loop, making NM hog the CPU and fill the logs. It continues like this until I manually disabled networking again at 11:21:18. In those 10 minutes of madness NM produced about 1GiB with debugging output, so make sure you have sufficient space where you uncompress the file.

Tore

--- Additional comment from Tore Anderson on 2013-12-19 14:33:17 CET ---

This file contains a PCAP of all the RS/RA/DHCPv6 activity seen on the host triggering the bug. It's all smooth sailing until frame 16 at 11:21:01, beginning with an RS is being sent by NM, and the looping begins. Note that there are some unrelated RS/RA messages before the bug was triggered (e.g. frames 5 and 6), these were caused by another host on the LAN and can probably be ignored.

The router in question is an AVM FritzBox 7390, so I cannot get the radvd.conf (or whatever it's using) config file. However, to respond specifically to comment #1, the PCAP shows that the RDNSS lifetime is consistently being advertised at 1200.

Tore

--- Additional comment from Dan Williams on 2013-12-19 19:15:36 CET ---

Fixes pushed to upstream branch dcbw/rdisc-fixes.  See upstream bug report (noted above) for review progress.

--- Additional comment from Dan Williams on 2013-12-19 19:32:29 CET ---



--- Additional comment from Dan Williams on 2013-12-19 19:33:21 CET ---

Tore (or Tom!), could you try the attached patch?  It should apply successfully to the current NetworkManager Fedora 20 SRPM.

--- Additional comment from Tore Anderson on 2013-12-19 23:38:26 CET ---

(In reply to Dan Williams from comment #7)
> Tore (or Tom!), could you try the attached patch?  It should apply
> successfully to the current NetworkManager Fedora 20 SRPM.

It's been stable for two hours now on both my laptop (using WiFi) and my desktop (wired). When I reproduced it earlier the looping began about 65 minutes after activating the connection, so if the bug indeed is based on the timers expiring due to not being reset, I guess it should have been triggered after the same amount of time. Which means that the fact that it has not yet is a good sign.

On the other hand, that it was the RDNSS timer expiring that triggered it doesn't make fully sense to me either. My router advertises a RDNSS lifetime of 1200 seconds, but it took over an hour for the bug to trigger. If it was the RDNSS lifetime, shouldn't the madness have started after only 20 minutes, instead?

Tore

--- Additional comment from Dan Williams on 2013-12-20 01:00:33 CET ---

To ensure stability of the connection, NetworkManager enforces a minimum RDNSS and DNSSL lifetime of 7200 seconds.  That does seem a bit long, so we could reduce that substantially.  But this is why you didn't see the issue after 20 minutes.  This *is* why you saw the issue after 60+ minutes, because...

NetworkManager sends the Router Solicitation at half the lifetime of the DNSSL or RDNSS option, and this is why you only saw the solicit-loop after a significant amount of time had passed.

The intention of this minimum time is to counter a problem we had earlier this year with routers that had significantly shorter RDNSS lifetimes than address lifetimes.  If the router did not send advertisements often, or was misconfigured to send them less often than the RDNSS timeout, NetworkManager would tear down the connection because the DNS servers expired.

NM should probably stop doing this, and just let users suffer without DNS rather than terminating the connection entirely.  Now that we have implemented router solicitations when the DNSSL and RDNSS options are near their expiration (much like DHCP rebinds before the lease has expired), this should be much less of an issue.

--- Additional comment from Dan Williams on 2013-12-20 01:01:41 CET ---

I should note that NM *should* correctly handle RDNSS/DNSSL lifetimes of '0', meaning "stop using this [server | domain]".  Only if the lifetime is > 0 is it padded to 7200.

--- Additional comment from Dan Williams on 2013-12-20 01:03:55 CET ---

dcbw/rdisc-fixes  merged to master upstream after reviews.

--- Additional comment from Fedora Update System on 2013-12-20 04:03:06 CET ---

NetworkManager-0.9.9.0-22.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-22.git20131003.fc20

--- Additional comment from Fedora Update System on 2013-12-21 03:15:36 CET ---

Package NetworkManager-0.9.9.0-22.git20131003.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing NetworkManager-0.9.9.0-22.git20131003.fc20'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-23712/NetworkManager-0.9.9.0-22.git20131003.fc20
then log in and leave karma (feedback).

--- Additional comment from Fedora Update System on 2013-12-22 06:36:13 CET ---

NetworkManager-0.9.9.0-22.git20131003.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

--- Additional comment from Seb L. on 2014-01-04 19:08:36 CET ---



--- Additional comment from Jonathan Abbey on 2014-01-05 04:31:07 CET ---

This is still happening with NetworkManager-0.9.9.0-22.git20131003.fc20.x86_64.  I'm using a DLink 825 configured to use a Hurricane Electric IPv6 tunnel, for what it's worth.

--- Additional comment from  on 2014-01-05 18:31:02 CET ---

On Fedora 20 with also IPv6 via Hurricane Electric tunnel this is also happening could this be more related to HE ipv6?  Version: NetworkManager.x86_64           1:0.9.9.0-22.git20131003.fc20

Regards

Robert

--- Additional comment from Dan Williams on 2014-01-08 22:45:42 CET ---

Can anyone with a Hurricane tunnel get a wireshark capture of the Router Advertisement for me?  Also, can you grab log messages from the journal from before the problem starts so I can see what's going on?

--- Additional comment from Jiri Popelka on 2014-01-20 15:25:31 CET ---

In my case this IPv6 router solicitation loop *started* with NetworkManager-0.9.9.0-22.git20131003.fc20.x86_64 (0.9.9.0-21 is OK).

I can see
<info> Activation (em1) Stage 5 of 5 (IPv6 Commit) scheduled...
<info> Activation (em1) Stage 5 of 5 (IPv6 Commit) started...
<info> Activation (em1) Stage 5 of 5 (IPv6 Commit) complete.
repeating every 3s.

Strangely I don't see any RS/RA with my ll address in wireshark.

--- Additional comment from Dan Williams on 2014-01-20 19:43:02 CET ---

(In reply to Jiri Popelka from comment #19)
> In my case this IPv6 router solicitation loop *started* with
> NetworkManager-0.9.9.0-22.git20131003.fc20.x86_64 (0.9.9.0-21 is OK).
> 
> I can see
> <info> Activation (em1) Stage 5 of 5 (IPv6 Commit) scheduled...
> <info> Activation (em1) Stage 5 of 5 (IPv6 Commit) started...
> <info> Activation (em1) Stage 5 of 5 (IPv6 Commit) complete.
> repeating every 3s.
> 
> Strangely I don't see any RS/RA with my ll address in wireshark.

This would be happening when the kernel makes a change to the routing table, this is NetworkManager reading the kernel configuration and updating its internal structures.  These messages should likely be suppressed for external changes.

--- Additional comment from  on 2014-01-23 14:00:37 CET ---

I also see this kind of message, every 5-10s in my case.

NetworkManager[605]: <info> Activation (em1) Stage 5 of 5 (IPv6 Commit) scheduled...
NetworkManager[605]: <info> Activation (em1) Stage 5 of 5 (IPv6 Commit) started...
NetworkManager[605]: <error> [1390480120.831917] [platform/nm-linux-platform.c:1127] add_object(): Netlink error: Invalid address for specified address family
NetworkManager[605]: :: inet6 dev 2 scope nowhere
NetworkManager[605]: valid-lifetime forever preferred-lifetime forever
NetworkManager[605]: [54B blob data]
NetworkManager[605]: <info> Activation (em1) Stage 5 of 5 (IPv6 Commit) complete.


rpm -q NetworkManager
NetworkManager-0.9.9.0-26.git20131003.fc20.x86_64

--- Additional comment from Dan Williams on 2014-01-24 20:39:27 CET ---

(In reply to hansvon from comment #21)
> I also see this kind of message, every 5-10s in my case.
> 
> NetworkManager[605]: <info> Activation (em1) Stage 5 of 5 (IPv6 Commit)
> scheduled...
> NetworkManager[605]: <info> Activation (em1) Stage 5 of 5 (IPv6 Commit)
> started...
> NetworkManager[605]: <error> [1390480120.831917]
> [platform/nm-linux-platform.c:1127] add_object(): Netlink error: Invalid
> address for specified address family
> NetworkManager[605]: :: inet6 dev 2 scope nowhere
> NetworkManager[605]: valid-lifetime forever preferred-lifetime forever
> NetworkManager[605]: [54B blob data]
> NetworkManager[605]: <info> Activation (em1) Stage 5 of 5 (IPv6 Commit)
> complete.
> 
> 
> rpm -q NetworkManager
> NetworkManager-0.9.9.0-26.git20131003.fc20.x86_64

This should be covered by bug 1048046 and is fixed upstream.

--- Additional comment from  on 2014-01-27 10:36:29 CET ---

Thank you, I confirm, the patch fixes the "Netlink" error. For now, I also commented the "nm_log_info" calls to get rid of the three "Stage 5 of 5 (IPv6 Commit)" messages.

--- Additional comment from Dan Williams on 2014-01-27 22:28:47 CET ---

(In reply to hansvon from comment #23)
> Thank you, I confirm, the patch fixes the "Netlink" error. For now, I also
> commented the "nm_log_info" calls to get rid of the three "Stage 5 of 5
> (IPv6 Commit)" messages.

Those "Stage 5 of 5" messages are also now suppressed upstream and should make it to an F20 build at some point here.

--- Additional comment from Fedora Update System on 2014-01-28 16:23:43 CET ---

NetworkManager-0.9.9.0-27.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-27.git20131003.fc20

--- Additional comment from Fedora Update System on 2014-01-30 04:41:45 CET ---

NetworkManager-0.9.9.0-27.git20131003.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

--- Additional comment from Jiri Popelka on 2014-01-30 09:48:40 CET ---

(In reply to Dan Williams from comment #24)
> Those "Stage 5 of 5" messages are also now suppressed upstream and should
> make it to an F20 build at some point here.

I can confirm this with NetworkManager-0.9.9.0-27.git20131003.fc20.x86_64
Thanks.

Comment 6 Petr Sklenar 2015-03-31 15:23:57 UTC
adding keyword regression as it wasn't happing before

Comment 8 Dan Williams 2015-03-31 16:06:15 UTC
Could you attach the full /var/log/messages from that time?  The partial log doesn't have information on the actual solicitations.  Or:

journalctl -b -u NetworkManager

might give only the NM logs, if you are worried about exposing too much other stuff.

Comment 14 Dan Williams 2015-04-07 22:39:08 UTC
Some analysis:

15:28:51 - the start of the solictiation loop
16:28:47 - end of the solictitation loop

<<< nm gets restarted a bunch >>>

17:17:17 - eth0 has IPv6 configuration, which is assumed:
  2620:52:0:2258:d836:d0ff:fe6e:5033/64 lft 2591939sec pref 604739sec
  fec0::f101:d836:d0ff:fe6e:5033/64 lft 78194sec pref 6194sec
  fe80::d836:d0ff:fe6e:5033/64 lft forever pref forever

17:17:17 - solicitation sent
17:17:17 - 1st RA received
  dhcp-level none
  gateway fe80::42f2:e9ff:fea0:8d33 pref 2 exp 1801
  address 2620:52:0:2258:d836:d0ff:fe6e:5033 exp 3601
  route 2620:52:0:2258::/64 via :: pref 0 exp 3601
  dns_server fec0::f101:42f2:e9ff:fea0:8d33 exp 7201

17:17:18 - 2nd RA; from a different router (::fe), advertising the same prefix, with a greatly increased lifetime, and no DNS servers given
  dhcp-level none
  gateway fe80:52:0:2258::fe pref 2 exp 1801
  address 2620:52:0:2258:d836:d0ff:fe6e:5033 exp 2592001
  route 2620:52:0:2258::/64 via :: pref 0 exp 2592001

17:20:19 - 3rd RA; from ::ffe, only the address has changed

<<< more RAs from ::ffe, but no solicitations are sent >>>

17:47:09 - gateway :8d33 finally times out

18:15:14 - RA received from ::ffe; the DNS servers from :8d33 are going to time out in 123 seconds

18:17:18 - the DNS servers from RA #1 have timed out, and a solicitation is sent

18:17:18 - RA received from :8d33; this includes new DNS servers and a new prefix with a much shorter lifetime than ::ffe, and since it's most recent, this new lifetime is preferred

18:17:18 - RA received from ::ffe, which updates the address lifetime to a huge value again

19:17:19 - DNS servers from :8d33 have timed out again, a solicitation is sent

This run looks fine so far.

----------

Conclusions:

- clearly there's something wrong with NM's RA option timeout handling here

- the network has two IPv6 routers, which is not odd.  What *is* odd is that (a) one router never sends announcements on its own (:8d33), and (b) the one that does send periodic announcements (::ffe) advertises the same prefix as :8d33 but doesn't send the DNS option, and it uses a huge address lifetime

I'd love to know why IS/IT has this IPv6 router configuration, as it's bound to make things a bit confusing for clients.

My guess is that duelling routers and possibly some lifetime=0 RAs triggered a bug in NM's timeout handling.

Comment 15 Dan Williams 2015-04-08 21:57:53 UTC
Petr, do you think you could reproduce the situation if I gave you a NetworkManager build with some IPv6 debugging info turned on?

Comment 16 Petr Sklenar 2015-04-09 07:02:37 UTC
(In reply to Dan Williams from comment #15)
> Petr, do you think you could reproduce the situation if I gave you a
> NetworkManager build with some IPv6 debugging info turned on?

I guess yes, what should I setup?

Comment 17 Dan Williams 2015-04-09 23:28:20 UTC
More refined theory:

1) Two routers send RAs.
2) One router sends DNS servers or domains, the other does not
3) The first router stops responding to solicitations, but the second continues to respond
4) When the DNS servers/domains reach 1/2 their lifetime, a solicitation is triggered to refresh them
5) The second router responds, but since its RA does not include any DNS servers or domains, the DNS servers/searches aren't refreshed.  In any case, check_timestamps() gets called in response to any RA to clean up any stale information.
6) check_timestamps() walks through the DNS servers/domains and since they are still past 1/2 their lifetime (since they weren't refreshed), another solicitation is sent
7) go to step 5

In any case, to prove/disprove this theory, I've done a scratch build with RA debugging turned on:

http://brewweb.devel.redhat.com/brew/taskinfo?taskID=8964737

Grab those RPMs, and "rpm -Fvh" them (not Uvh, so you don't install new RPMs that you don't already have installed).  Could you run that for a while and if you notice a loop, grab the logs?  Thanks!

Comment 18 Dan Williams 2015-04-09 23:50:00 UTC
The issue could also happen if the IPv6 router removes the DNS servers/domains from its RA, but keeps responding to solicitations.

Comment 19 Dan Williams 2015-04-10 19:11:32 UTC
Fixes posted upstream as dcbw/rh1207730-rdisc-fixes.

Comment 20 Igor Donev 2015-04-13 09:44:18 UTC
This is a rouge router:
 dhcp-level none
  gateway fe80::42f2:e9ff:fea0:8d33 pref 2 exp 1801
  address 2620:52:0:2258:d836:d0ff:fe6e:5033 exp 3601
  route 2620:52:0:2258::/64 via :: pref 0 exp 3601
  dns_server fec0::f101:42f2:e9ff:fea0:8d33 exp 7201

I am trying to figure out who owns this machine and why he is doing this. The other RA is from the IT network switch. We do not send DNS information.

Comment 22 Dan Williams 2015-04-21 14:11:54 UTC
Petr, any chance you've been able to test out the packages above and see if they fix the issue?

Comment 23 Petr Sklenar 2015-04-22 06:20:23 UTC
(In reply to Dan Williams from comment #22)
> Petr, any chance you've been able to test out the packages above and see if
> they fix the issue?

I am sorry that I didnt reply. I am on it right now, I let you know the results in day or two

Comment 24 Thomas Haller 2015-04-28 16:53:47 UTC
(In reply to Dan Williams from comment #19)
> Fixes posted upstream as dcbw/rh1207730-rdisc-fixes.

>> rdisc: split fake & linux test code; add testcases

could you refactor the test files with the main() function to use nm-test-utils.h and NMTST_DEFINE()?

Pushed a fixup for that.


> rdisc: fix double-addition of gateways & routes if priority increases


this seems right:

+    if (!new->lifetime)
+         return FALSE;
     g_array_insert_val (rdisc->addresses, i, *new);
     return TRUE;

this seems wrong:

+    if (new->lifetime)
+         g_array_insert_val (rdisc->routes, CLAMP (insert_idx, 0, G_MAXINT), *new);
     return TRUE;

(at several places)




The linux test fails for me:

$ sudo NMTST_DEBUG=debug src/rdisc/tests/test-rdisc-linux 
(src/rdisc/tests/test-rdisc-linux:32513): NetworkManager-WARNING **: <error> [1430239935.402838] [rdisc/nm-lndp-rdisc.c:68] send_rs(): (lo): cannot send router solicitation: -101.


And the test-rdisc-fake test takes painfully long. Can we reduce the timeouts there?

Comment 25 Dan Williams 2015-04-29 21:09:41 UTC
(In reply to Thomas Haller from comment #24)
> (In reply to Dan Williams from comment #19)
> > Fixes posted upstream as dcbw/rh1207730-rdisc-fixes.
> 
> >> rdisc: split fake & linux test code; add testcases
> 
> could you refactor the test files with the main() function to use
> nm-test-utils.h and NMTST_DEFINE()?
> 
> Pushed a fixup for that.

Looks good, thanks.

> > rdisc: fix double-addition of gateways & routes if priority increases
> 
> 
> this seems right:
> 
> +    if (!new->lifetime)
> +         return FALSE;
>      g_array_insert_val (rdisc->addresses, i, *new);
>      return TRUE;
> 
> this seems wrong:
> 
> +    if (new->lifetime)
> +         g_array_insert_val (rdisc->routes, CLAMP (insert_idx, 0,
> G_MAXINT), *new);
>      return TRUE;
> 
> (at several places)

Yeah, true.  That also means that we have to add some code to return TRUE if the lifetime=0 in a few places, which I've done in a fixup.  In half of the functions both removal and addition would have ended at the bottom here, so I made it consistent in the fixup so that removal always returns TRUE.

> The linux test fails for me:
> 
> $ sudo NMTST_DEBUG=debug src/rdisc/tests/test-rdisc-linux 
> (src/rdisc/tests/test-rdisc-linux:32513): NetworkManager-WARNING **: <error>
> [1430239935.402838] [rdisc/nm-lndp-rdisc.c:68] send_rs(): (lo): cannot send
> router solicitation: -101.

ENETUNREACH; I think that's because 'lo' doesn't actually have a valid IPv6LL address, so the packets can't go anywhere.  I'm not sure there's a lot we can do about that, since 'lo' doesn't get an IPv6LL address ever.  Running the test on a real ethernet interface with an IPv6LL address works though.  Thoughts?

> And the test-rdisc-fake test takes painfully long. Can we reduce the
> timeouts there?

I've reduced the timeouts on the first three tests, but I don't want to reduce it too much for the solicitation loop test because I want to make sure that it still works correctly even on a more loaded machine.  In any case, the tests now take 20 seconds instead of much more.  Is that OK?

Repushed.

Comment 26 Jirka Klimes 2015-04-30 08:20:40 UTC
The branch looks good to me and all tests pass.

Comment 27 Dan Williams 2015-05-01 21:37:30 UTC
Merged upstream to git master and nm-1-0

Comment 28 Dan Williams 2015-05-26 14:55:21 UTC
The following commits from nm-1-0 should be backported to 7.2:

5955f82f01f2da4b0107be6bc4f7c65d26d77b24 rdisc: add missing chain up to parent finalize/dispose

0be367846614de8e9678934e7be7c70871f7c983 rdisc: move most RA processing logic into base class

272943db4ba28a1681282332861d273e4bcc6e13 rdisc: fix leak of DNS domains

39fd8f7683d9bbda09a3b69ef7a3e7927b96b851 rdisc: split fake & linux test code; add testcases

415b7b3e257c5d5526618b0d6a89a6d7fb235f98 rdisc: fix double-addition of gateways & routes if priority increases

d96b05bd364a334f20b10f5322aabf01ba28423d rdisc: prevent solicitation loop for expiring DNS information (rh #1207730) (rh #1151665)

Comment 31 Vladimir Benes 2015-11-02 11:16:41 UTC
I am not able to reproduce when using configuration from comment #0, commenting out RDNSS and DNSSL sections, killing radvd daemon and restarting it. After an hour there is still small number of solicitation from NM. No flood seen.

Comment 32 errata-xmlrpc 2015-11-19 11:01:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-2315.html


Note You need to log in before you can comment on or make changes to this bug.