Bug 1031102 - Firewalld should use rmmod instead of modprobe -r [NEEDINFO]
Firewalld should use rmmod instead of modprobe -r
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: firewalld (Show other bugs)
7.0
Unspecified Unspecified
unspecified Severity high
: rc
: 7.0
Assigned To: Thomas Woerner
Tomas Dolezal
:
: 1048928 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-15 10:46 EST by Martin
Modified: 2014-09-14 20:04 EDT (History)
18 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-06-13 05:17:29 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
jscotka: needinfo? (twoerner)


Attachments (Terms of Use)

  None (edit)
Description Martin 2013-11-15 10:46:57 EST
Description of problem:
Stoping FirewallD removes libvirt's network interfaces

Version-Release number of selected component (if applicable):
firewalld-0.3.8-1.el7.noarch
libvirt-daemon-1.1.1-12.el7.x86_64
kernel-3.10.0-50.el7.x86_64

How reproducible:
always

Steps to Reproduce:
systemctl stop firewalld.service

Actual results:
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether f0:de:f1:6b:1d:b8 brd ff:ff:ff:ff:ff:ff
    inet 10.34.131.239/23 scope global dynamic em1
       valid_lft 85217sec preferred_lft 85217sec
    inet6 2620:52:0:2282:f2de:f1ff:fe6b:1db8/64 scope global dynamic 
       valid_lft 2591984sec preferred_lft 604784sec
    inet6 fe80::f2de:f1ff:fe6b:1db8/64 scope link 
       valid_lft forever preferred_lft forever
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:24:d7:cd:a6:d0 brd ff:ff:ff:ff:ff:ff
    inet 10.200.138.49/22 scope global dynamic wlp3s0
       valid_lft 6018sec preferred_lft 6018sec
    inet6 fe80::224:d7ff:fecd:a6d0/64 scope link 
       valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN qlen 500
    link/ether 52:54:00:65:83:75 brd ff:ff:ff:ff:ff:ff
7: virbr1-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN qlen 500
    link/ether 52:54:00:75:c0:5c brd ff:ff:ff:ff:ff:ff

Expected results:
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether f0:de:f1:6b:1d:b8 brd ff:ff:ff:ff:ff:ff
    inet 10.34.131.239/23 scope global dynamic em1
       valid_lft 85772sec preferred_lft 85772sec
    inet6 2620:52:0:2282:f2de:f1ff:fe6b:1db8/64 scope global dynamic 
       valid_lft 2591995sec preferred_lft 604795sec
    inet6 fe80::f2de:f1ff:fe6b:1db8/64 scope link 
       valid_lft forever preferred_lft forever
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:24:d7:cd:a6:d0 brd ff:ff:ff:ff:ff:ff
    inet 10.200.138.49/22 scope global dynamic wlp3s0
       valid_lft 6573sec preferred_lft 6573sec
    inet6 fe80::224:d7ff:fecd:a6d0/64 scope link 
       valid_lft forever preferred_lft forever
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN 
    link/ether 52:54:00:65:83:75 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 500
    link/ether 52:54:00:65:83:75 brd ff:ff:ff:ff:ff:ff
6: virbr1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN 
    link/ether 52:54:00:75:c0:5c brd ff:ff:ff:ff:ff:ff
    inet 192.168.254.1/24 brd 192.168.254.255 scope global virbr1
       valid_lft forever preferred_lft forever
7: virbr1-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr1 state DOWN qlen 500
    link/ether 52:54:00:75:c0:5c brd ff:ff:ff:ff:ff:ff

Additional info:
When I try to run virtual machine:

Error starting domain: Cannot get interface MTU on 'virbr1': No such device

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 100, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 122, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/domain.py", line 1220, in startup
    self._backend.create()
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 698, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: Cannot get interface MTU on 'virbr1': No such device
Comment 2 Eric Blake 2013-11-15 11:10:25 EST
(In reply to Martin Holec from comment #0)
> Description of problem:
> Stoping FirewallD removes libvirt's network interfaces

Well yeah, since libvirt depends on the firewall to set up its interfaces.

> 
> Version-Release number of selected component (if applicable):
> firewalld-0.3.8-1.el7.noarch
> libvirt-daemon-1.1.1-12.el7.x86_64
> kernel-3.10.0-50.el7.x86_64
> 
> How reproducible:
> always
> 
> Steps to Reproduce:
> systemctl stop firewalld.service

Maybe this implies our systemd config file needs to set up a hard dependency, where firewalld cannot be stopped if libvirtd is still running?
Comment 3 Daniel Berrange 2013-11-15 11:15:10 EST
(In reply to Eric Blake from comment #2)
> (In reply to Martin Holec from comment #0)
> > Description of problem:
> > Stoping FirewallD removes libvirt's network interfaces
> 
> Well yeah, since libvirt depends on the firewall to set up its interfaces.

That doesn't make any sense. Firewalld is only concerned with firewall rules. Stopping it shouldn't cause the virbr1 bridge device to be deleted or offlined.

> > Version-Release number of selected component (if applicable):
> > firewalld-0.3.8-1.el7.noarch
> > libvirt-daemon-1.1.1-12.el7.x86_64
> > kernel-3.10.0-50.el7.x86_64
> > 
> > How reproducible:
> > always
> > 
> > Steps to Reproduce:
> > systemctl stop firewalld.service
> 
> Maybe this implies our systemd config file needs to set up a hard
> dependency, where firewalld cannot be stopped if libvirtd is still running?

I don't think so. Stopping firewalld should not affect running networking. It should merely mean that you can't make further changes to the firewall rules until it is started again.
Comment 4 Martin 2013-11-15 11:57:58 EST
I agree with Daniel.

FirewallD shouldn't interfere with network interfaces created by other daemons.

Eric,
if I am prevented from stoping FirewallD or running libvirt without FirewallD, I couldn't decide whether my scenario fails because of firewall misconfiguration, libvirt networking setup or service daemon configuration. 

FirewallD as hard dependency for *any* service is very bad idea. When I'm disabling the firewall, I know what I'm doing and why and it's only temporally. In principle it's the same as setting SELinux to "permissive" mode temporally.
Comment 5 Laine Stump 2013-11-26 11:55:03 EST
This is reproducible without libvirt in the equation, so I'm reassigning to firewalld:

# brctl addbr xyzzy
# brctl show
bridge name	bridge id		STP enabled	interfaces
xyzzy		8000.000000000000	no		
# systemctl stop firewalld.service
# brctl show
bridge name	bridge id		STP enabled	interfaces
#

And of course the destroyed bridge doesn't return when you restart firewalld.

This is obviously *very bad* for libvirt (and any management based on libvirt), since it means that any time firewalld is restarted, all virtual guests will be disconnected from the network, and the only reasonable method of reconnecting them will be to reboot them all.

BTW, I witnessed the same behavior on Fedora 20 (firewalld-0.3.8-1.fc20.noarch)
Comment 6 Laine Stump 2013-11-26 12:21:06 EST
Info from /var/log/messages at the request of twoerner:

Nov 26 18:47:10 vlap NetworkManager[1148]: <info> (xyzzy): ignoring bridge not created by NetworkManager
Nov 26 18:47:10 vlap NetworkManager[1148]: <info> (xyzzy): ignoring bridge not created by NetworkManager
Nov 26 18:47:28 vlap systemd: Stopping firewalld - dynamic firewall daemon...
Nov 26 18:47:29 vlap avahi-daemon[1002]: Withdrawing workstation service for xyzzy.
Nov 26 18:47:29 vlap kernel: [191096.787315] Ebtables v2.0 unregistered
Nov 26 18:47:29 vlap systemd: Stopped firewalld - dynamic firewall daemon.
Nov 26 18:47:31 vlap kernel: [191099.288209] Bridge firewalling registered

This is starting to sound familiar - perhaps NM isn't *really* ignoring the bridge?
Comment 7 Thomas Woerner 2013-11-26 13:08:55 EST
Reassigning to kernel:

Why is there an entry for bridge to ebtable_broute? modprobe -r ebtable_broute unloads bridge also. rmmod ebtable_broute is working, though.

ebtable_nat            12807  0 
ebtable_broute         12731  0 
ebtable_filter         12827  0 lsmod | grep bridge
ebtables               30758  3 ebtable_broute,ebtable_nat,ebtable_filter
bridge                110617  1 ebtable_broute

Steps to reproduce the problem:

1) systemctl stop firewalld.service; systmctl stop libvirtd.service
   -> just to make sure that nothing is using bridges and the firewall
2) brctl addbr xyzzy
3) brctl show
4) lsmod | grep bridge
   -> bridge is not using ebtable_broute
5) modprobe ebtable_broute
6) lsmod | grep bridge
   -> now bridge is using ebtable_broute
7) modprobe -r ebtable_broute
   -> removing ebtable_broute and bridge
8) brctl show
   -> no bridge anymore
9) lsmod | grep bridge
   -> no bridge anymore

This seems to be a problem for all newer kernels also in Fedora.
Comment 8 Jiri Benc 2013-11-26 13:33:40 EST
This is a modprobe bug:

[root@localhost ~]# strace -f modprobe -r ebtable_broute
...
delete_module("bridge", O_RDONLY)       = 0
...
Comment 14 Václav Pavlín 2013-12-11 08:53:43 EST
Man page says: 

If the modules it depends on are also unused, modprobe will try to remove them too.

Josh, any idea why does modprobe do this when rmmod doesn't? From the modprobe logic I see "unused" means there are no other modules using the module that's being removed, but it doesn't say anything about actual software using the module.
Comment 15 Josh Boyer 2013-12-11 09:31:10 EST
modprobe won't remove the module if the kernel's refcnt for that module is non-zero.  If there is software using the module then I would expect the refcnt for that module (bridge in this case) to be non-zero.

So what is the refcnt of the bridge module?  You can get it directly from the kernel in /sys, so:

[jwboyer@zod linux]$  lsmod | grep ebtable
ebtable_nat            12807  0 
ebtable_broute         12731  0 
bridge                110624  1 ebtable_broute
ebtable_filter         12827  0 
ebtables               30758  3 ebtable_broute,ebtable_nat,ebtable_filter
[jwboyer@zod linux]$ cat /sys/module/ebtable_broute/refcnt 
0
[jwboyer@zod linux]$ cat /sys/module/bridge/refcnt 
1
[jwboyer@zod linux]$ 

is what I see on my f19 machine.  If you remove ebtable_broute with rmmod, and the refcnt goes to 0, modprobe will remove that too.  If that's the case, is any software directly using the bridge module?
Comment 16 Daniel Berrange 2013-12-11 09:39:01 EST
(In reply to Josh Boyer from comment #15)
> modprobe won't remove the module if the kernel's refcnt for that module is
> non-zero.  If there is software using the module then I would expect the
> refcnt for that module (bridge in this case) to be non-zero.

The refcount for the bridge module is only affected by other kernel module deps. Whether you have actual bridge devices in existance doesn't affect it.

ie i have a bridge device:

# brctl show
bridge name	bridge id		STP enabled	interfaces
virbr0		8000.52540054952a	yes		virbr0-nic
							vnet0
							vnet1


and the ref count is still zero

# lsmod  | grep bridge
bridge                110624  0 


IMHO this is a firewalld bug - it should not be telling the kernel to recursively remove unused moduls.
Comment 17 Václav Pavlín 2013-12-11 09:45:27 EST
The problem is that recursive removing of unused modules is default behaviour of modprobe -r command. So the workaround is to use rmmod instead, but I don't think the modprobe behaviour is correct (or doing this based on refcnt is not correct).
Comment 18 Jiri Benc 2013-12-11 09:46:40 EST
(In reply to Josh Boyer from comment #15)
> modprobe won't remove the module if the kernel's refcnt for that module is
> non-zero.  If there is software using the module then I would expect the
> refcnt for that module (bridge in this case) to be non-zero.

That's not correct, sorry. The refcnt is a counter of how many other modules would blow up if this module was unloaded. Thus, its meaning is solely whether the module can or cannot be removed at the particular time. It has nothing to do with the fact whether a feature provided by the module is used or not.

In other words, when refcnt is zero, the module is able to clean up after itself, releasing all acquired kernel resources. If it's not zero, it's not able to clean up and must not be removed.

As an example, you can unload driver for a network card. That will destroy the network interfaces that belong to that card. The reference counter says nothing about whether there's a traffic going through those interfaces.

> So what is the refcnt of the bridge module?  You can get it directly from
> the kernel in /sys, so:

It's 0, as no other module needs it. That does not mean there are no bridges registered and in use.

> [jwboyer@zod linux]$  lsmod | grep ebtable
> ebtable_nat            12807  0 
> ebtable_broute         12731  0 
> bridge                110624  1 ebtable_broute
> ebtable_filter         12827  0 
> ebtables               30758  3 ebtable_broute,ebtable_nat,ebtable_filter
> [jwboyer@zod linux]$ cat /sys/module/ebtable_broute/refcnt 
> 0
> [jwboyer@zod linux]$ cat /sys/module/bridge/refcnt 
> 1
> [jwboyer@zod linux]$ 
> 
> is what I see on my f19 machine.  If you remove ebtable_broute with rmmod,
> and the refcnt goes to 0, modprobe will remove that too.  If that's the
> case, is any software directly using the bridge module?

Not sure what you mean by "software" but there surely can be bridges registered and in use.
Comment 19 Jiri Benc 2013-12-11 09:56:28 EST
(In reply to Daniel Berrange from comment #16)
> IMHO this is a firewalld bug - it should not be telling the kernel to
> recursively remove unused moduls.

The kernel does not do that, it's done by modprobe. But you're probably right that firewalld should use rmmod instead. The behavior of modprobe -r does not make much sense but I suppose it's doing that since ages and cannot be easily changed, as there may be people relying on it.

Interesting what one learns, I always thought rmmod and modprobe -r were more or less equivalents.
Comment 20 Daniel Berrange 2013-12-11 09:58:29 EST
(In reply to Václav Pavlín from comment #17)
> The problem is that recursive removing of unused modules is default
> behaviour of modprobe -r command. So the workaround is to use rmmod instead,
> but I don't think the modprobe behaviour is correct (or doing this based on
> refcnt is not correct).

Well the modprobe behaviour is correct within the confines of its defined semantics. It is just that those semantics aren't particularly useful! I question why firewalld is trying to remove modules at all rather than just leaving them alone. The modprobe manpage itself says

  "There is usually no reason to remove modules, but some buggy modules require it. Your distribution kernel may not have been built to support removal of modules at all."
Comment 21 Josh Boyer 2013-12-11 10:03:51 EST
(In reply to Jiri Benc from comment #18)
> (In reply to Josh Boyer from comment #15)
> > modprobe won't remove the module if the kernel's refcnt for that module is
> > non-zero.  If there is software using the module then I would expect the
> > refcnt for that module (bridge in this case) to be non-zero.
> 
> That's not correct, sorry. The refcnt is a counter of how many other modules
> would blow up if this module was unloaded. Thus, its meaning is solely
> whether the module can or cannot be removed at the particular time. It has
> nothing to do with the fact whether a feature provided by the module is used
> or not.

Perhaps it depends on the module itself.  I have e.g. fuse here loaded, with no other modules depending on it and it has a refcnt of 3.  So something other than a module has increased that.

> In other words, when refcnt is zero, the module is able to clean up after
> itself, releasing all acquired kernel resources. If it's not zero, it's not
> able to clean up and must not be removed.

We agree on that part at least.

> As an example, you can unload driver for a network card. That will destroy
> the network interfaces that belong to that card. The reference counter says
> nothing about whether there's a traffic going through those interfaces.

Then maybe network drivers operate differently.

(In reply to Václav Pavlín from comment #17)
> The problem is that recursive removing of unused modules is default
> behaviour of modprobe -r command. So the workaround is to use rmmod instead,
> but I don't think the modprobe behaviour is correct (or doing this based on
> refcnt is not correct).

This has been the case for modprobe for basically forever.  It was already present in module-init-tools long ago.  I don't think changing the behavior of -r now is the correct thing to do.  It also has nothing other than the refcnt to go on, so I don't think there's a suitable alternative.


(In reply to Daniel Berrange from comment #16)
> IMHO this is a firewalld bug - it should not be telling the kernel to
> recursively remove unused moduls.

It could do that by using rmmod instead.
Comment 22 Jiri Benc 2013-12-11 10:07:03 EST
(In reply to Daniel Berrange from comment #20)
> Well the modprobe behaviour is correct within the confines of its defined
> semantics. It is just that those semantics aren't particularly useful!

:-)

> I question why firewalld is trying to remove modules at all rather than just
> leaving them alone.

You have very good point, generally.

However, in this particular case (ebtables), just leaving the modules loaded with empty tables has a small, yet measurable performance impact on bridge throughput. I 100% agree it's a kernel bug and empty tables should be equal to having the module unloaded. Hopefully, it will be fixed when we switch to nftables.
Comment 23 Jiri Benc 2013-12-11 10:13:56 EST
(In reply to Josh Boyer from comment #21)
> (In reply to Jiri Benc from comment #18)
> > That's not correct, sorry. The refcnt is a counter of how many other modules
> > would blow up if this module was unloaded. Thus, its meaning is solely
> > whether the module can or cannot be removed at the particular time. It has
> > nothing to do with the fact whether a feature provided by the module is used
> > or not.
> 
> Perhaps it depends on the module itself.  I have e.g. fuse here loaded, with
> no other modules depending on it and it has a refcnt of 3.  So something
> other than a module has increased that.

I probably oversimplified. The point is, the module refcnt increases when some kernel code calls module_get and decreases when module_put is called, and that's basically everything that can be said wrt. refcnt semantics. In kernel, in general, references are not taken unless really needed. If something can be done safely without increasing the reference count, it's done that way.

As a special case, that probably led to the implementation of the dubious modprobe -r feature, when one module uses symbols exported from another module, the module_get is called by the kernel module loader.
Comment 24 Josh Boyer 2013-12-11 10:16:26 EST
(In reply to Jiri Benc from comment #23)
> (In reply to Josh Boyer from comment #21)
> > (In reply to Jiri Benc from comment #18)
> > > That's not correct, sorry. The refcnt is a counter of how many other modules
> > > would blow up if this module was unloaded. Thus, its meaning is solely
> > > whether the module can or cannot be removed at the particular time. It has
> > > nothing to do with the fact whether a feature provided by the module is used
> > > or not.
> > 
> > Perhaps it depends on the module itself.  I have e.g. fuse here loaded, with
> > no other modules depending on it and it has a refcnt of 3.  So something
> > other than a module has increased that.
> 
> I probably oversimplified. The point is, the module refcnt increases when
> some kernel code calls module_get and decreases when module_put is called,
> and that's basically everything that can be said wrt. refcnt semantics. In
> kernel, in general, references are not taken unless really needed. If
> something can be done safely without increasing the reference count, it's
> done that way.

OK, I knew that and I agree.  I suppose I had forgotten that the kernel doesn't protect userspace from shooting itself in the foot :)
Comment 25 Václav Pavlín 2013-12-12 03:54:53 EST
Ok, so the behavior of kmod is correct (although 'those semantics aren't particularly useful') and firewalld should use rmmod rather than modprobe -r (or shouldn't unload modules at all). Moving back to firewalld.
Comment 27 Jiri Benc 2013-12-12 09:30:25 EST
+ # Unloading of modul with rmmod sometimes fails for no obvious reason
+ # (even if no other modul uses it). It was ok with modprobe -r.

That's strange and should not happen. Do you have more details? Anything in dmesg?
Comment 28 Jiri Popelka 2013-12-12 09:41:58 EST
Nothing in dmesg or journal. rmmod only tells me that for example
rmmod: ERROR: Module nf_conntrack_ipv6 is in use
Comment 29 Jiri Benc 2013-12-12 09:52:25 EST
Does it help if you wait a short while between clearing the tables and removing the modules?
Comment 30 Jiri Popelka 2013-12-12 10:25:54 EST
No, it's not a matter of time, it's a dependency thing.
We sort the modules before unloading, to unload leaves first.

# lsmod | grep nf_conntrack_ipv6
nf_conntrack_ipv6      18738  1 
nf_defrag_ipv6         34595  1 nf_conntrack_ipv6
nf_conntrack           86430  6 nf_nat,nf_nat_ipv4,nf_nat_ipv6,iptable_nat,nf_conntrack_ipv4,nf_conntrack_ipv6

shows that nf_conntrack_ipv6 is used by some other module but doesn't state by what. I've discovered that it's nf_nat_ipv6 - problem is that it's stated as a dependency of nf_conntrack and not nf_conntrack_ipv6. Previously 'modprobe -r' recursively removed nf_nat_ipv6 before trying to remove nf_conntrack_ipv6 so we had not seen this problem.
Comment 31 Laine Stump 2013-12-12 10:40:35 EST
Just how much is the "small but measurable" performance impact of leaving ebtables loaded anyway? Has that been re-measured in recent times?
Comment 32 Jiri Benc 2013-12-12 11:03:32 EST
(In reply to Jiri Popelka from comment #30)
> shows that nf_conntrack_ipv6 is used by some other module but doesn't state
> by what. I've discovered that it's nf_nat_ipv6 - problem is that it's stated
> as a dependency of nf_conntrack and not nf_conntrack_ipv6. Previously
> 'modprobe -r' recursively removed nf_nat_ipv6 before trying to remove
> nf_conntrack_ipv6 so we had not seen this problem.

I see. Two passes indeed solve that, then. Thanks for the explanation!
Comment 34 Jiri Popelka 2014-01-06 09:30:10 EST
*** Bug 1048928 has been marked as a duplicate of this bug. ***
Comment 38 Ludek Smid 2014-06-13 05:17:29 EDT
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Note You need to log in before you can comment on or make changes to this bug.