Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Created attachment 518370[details]
Patch to init script to start memcached as user specified in sysconfig/memcached
Description of problem:
If the 'MAXCONN' variable in /etc/sysconfig/memcached is set to a value greater than 1024 on a stock RHEL 6.1 install, attempting to start memcached will fail with the message:
failed to set rlimit for open files. Try running as root or requesting smaller maxconns value.
However, when the nofile limits for the memcached user are increased in /etc/security/limits.conf like this:
memcached hard nofile 65536
memcached soft nofile 65536
The problem continues. Looking closer, there is an avc denial logged:
Aug 15 13:25:26 cache101 audispd: node=cache101 type=AVC msg=audit(1313439926.993:45010): avc: denied { sys_resource } for pid=19815 comm="memcached" capability=24 scontext=unconfined_u:system_r:memcached_t:s0 tcontext=unconfined_u:system_r:memcached_t:s0 tclass=capability
And here's the actual failing syscall (from strace):
[pid 24837] getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
[pid 24837] setrlimit(RLIMIT_NOFILE, {rlim_cur=4*1024, rlim_max=4*1024}) = -1 EPERM (Operation not permitted)
[pid 24837] write(2, "failed to set rlimit for open files. Try running as root or requesting smaller maxconns value.\n", 95failed to set rlimit for open files. Try running as root or requesting smaller maxconns value.
) = 95
A quick look at memcached.c and the included init script confirm that redhat starts the memcached binary as root, and that memcached does a setuid() immediately after the failing setrlimit. The problem here is that since it is started as root, the setrlimit is run as root, and since root's nofiles limit is still 1024, it fails. in the absence of SELinux it would normally succeed, by (ab)using the CAP_SYS_RESOURCE capability to override limits since it's a root owned process. My first thought was just to reloacte the setuid() call before the setrlimit() calls, but since limits.conf is only sourced on a new session, this doesn't help (the setuid() doesn't create a new session, so nofile is still set to 1024).
One possible fix would be to allow memcached the sys_resource capability in the selinux policy, but this seems to be the wrong way to go in general. I'm not entirely clear on RedHat's stance on policy changes, but I would think granting an application additional access that is technically doesn't need would be frowned upon.
A simple fix is adding nofiles entries to limits.conf for root, which is what we've done as a workaround. This works fine, but is unintuitive and likely to confuse some future admin. If this is the correct fix, I'd hope to at least see it documented somewhere.
One other option is simply starting the memcached program as the user specified in /etc/sysconfig/memcached. This requires a very simple change to the init script, and at least with the stock config I don't see any problems with it. If a user wanted to bind memcached to a non-ephemeral port this would be a problem, but that would require selinux policy changes anyway and so changing that may not be a big deal.
I've attached the (1 line) patch described above as a possible fix, and would like to know what Redhat's recommended long-term fix to this is.
Version-Release number of selected component (if applicable):
1.4.4-3.el6
How reproducible:
Always
Steps to Reproduce:
1. add nofiles entries to limits.conf for memcached user
2. increase MAXCONN in /etc/sysconfig/memcached to "4096"
3. attempt to (re)start memcached
Actual results:
failed to set rlimit for open files. Try running as root or requesting smaller maxconns value.
[ FAILED ]
Expected results:
[ OK ]
Additional info:
Comment 2RHEL Program Management
2011-08-15 23:47:53 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unfortunately unable to
address this request at this time. Red Hat invites you to
ask your support representative to propose this request, if
appropriate and relevant, in the next release of Red Hat
Enterprise Linux. If you would like it considered as an
exception in the current release, please ask your support
representative.
Perhaps a new sysconfig option could be introduced for this configuration, to set the user under which memcached should be started. But allowing the call in selinux seems like a cleaner fix to me.
Paul, what do you think?
We encountered the same problem on RHEL 6.0 with SELinux enabled.
Adding the following SELinux policy works for us:
require {
type memcached_t;
class capability sys_resource;
}
#============= memcached_t ==============
allow memcached_t self:capability sys_resource;
Maybe an sebool value would be practical to enable such elevated privileges?
allow_memcached_maxconn --> on (default: off)
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
http://rhn.redhat.com/errata/RHBA-2011-1511.html
Created attachment 518370 [details] Patch to init script to start memcached as user specified in sysconfig/memcached Description of problem: If the 'MAXCONN' variable in /etc/sysconfig/memcached is set to a value greater than 1024 on a stock RHEL 6.1 install, attempting to start memcached will fail with the message: failed to set rlimit for open files. Try running as root or requesting smaller maxconns value. However, when the nofile limits for the memcached user are increased in /etc/security/limits.conf like this: memcached hard nofile 65536 memcached soft nofile 65536 The problem continues. Looking closer, there is an avc denial logged: Aug 15 13:25:26 cache101 audispd: node=cache101 type=AVC msg=audit(1313439926.993:45010): avc: denied { sys_resource } for pid=19815 comm="memcached" capability=24 scontext=unconfined_u:system_r:memcached_t:s0 tcontext=unconfined_u:system_r:memcached_t:s0 tclass=capability And here's the actual failing syscall (from strace): [pid 24837] getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0 [pid 24837] setrlimit(RLIMIT_NOFILE, {rlim_cur=4*1024, rlim_max=4*1024}) = -1 EPERM (Operation not permitted) [pid 24837] write(2, "failed to set rlimit for open files. Try running as root or requesting smaller maxconns value.\n", 95failed to set rlimit for open files. Try running as root or requesting smaller maxconns value. ) = 95 A quick look at memcached.c and the included init script confirm that redhat starts the memcached binary as root, and that memcached does a setuid() immediately after the failing setrlimit. The problem here is that since it is started as root, the setrlimit is run as root, and since root's nofiles limit is still 1024, it fails. in the absence of SELinux it would normally succeed, by (ab)using the CAP_SYS_RESOURCE capability to override limits since it's a root owned process. My first thought was just to reloacte the setuid() call before the setrlimit() calls, but since limits.conf is only sourced on a new session, this doesn't help (the setuid() doesn't create a new session, so nofile is still set to 1024). One possible fix would be to allow memcached the sys_resource capability in the selinux policy, but this seems to be the wrong way to go in general. I'm not entirely clear on RedHat's stance on policy changes, but I would think granting an application additional access that is technically doesn't need would be frowned upon. A simple fix is adding nofiles entries to limits.conf for root, which is what we've done as a workaround. This works fine, but is unintuitive and likely to confuse some future admin. If this is the correct fix, I'd hope to at least see it documented somewhere. One other option is simply starting the memcached program as the user specified in /etc/sysconfig/memcached. This requires a very simple change to the init script, and at least with the stock config I don't see any problems with it. If a user wanted to bind memcached to a non-ephemeral port this would be a problem, but that would require selinux policy changes anyway and so changing that may not be a big deal. I've attached the (1 line) patch described above as a possible fix, and would like to know what Redhat's recommended long-term fix to this is. Version-Release number of selected component (if applicable): 1.4.4-3.el6 How reproducible: Always Steps to Reproduce: 1. add nofiles entries to limits.conf for memcached user 2. increase MAXCONN in /etc/sysconfig/memcached to "4096" 3. attempt to (re)start memcached Actual results: failed to set rlimit for open files. Try running as root or requesting smaller maxconns value. [ FAILED ] Expected results: [ OK ] Additional info: