Bug 1364332 - systemd --user does not load limits from /etc/security/limits.d/
Summary: systemd --user does not load limits from /etc/security/limits.d/
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 26
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1366880 1377844 1607628 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-05 06:10 UTC by Sandro Bonazzola
Modified: 2019-05-04 16:53 UTC (History)
35 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1607628 (view as bug list)
Environment:
Last Closed: 2018-05-29 11:55:20 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Sandro Bonazzola 2016-08-05 06:10:44 UTC
Description of problem:
Looks like the pam configuration for gdm password login is ignoring system defined limits.

Version-Release number of selected component (if applicable):
gdm-3.20.1-3.fc24.x86_64

How reproducible:
100%

Steps to Reproduce:
1. create /etc/security/limits.d/10-nofile.conf with content:
<replace with your user> hard nofile 10240
<replace with your user> soft nofile 10240
2. login with your user

Actual results:
$ ulimit -n
1024

Expected results:
$ ulimit -n
10240

Additional info:
Note that after login with gdm with your user:
$ su - <replace with your user>
$ ulimit -n
10240

So the pam modules for su are reading correctly the limits.
Setting severity to medium since a workaround exist ( su - ) but really annoying.

Comment 1 Nicholas Miell 2016-08-05 21:47:32 UTC
I also noticed this problem.

Running

trace-cmd record -e syscalls:sys_enter_setrlimit -f 'resource == 4' -e syscalls:sys_enter_prlimit64 -f 'resource == 4'

during the login sequence results in the following report:

gdm-session-wor-3552  [000]   580.980904: sys_enter_setrlimit:  resource: 0x00000004, rlim: 0x7ffe1819f030

which suggests that gdm is definitely at fault because nothing else is modifying the limit afterwards.

Comment 2 Ray Strode [halfline] 2016-08-08 16:36:17 UTC
i wonder if this is a case of dbus activated things getting the wrong limits since dbus-daemon is started by systemd --user.

Does modifying the DefaultLimit*= options in /etc/systemd/user.conf fix it ?

Comment 3 Nicholas Miell 2016-08-08 17:09:55 UTC
No.

The limits from /etc/systemd/user.conf get applied, except whatever does the applying doesn't have sufficient privileges to raise the hard limit.

And I also tested with modifications to /etc/security/limits.conf that decreases the nofiles limit to 1023/4095, which shouldn't require any privileges yet still doesn't get applied.

Comment 4 Nicholas Miell 2016-08-08 19:42:43 UTC
Oh, you meant how gnome-terminal-server.service is dbus activated.

Yeah, that appears to be it -- everything started directly from gnome-shell gets the proper limits.

Comment 5 Ray Strode [halfline] 2016-08-08 20:05:34 UTC
So I think a config file in one or more of /etc/systemd/system.conf.d /etc/systemd/user.conf.d and /etc/systemd/system/user@.service.d
should fix this for you, but I'm not entirely sure.  

Maybe one the systemd guys can chime in.

Maybe /etc/pam.d/systemd-user should have pam_limits in it, not sure (but my gut says no).

Are you okay if this gets closed NOTABUG ?

I don't think GDM's really involved here, regardless, so reassigning.

Comment 6 Jóhann B. Guðmundsson 2016-08-08 20:31:06 UTC
Limits are configured within systemd itself not from /etc/security/limits* so if you want to increase limits in type services you need to do so by copying the type service file into the /etc/systemd/system/ directory and edit the copy or create an overwrite snippet like so.

/etc/systemd/system/gnome-terminal-server.service/limit.conf
[Service]
LimitNOFILE=10240

Try the above and see if that works for you.

Comment 7 Jóhann B. Guðmundsson 2016-08-08 20:32:29 UTC
The path will need to be ( forget the .d )
/etc/systemd/system/gnome-terminal-server.service.d/limit.conf

Comment 8 Jóhann B. Guðmundsson 2016-08-08 20:33:10 UTC
(In reply to Jóhann B. Guðmundsson from comment #7)
> The path will need to be ( forget the .d )
> /etc/systemd/system/gnome-terminal-server.service.d/limit.conf

I mean forgot the .d ;)

Comment 9 Sandro Bonazzola 2016-08-09 06:33:35 UTC
I created the file, rebooted (since login / logout didn't change anything)
But still:
$ ulimit -n
1024

Also, looks really weird that a user limit in a terminal has to be set from a systemd limit.
Note that the solution proposed in comments #6 to #8 will apply the limit to all desktop users, while the limit file is trying to set only for my user.
So, in any case, this solution is not acceptable.

Comment 10 Sandro Bonazzola 2016-08-09 06:34:25 UTC
(In reply to Sandro Bonazzola from comment #9)
> I created the file, rebooted (since login / logout didn't change anything)
> But still:
> $ ulimit -n
> 1024
> 
> Also, looks really weird that a user limit in a terminal has to be set from
> a systemd limit.
> Note that the solution proposed in comments #6 to #8 will apply the limit to
> all desktop users, while the limit file is trying to set only for my user.
> So, in any case, this solution is not acceptable.

sorry, is meant to apply, it's not working.

Comment 11 Jóhann B. Guðmundsson 2016-08-09 08:02:58 UTC
The above was just example to overwrite for the type unit that was mentioned in comment 4. 

So start by create the snippets directory for your user service ( note this only applies to your user no one else )

mkdir /etc/systemd/system/user.d
( If your user uid is something other than 1000 change to that ) 

Create the change snippet
vim /etc/systemd/system/user.d/limit.conf

Have it contain
[Service]
LimitNOFILE=10240

Reload the systemd daemon to pick up the changes
systemctl daemon-reload

logout or reboot

Then open gnome terminal as your logged in user and you should see
$ ulimit -n
10240

If you want it to apply to all users you could use
mkdir /etc/systemd/system/user@.service.d
instead and create the limits.conf file there.

Comment 12 Mark Knoop 2016-08-13 16:04:37 UTC
I have done exactly as recommended in comment 11, however it has no effect.

$ cat /etc/systemd/system/user.d/limit.conf
[Service]
LimitMEMLOCK=4194304

$ cat /etc/security/limits.d/95-jack.conf
# Default limits for users of jack-audio-connection-kit

@jackuser - rtprio 70
@jackuser - memlock 4194304

$ id -u
1000

In gnome-terminal:
$ ulimit -l
64

In xterm (started from Gnome Shell):
$ ulimit -l
4194304

Note this breaks using jackd from gnome-terminal.

Perhaps you could point to some documentation about the use of the /etc/systemd/system/user.d/ config directory - I couldn't find this directory referenced in any of the (161!!) systemd man pages.

Also, re comment 5, the pam.d file would seem to already include pam_limits.so, via the inclusion of system-auth. Not sure why it doesn't do anything...

Comment 13 Nicholas Miell 2016-08-13 16:11:30 UTC
(In reply to Jóhann B. Guðmundsson from comment #6)
> Limits are configured within systemd itself not from /etc/security/limits*
> so if you want to increase limits in type services you need to do so by
> copying the type service file into the /etc/systemd/system/ directory and
> edit the copy or create an overwrite snippet like so.

systemd can configure limits for everybody or for a specific user, but not for groups. This is a regression in functionality from pam_limits.

Comment 14 Mark Knoop 2016-08-13 16:37:12 UTC
Aha, this works:

$ cat ~/.config/systemd/user/gnome-terminal-server.service.d/limits.conf
[Service]
LimitRTPRIO=70
LimitMEMLOCK=4194304

So presumably something must be checking with pam_limits otherwise users could assign themselves limits above their entitlement.

Comment 15 Mark Knoop 2016-08-15 08:28:57 UTC
Strangely, this technique does not seem to work for the open files limit though.

$ cat ~/.config/systemd/user/gnome-terminal-server.service.d/limits.conf
[Service]
LimitRTPRIO=70
LimitMEMLOCK=4G
LimitNOFILE=100000

with corresponding entries in /etc/security/limits.d/ files, results in

$ ulimit -n
4096

in gnome-terminal, but

$ ulimit -n
100000

in xterm. 

Is this then a bug, or is something else inter-mediating the open files limit?

Comment 16 Jóhann B. Guðmundsson 2016-08-15 11:39:40 UTC
Systemd does not use /etc/security/limits configurations or files so trying to set limits there for something that uses systemd wont work hence comparing something that uses systemd like gnome terminal to something that does not like xterm ( unless it has been configured to do so ) is like comparing apple and oranges as in whatever limit you configure or set that uses systemd and it's units wont work for anything that uses /etc/security/limits and whatever limits you configure or set in /etc/security/limits wont work for anything that uses systemd.

That said if your limits dont get applied then you aren't configuring the correct type units like in the case of jack mention in comment 12 there you probably need to add the limits to the jack service itself like ~/.config/systemd/user/jack.service.d/limits.conf or something similar

The current documentation for this can be found here for system [1] and here for user [2] and how you should overwrite or configuring this for upstream projects should be documented there so for anything Gnome related it should be documented there on how to set this or for jack it should be documented there how to set this etc. 



1. https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html
2. https://www.freedesktop.org/software/systemd/man/systemd-user.conf.html

Comment 17 Mark Knoop 2016-08-15 12:33:29 UTC
Thanks Jóhann. Regardless of the comparison with /etc/security/limits and xterm, could you explain why this file:

$ cat ~/.config/systemd/user/gnome-terminal-server.service.d/limits.conf
[Service]
LimitRTPRIO=70
LimitMEMLOCK=4G
LimitNOFILE=100000

is successful in setting the real-time priority and max locked memory limits available in gnome-terminal, but has no effect on the open files limit?

And also why the method you recommended in comment 11 has no effect at all for both me and Sandro?

Comment 18 Ray Strode [halfline] 2016-08-15 14:10:49 UTC
my guess (just a guess) is that user services can't set their limits higher than the hard limit of the systemd --user instance they're running from.  This makes sense, since systemd --user is run unprivileged.  So you probably need to have (in addition to that gnome-terminal-server.service.d entry), also an entry to raise the hard limit of /lib/systemd/system/user@.service ala comment 11.

or maybe just fudge DefaultLimit* In /etc/systemd/user.conf and /etc/systemd/system.conf to do it for everything across the board.

Comment 19 Jóhann B. Guðmundsson 2016-08-15 15:11:33 UTC
Sandro has not mentioned if comment 11 did work or not work for him and is the user id of your "jack user" 1000? 

I'll will need to test adding those lines to the user configuration limit I created to see if they get added for the user or not.

If you are trying to configure jack for your user session with your vm of choice you dont fiddle with gnome-terminal-server unit you do so in the jack(d) unit for your user so that service gets started when the user logs into his session unless ofcourse there might be some legit reason for you wanting to run jack(d) from the command line within a gnome terminal session but all those that I'm aware of either have jackd.service or jack.service ( JACK 2 dbus user session service  ) for their user in their vm of choice but the same logic should apply as Ray pointed out a) first set the limit in @user for your user then in that users relevant system/session units.

Comment 20 Mark Knoop 2016-08-15 18:42:43 UTC
(In reply to Jóhann B. Guðmundsson from comment #19)
> is the user id of your "jack user" 1000?

Yes.

> I'll will need to test adding those lines to the user configuration limit I
> created to see if they get added for the user or not.

$ id -u
1000
$ systemctl show user | egrep 'Path|NOFILE'
LimitNOFILE=100000
LimitNOFILESoft=100000
FragmentPath=/usr/lib/systemd/system/user@.service
DropInPaths=/etc/systemd/system/user.d/limit.conf
$ ulimit -n
1024

> If you are trying to configure jack for your user session with your vm of
> choice you dont fiddle with gnome-terminal-server unit you do so in the
> jack(d) unit for your user so that service gets started when the user logs
> into his session unless ofcourse there might be some legit reason for you
> wanting to run jack(d) from the command line within a gnome terminal session
> but all those that I'm aware of either have jackd.service or jack.service (
> JACK 2 dbus user session service  ) for their user in their vm of choice but
> the same logic should apply as Ray pointed out a) first set the limit in
> @user for your user then in that users relevant system/session units.

AFAIK there are no jackd.service or jack.service systemd files in the fedora jack-audio-connection-kit packages.

Comment 21 Ray Strode [halfline] 2016-08-23 14:48:35 UTC
*** Bug 1366880 has been marked as a duplicate of this bug. ***

Comment 22 Tomas Mraz 2016-08-23 14:55:49 UTC
Why does the systemd apply the limits to processes within user session?

Comment 23 Ray Strode [halfline] 2016-08-23 15:00:56 UTC
it doesn't. it applies the limits of processes started by systemd --user (which includes gnome-terminal). These aren't run within the PAM session

Comment 24 Tomas Mraz 2016-08-23 16:16:30 UTC
Ah, so these are not the children of the PAM session? But then why the hard limit change was applied in bug 1366880?

Comment 25 Ray Strode [halfline] 2016-08-23 16:37:00 UTC
not exactly sure, i'll let the systemd people answer.

I do see:

session include system-auth

in /etc/pam.d/systemd-user

and

PAMName=systemd-user

in /lib/systemd/system/user@.service

So I guess the systemd --user session does get run through pam_limits, but then the soft limits are reset based on user.conf

Comment 26 Orcan Ogetbil 2016-09-09 03:06:11 UTC
Does this bug imply that we are shipping a broken Gnome+JACK combination (I don't use Gnome, I can't tell)? If yes, this is a big regression.

What is the easiest workaround?

BTW possible duplicate: bug #1369197

Comment 27 J. Bruce Fields 2016-09-09 15:15:15 UTC
(In reply to Orcan Ogetbil from comment #26)
> Does this bug imply that we are shipping a broken Gnome+JACK combination (I
> don't use Gnome, I can't tell)?

Yes: I found that with the defaults (including using Gnome), and just following documentation (add self to jackuser, etc.), jack was still unable to lock memory and get realtime priority on startup.

Comment 28 Nicholas Miell 2016-09-09 15:21:50 UTC
JACK isn't D-Bus activated, though, so I don't see how it could be affected.

Comment 29 Mark Knoop 2016-09-09 15:29:36 UTC
(In reply to Nicholas Miell from comment #28)
> JACK isn't D-Bus activated, though, so I don't see how it could be affected.

It's affected if you start it in gnome-terminal. It's not if you start it with QJackCtl or in xterm. Which exemplifies why this "feature" of systemd-user ignoring system limits is a disaster for documentation and debugging.

Comment 30 Nicholas Miell 2016-09-09 15:33:43 UTC
Which means the GNOME+JACK combination isn't broken in the normal case

Comment 31 Mark Knoop 2016-09-09 15:47:00 UTC
(In reply to Nicholas Miell from comment #30)
> Which means the GNOME+JACK combination isn't broken in the normal case

Who/what/where defines "the normal case"?

Comment 32 J. Bruce Fields 2016-09-09 16:22:10 UTC
For what it's worth, I was starting qjackctl from gnome-terminal.  So I guess that's why I ran into this.

In all the documentation I've seen (for example, /usr/share/doc/jack-audio-connection-kit/README.Fedora), examples show jack and qjackctl started from the command line.  And given Fedora defaults that likely means gnome-terminal.

More generally: I do almost all of my work from gnome-terminal.  Given that a lot of Fedora users are heavy command line users, and that gnome-terminal is the default terminal app, I'm probably not unusual.  If it's really intended that gnome-terminal (partially) ignore the settings in limits.conf, then at a minimum that needs to be documented (in limits.conf(5), for a start).

Comment 33 Orcan Ogetbil 2016-09-10 01:48:52 UTC
Sorry folks, it's been ages since I last used GNOME. I admit I am ignorant about all things GNOME, e.g. the difference between gnome-terminal, gnome-shell, and pure "gnome". Are you folks saying that gnome-terminal (whatever that is) is behaving differently than the other ones?

Comment 34 Tomas Mraz 2016-09-12 09:14:04 UTC
(In reply to J. Bruce Fields from comment #32)
> If it's really
> intended that gnome-terminal (partially) ignore the settings in limits.conf,
> then at a minimum that needs to be documented (in limits.conf(5), for a
> start).

No, I would consider this to be a bug. Unless we officially declare limits.conf as unsupported/deprecated and remove its usage, together with removing pam_limits from the default PAM configuration, I would say this is clearly a bug on the gnome/systemd side. And I am not going to add stupid workarounds like documenting that some software is buggy into a configuration file.

Would it be really really so hard to either call pam with pam_limits for the gnome-terminal sessions or parse limits.conf on its own in the systemd?

Comment 35 Orcan Ogetbil 2016-09-22 03:06:17 UTC
*** Bug 1377844 has been marked as a duplicate of this bug. ***

Comment 36 Christian Stadelmann 2016-09-23 10:48:04 UTC
I've been running into this bug pretty often lately causing many crashes with applications being unable to work due to the file limit.

(In reply to Jóhann B. Guðmundsson from comment #6)
> Limits are configured within systemd itself not from /etc/security/limits*

So that's a bug in systemd which should be fixed. If you want to keep it this way, make /etc/security/limits* go away please. I don't care which one stays, but having two places for the same configuration options is error-prone.

(In reply to Tomas Mraz from comment #34)
> (In reply to J. Bruce Fields from comment #32)
> > If it's really
> > intended that gnome-terminal (partially) ignore the settings in limits.conf,
> > then at a minimum that needs to be documented (in limits.conf(5), for a
> > start).
> 
> No, I would consider this to be a bug. Unless we officially declare
> limits.conf as unsupported/deprecated and remove its usage, together with
> removing pam_limits from the default PAM configuration, I would say this is
> clearly a bug on the gnome/systemd side. And I am not going to add stupid
> workarounds like documenting that some software is buggy into a
> configuration file.

+1

Comment 37 Jóhann B. Guðmundsson 2016-09-23 12:25:53 UTC
It's is not a bug in systemd that systemd has chosen not read or use limits.conf since it itself has no need for that file. It's also not a bug for other upstream to use what systemd provides but it's up to those other upstreams to ensure backwards compatibility and or provide migration path, update documentation etc has it previously used limits.conf.

Comment 38 Tomas Mraz 2016-09-23 12:40:14 UTC
It might not be a bug in systemd upstream, it is a bug of Fedora. Which component is decided to be the place to fix it is not relevant to the fact that the current behavior is a bug.

Comment 39 Jóhann B. Guðmundsson 2016-09-26 03:00:34 UTC
I'm not sure what you are trying to convey here. 

It's not a bug in systemd, it's not a bug in other upstream components that chose to use features of systemd. It's not a bug in Fedora since the feature process a) is just a marketing stunt b) does not work even if it was not an marketing stunt so package maintainers skip it/ignore it and c) is not obligatory if it would actually work.
If the Gnome ( or anyone else for that matter ) package maintainers in the distribution would have documented these changes ( which they might already have done ) nobody would have a) read those b) more importantly tested it so the only way for anyone to avoid any kind of surprises is to follow upstream closely *always*. 

Fedora ( to certain extent ) and all the upstream components that make up the distribution is built upon volunteering effort which means all you can expect is best effort from anybody based on whatever time they have to contribute anywhere ( in the downstream distribution or upstream community surrounding each component ).

This based on observation after spending what 8 years of my live contributing my free time to Fedora having to deal with Red Hat, it's employees and effects it has on the distribution and whatever Red Hat want to thrust upon the community for early adoption each cycle to see what sticks, broken distribution bureaucracy processes ( and processes in general ). Under volunteered to be effective enough to maintain an distribution of it's size ( Fedora can handle at best being at 5k - 7k package size ). All that package crammed into too short release cycle which seem to be aligned with single desktop environment release ( which does not work ). After doing countless testing and sitting countless release readiness meeting through those 8 years it never seized to amaze me that reoccurring miracle that was a functioning Fedora release.

In the end of the day upstream/downstream does not have anymore time to provide documentation about changes ( let alone provide backwards compatibility ) then users have time to follow those changes so if you want something done you have to do it yourself and if you dont want to be caught by surprises follow upstream and downstream ( if it deviates from upstream ) closely.

That's how the entire linux opensource ecosystem works and Fedora is no exception to that.

Comment 40 Christian Stadelmann 2016-09-26 09:38:17 UTC
Ok, back from the rant to this bug report:

This report is about an inconsistency with pam's limits and systemd's limits. I had hoped that there could be one single place to change stuff like process limits, but it looks like that we won't see that happen.

How about having both man pages provide a hint to the other configuration option? 

Short-term "solution":
Since /etc/security/limits* is the older config option, how about adding a README file into this directory and a note in limits.conf so that people know systemd has its own limits option(s)?

Real solution:
On the long term, one of both solutions should go away imho. Having two options causes situations like above with JACK where depending on how the process started it has different limits (albeit running as same user). Removing this feature from pam or systemd should be no problem because they are mostly redundant.

Comment 41 Jóhann B. Guðmundsson 2016-09-26 11:40:32 UTC
Well systemd has never used /etc/security/limits* and the single cross distribution place to do just that is systemd at least for anything related to type systemd units,sessions and scopes. 

The short term solution you propose will not work since you cannot maintain an include/exclude list which consist of up to 14k components and re-direct users to systemd documentation should their component be included in that list and you cannot redirect people without such list until everybody have started to use the systemd solution like for example in this case Gnome community has decided to use systemd for this while I'm pretty sure KDE community has not.

That obviously tells you that Gnome maintainers in the distribution should have taken better care of documenting these changes ( or provide that backwards compatibility being asked for since it most likely exist anyway upstream for those distribution that do not use systemd and or offer systemd as one of many init systemd to use like Debian does ) and let everyone know that they were no longer using /etc/security/limits* as they probably will have to do for those upcoming changes for user environment and environment file handling as well ( which will probably cause even more confusion and end user explosion than this, if handle incorrectly) and provide documentation on how to configure this for GDM and anything started in Gnome terminal etc.

The Red Hat desktop team ( aka Gnome half of it anyway) get's away with doing anything in this distribution so dont get your hopes up they get their act together now after what ten years since they do whatever they want, whenever they want regardless how it affects the distribution or other components which they do not maintain themselves ( including dropping maintenance on components they previously maintain quarter to release which does not allow for other desktop environments and or maintainers to pick up the maintenance and or adapt to it being obsoleted ).

With regards to pam, pam itself has no future and needs to be rewritten or obsoleted by something else, better. Everyone knows this but nobody is willing step up to do the necessary leg work to either rewrite pam as pam2 or introduce something new to replace it ( things have not escalated to the point that has forced someone to do that but it's slowly getting there ) and I would not be surprised if pam itself is just in maintenance mode so it wont accept any new features or invasive changes.

Comment 42 Orcan Ogetbil 2016-09-27 23:32:24 UTC
What is the proposed solution for downstream packages, like jack? 

We currently include /etc/security/limits.d/95-jack.conf
Should we ship another limits file for systemd?

Comment 43 Jóhann B. Guðmundsson 2016-09-28 04:33:41 UTC
Based on what I have gathered about jackd and what comes logically to me after doing so end testing various setups what I recommend for you is to create a type user service unit and use that as in.. 

# vim /etc/systemd/user/jack.service
[Unit]
Description=JACK Audio Connection Kit
Documentation=man:jackd(1)
After=sound.target

[Service]
LimitRTPRIO=70
LimitMEMLOCK=4194304
ExecStart=/usr/bin/jackd -r -d alsa -r 44100

# systemctl daemon-reload

Then *any* end users can just run the following from a ( Gnome )terminal...
$ systemctl --user start jack.service
$ systemctl --user status jack.service
$ systemctl --user stop jack.service
$ systemctl --user enable jack.service
$ systemctl --user disable jack.service

Then I as an administrator/end user would expect that I would have to increase just the limits in the type user service unit or the default soft/hard limits in /etc/systemd/user.conf or /etc/systemd/system.conf should I need to increase those for anything that uses systemd but as has been pointed out and I have tested thus far that the "limits" seem to be broken and buggy in many places.

I have no idea if using systemd unit is acceptable from jack user stand point and implementation since I'm not a jack user but the service started and stopped cleanly as an regular user from a gnome terminal and I could run jack_lsp in another terminal window and it successfully contact the jackd server.

But there exist no packaging guideline that I'm aware of that handles user type unit in the distribution and it's going to take time to get such changes through FPC based on personal experience. 

In anycase I've been at this for 6 hours now and have to get up after 4hours for dayjob but I can give a more detail explanation if required later why this approach is better than many others at least this seems to be the most painless approach for maintainer(s) and from end user usage perspective for the jackd component.

Comment 44 Benjamin Xiao 2016-09-28 06:59:36 UTC
This won't work with things like alsa_in and alsa_out, which are command line utilities to add additional sound devices into the current JACK session. Putting these command line utilities in a service file will be extremely not user friendly.

Plus currently, it isn't the JACK service that's running into resource limit issues because we aren't starting JACK in the terminal to begin with. We use QJackCtl from the Gnome Menu to start JACK and that works properly with the currently 95-jack.conf file.

Honestly, there should be a way to set resource limits in Gnome Terminal properly. The problem isn't with JACK and how its packaged. It's Gnome Terminal breaking existing configuration options. Rather than fixing every package that requires different resource limits, fix the root of the problem.

Also why does Gnome Terminal get proper resource limits when selinux is permissive, but not when its enforcing?

Comment 45 J. Bruce Fields 2016-09-28 15:02:38 UTC
(In reply to Benjamin Xiao from comment #44)
> This won't work with things like alsa_in and alsa_out, which are command
> line utilities to add additional sound devices into the current JACK
> session. Putting these command line utilities in a service file will be
> extremely not user friendly.

Or the various soft synths--anything that processes sound in real time needs these limits raised, but we probably don't want to make unit files for all of them.

> Plus currently, it isn't the JACK service that's running into resource limit
> issues because we aren't starting JACK in the terminal to begin with.

For what it's worth, I have been starting it from the terminal, in part because that's what's documented (for example, in /usr/share/doc/jack-audio-connection-kit/README.Fedora).

> Also why does Gnome Terminal get proper resource limits when selinux is
> permissive, but not when its enforcing?

Also I don't think there's been an explanation of why only soft limits are affected?  (At least in my case hard limits were still those from /etc/security/limits.d/.)

Comment 46 Kevin Perros 2016-10-08 14:13:15 UTC
Hi,

I had the same problem with Fedora 24, not with Fedora23.

I tried to create limits file :
~/.config/systemd/user/gnome-terminal-server.service.d/limits.conf
~/.config/systemd/user/display-manager.service.d/limits.conf
/etc/systemd/user/gnome-terminal-server.service.d/limits.conf
/etc/systemd/user/display-manager.service.d/limits.conf

I also changed the default limits in /etc/systemd/user.conf

It only worked for the locked memory limit, but the rtprio was limited to 20 (against 0 before).

Then I created /etc/systemd/system/user.d/limit.conf with the proper limits inside.

Now I can launch Ardour4 from a gnome terminal, and it has RT capabilities.

the /proc/xxx/limits is:
Max locked memory         4294967296           4294967296           bytes
Max realtime priority     95                   95

However when I launch Ardour4 from the menu, it does not inherit the RT capabilities:

the /proc/xxx/limits is:
Max locked memory         4294967296           4294967296           bytes
Max realtime priority     20                   20

My qjackctl is launched automaticaly on gnome startup, also has limits:
Max locked memory         4294967296           4294967296           bytes
Max realtime priority     20                   20

I understand that /etc/systemd/system/user.d/limit.conf is enforced because I also created limit files /..../gnome-terminal-server.service.d/limits.conf and that I should do the same thing for gnome-shell. BUT I don't see a configuration file for a gnome-shell service.

Anyway, Jack is an RT app and it should be setup with RT caps when the RPM is installed, without having to mess with any config file. Those config files keep changing all the time and are very hard to cope with when configuring his distro is not a hobby.

Comment 47 Kevin Perros 2016-10-08 17:03:43 UTC
I have investigated a bit further, trying to understand a bit how systemd works (I previsously used a 5+ years old 32 bits distro :).

So here is a summurized hierarchy, when I launch qjackctl from a gnome terminal and ardour4 from the gnome menu:
[kevin@localhost ~]$ systemd-cgls
Control group /:
-.slice
├─init.scope
...
├─system.slice
...
└─user.slice
  ├─user-1000.slice
  │ ├─user
  │ │ ├─gnome-terminal-server.service
  │ │ │ ├─1997 /usr/libexec/gnome-terminal-server
  │ │ │ ├─2003 bash
  │ │ │ ├─2074 bash
  │ │ │ ├─2109 qjackctl
  │ │ │ ├─2127 systemd-cgls
  │ │ │ └─2128 less
...
  │ └─session-2.scope
  │   ├─1368 gdm-session-worker [pam/gdm-password]
...
  │   └─1949 /usr/lib64/ardour4/ardour-4.6.387
  └─user-42.slice
    ├─session-c1.scope
    │ ├─1027 gdm-session-worker [pam/gdm-launch-environment]
...

With the configuration in my previous comment (i.e. a /etc/systemd/system/user.d/limits.conf file), my qjackctl inherits the desired resource capabilities. My ardour4 does not. ardour4 appears in the "session-2.scope" subtree and qjackctl appears in the "session-2.scope" subtree.

So the limits are enforced for the user subtree not for the session-2.scope subtree.

By the way, systemd could be a great improvement if I could only give RT/memlock capabilities to jack/ardour. I don't like much that every app I use have those caps, especially those accessing the internet.

Comment 48 Illtud Daniel 2016-10-10 22:06:26 UTC
Took me a whole lot of searching to find this bug. I can't raise my user open file limits to anything approaching useful for the kind of work I do.

I've set the limits in the places suggested for systemctl:
[user@vaio ~]$ id -u
1000
[user@vaio ~]$ systemctl show user | egrep 'Path|NOFILE'
LimitNOFILE=65536
LimitNOFILESoft=65536
[user@vaio ~]$ systemctl show gnome-terminal-server.service | egrep 'Path|NOFILE'
LimitNOFILE=65536
LimitNOFILESoft=65536

but I'm still stuck at 4096
[user@vaio ~]$ strace -o /tmp/foo ulimit -n 5000 
/usr/bin/ulimit: line 2: ulimit: open files: cannot modify limit: Operation not permitted
[user@vaio ~]$ grep rlimit /tmp/foo
getrlimit(RLIMIT_NPROC, {rlim_cur=31382, rlim_max=31382}) = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=4*1024, rlim_max=4*1024}) = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=4*1024, rlim_max=4*1024}) = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=4*1024, rlim_max=4*1024}) = 0
setrlimit(RLIMIT_NOFILE, {rlim_cur=5000, rlim_max=5000}) = -1 EPERM (Operation not permitted)

I've set higher limits in limits.conf, although it now appears that's ignored with gnome-terminal.

Can *anybody* successfully raise a user's NOFILE limit above 4096 in FC24? If so, can anybody share the secret with me?

Comment 49 Zbigniew Jędrzejewski-Szmek 2016-10-18 17:08:47 UTC
This issue is most likely confounded by #1341829, which essentially causes some rlimit settings set by systemd (PID 1) to be ignored.

Comment 50 Kevin Perros 2016-10-18 17:35:45 UTC
I am no expert about the case, but I have read a bit the docs.

I think that everything that appears in the subtree user I detailed has the proper limits set, but that the bug #1341829 might cause part of them not to be set. It is because a slice is user configurable, according to the man.

The subtree session-2.scope does not have the proper limits. A scope is *NOT* user configurable. It is only configurable programatically, according to the man. So the application that launches that scope should set properly the limits. I suppose that app is gdm, maybe gnome-shell. So there is a bug in GDM or gnome-shell. An enormous one, because it prevents the user from launching apps that either needs to manipulate many files, or needs RT caps, or memory locking caps.

Comment 51 Ray Strode [halfline] 2016-10-19 20:34:57 UTC
> The subtree session-2.scope does not have the proper limits. A scope is
> *NOT* user configurable. It is only configurable programatically, according
> to the man. So the application that launches that scope should set properly
> the limits. I suppose that app is gdm, maybe gnome-shell. So there is a bug
> in GDM or gnome-shell.

No, GDM and gnome-shell shouldn't be messing with limits directly.

session-2.scope is created by the pam_open_session call in gdm-session-worker.  the pam session limits are currently managed by pam_limits / /etc/security/limits.conf 

I agree that it's less than optimal to have to set both /etc/security/limits.conf and the user@ service configuration.

Comment 52 Kevin Perros 2016-10-20 08:59:46 UTC
Excuse-me, my english is poor. I don't suggest or support the fact that GDM sets the limits. 

But, in my system, there is a systemd scope named session-2.scope, whose subtree is:
  │ └─session-2.scope
  │   ├─1305 gdm-session-worker [pam/gdm-password]
  │   ├─1331 /usr/bin/gnome-keyring-daemon --daemonize --login
...
  │   ├─1462 /usr/bin/pulseaudio --start --log-target=syslog
...

  │   └─5451 evince /tmp/mozilla_kevin0/SVT4PRO_OM.pdf


The systemd man says that a scope is configured programaticaly, so I understand that there is no way to tell that scope what the limits should be. I believe that GDM is the application that spawns that scope.

In that scope, there is gnome-shell, so every app launched by clicking in gnome shell has non-manageable limits, what is an enormous bug.

There is also pulse-audio, that as a consequence cannot run with RT caps, what is an enormous bug for me who is a musician.

I don't know why but the terminal server is not in that scope, it is in a systemd slice, and a slice is user configurable. So I can use the workaround of launching my RT apps from the terminal.

From a security point of view, having to manage security limits on a per app, per user basis can be a total mess. If there is not a central place (a directory, a file) where the exceptions to the default limits are specified, then the sysadmin will be very prone to have forgotten who and what apps have dangerous limits, and that is a security flaw. However, it is a good thing in theory to manage per apps limits. With such a system root owns the apps, and the user can only use root controlled apps with dangerous limits.

Comment 53 Kevin Perros 2016-10-20 09:01:52 UTC
To go further, a better system could be to give max limits in /etc/security/limits.d and to prevent systemd from giving limits beyond those.

Comment 54 Kevin Perros 2016-10-20 09:04:26 UTC
And by the way, this bug severity is set as medium. It is not, it is a major bug that must be solved.

Comment 55 Jóhann B. Guðmundsson 2016-10-20 09:14:31 UTC
Priorities in an contributed communities are driven by individuals own time schedule in fixing those so you do realize that bug severity has no meaning right? 

The only way for you wanting to have something fixed faster ( at a higher priority or severity ) is if you fix it yourself or you pay someone to fix it for you and personally I cant recommend Red Hat in that regard since we here at dayjob have an open bug against their spacewalk product which we reported on 2014-10-24 and it's still unfixed...

Comment 56 Kevin Perros 2016-10-20 09:31:53 UTC
No, Redhat is a 3 billion USD company, with a 200 millions dollars net income in 2015. The core of GNU/Linux is written by professional developers (in our case systemd and GDM), that do business mostly by selling software and serices. RHEL costs 49$ a year in my country. So I believe that as a server and service seller, redhat does want that the "number of open files" limit works properly, including in GNOME, because it can be useful for young developers.

Comment 57 Jóhann B. Guðmundsson 2016-10-20 10:18:06 UTC
No what? 

This is precisely how it works in community driven project like Fedora ( allegedly ) is as well as the upstream projects the distribution consists of. Peoples priorities are based on their own time schedule and they will fix things if and then when they chose to do so. People contributing their own free time for some cause, project or believes are not slaves to other peoples demands.

And FYI Red Hat despite being a 3 billion USD company, with a 200 millions dollars net income in 2015 have not managed to fix a single bug reported in 2014 in their own satellite product which has been reported through their own ( paid ) support channel to this day. It's track record and own incompetence in supporting and maintaining it's own product speaks for itself ( and that's excluding the ridiculousness of Red Hat certifications and partnership model they have changed to in recent years. It's focus should be entirely on creating foundations which is probably the only thing that company is good at these days ). 

Your believes are irrelevant in this case ( Go attend an Red Hat summit if you want to listen to gospel and buzzwords ) and FYI these changes that you are criticizing. got proposed and implemented by Red Hat employees in this precis manner in which I whole heartily agree with you do not seem to have been thoroughly thought through. 

And this sudden unexploited and breakage for people wait for it wait for it.. got introduced into the distribution by the Gnome half of the Red Hat desktop team which manages the "workstation" product...

Comment 58 Illtud Daniel 2016-10-20 10:23:10 UTC
Ray Strode [halfline] 2016-10-19 16:34:57 EDT said:

> session-2.scope is created by the pam_open_session call in
> gdm-session-worker.  the pam session limits are currently managed by
> pam_limits / /etc/security/limits.conf 
> 
> I agree that it's less than optimal to have to set both
> /etc/security/limits.conf and the user@ service configuration.

...but the issue here is not over convenience, it's over impossibility.

If you've managed to raise the limits using limits.conf & user@ systemd configuration, please give me a HOWTO!

Both Kevin and myself have reported that we've upped the limits in both systemd and limits.conf to no avail! See his comment 46 & my comment 48.

Comment 59 Illtud Daniel 2016-10-20 10:26:27 UTC
Jóhann, agreed that priorities are set by the contributors, but Kevin did not request a change to priority, he requested a change to severity, which is a separate field and completely related to customer impact:

https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity

I would suggest a 'high'

Comment 60 Jóhann B. Guðmundsson 2016-10-20 10:38:21 UTC
Whatever you read there that's only applicable to Red Hat customers but not Fedora. 

If Fedora was running and managing it's own bug tracker things would look quite different in terminology workflow, procedures and processes

Comment 61 Kevin Perros 2016-10-20 13:35:16 UTC
Maybe you're right Johann. I don't know much about the bug management system, as I am new to fedora : what is a redhat bug, a fedora bug, what is fixed in redhat but not in fedora... In that case I understand that the bug is upstream in GDM, maybe I am wrong. I don't know where GDM is taking its limits from.

Daniel, actually I use that configuration (notice that I use the systemd/system direcotry not systemd/user):
sudo cat /etc/systemd/system/user.d/limit.conf 
[Service]
LimitRTPRIO=95
LimitMEMLOCK=4G

The gnome ternminal server being in the user systemd slice, my limits are set properly for applications launched within a gnome terminal. I have read above that people seemed not to be able to set the number of file limit that way... Too lazy to test myself :)

My usecase is qjackctl and ardour4, so RT and memlock.

Comment 62 Benjamin Xiao 2016-10-22 18:38:15 UTC
So are there any plans to eventually unify this? Regardless of which route we take, /etc/security/limits.conf or user.d/limit.conf, I think there should be only one file. It's just weird how for the same user, we get different resource limits depending on whether we launch from Gnome Shell or launch from Gnome Terminal.

Comment 63 Illtud Daniel 2016-12-01 22:08:39 UTC
I can't get my NOFILE limit above 4000 using user & limits.conf.

If anybody out there has managed to get their open file descriptor limits above 4096, *please* post how!

Comment 64 Illtud Daniel 2016-12-02 13:22:36 UTC
I can do it if I 'ssh -X localhost' and run my qgis that way (avoiding gdm), but as you may imagine, it's not that fast. I can't believe that nobody's found a workaround.

Comment 65 Orcan Ogetbil 2016-12-04 01:49:27 UTC
Could not running GDM, gnome-shell be considered a workaround? Maybe Gnome is not the best desktop to do this kind of work (?)

Comment 66 Christian Stadelmann 2017-05-17 13:57:38 UTC
I think one part of issue is fixed in Fedora 26 with systemd-233-3. It hit me by accident and broke my GDM, but I'm glad it works now: Limits from /etc/security/limits.d/ are respected now.

Still, there are two places where limits can be specified: systemd and /etc/security/limits.d/. One of both should be marked as deprecated and go away in the far future.

Comment 67 Fedora End Of Life 2017-07-25 22:15:35 UTC
This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 68 Sandro Bonazzola 2017-07-31 13:59:30 UTC
Confirmed still there in fedora 26

Comment 69 Fedora End Of Life 2018-05-03 08:12:34 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 70 Fedora End Of Life 2018-05-29 11:55:20 UTC
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 71 Stefan Sydow 2018-07-23 20:06:57 UTC
This is still broken in Fedora 28.

Jack and Ardour can't get realtime priority this way.

Comment 72 Yann Droneaud 2018-09-12 17:30:59 UTC
I've opened a similar issue as bug #1628297, for Fedora 29.

Comment 73 Zbigniew Jędrzejewski-Szmek 2019-05-04 16:53:56 UTC
*** Bug 1607628 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.