I am trying to set up GPU access for Nix packages in Home Manager, following https://nix-community.github.io/home-manager/index.xhtml#sec-usage-gpu-non-nixos. The instructions ask me to execute `sudo /nix/store/n16vlkkvvi5ybvvhsq6wjiv3m4snwf27-non-nixos-gpu/bin/non-nixos-gpu-setup`, which is: ``` #!/nix/store/qsydfxm1vq6q9jac2kq3r8kn0xdmsldf-bash-5.3p3/bin/bash set -e # Install the systemd service file and ensure that the store path won't be # garbage-collected as long as it's installed. unit_path=/etc/systemd/system/non-nixos-gpu.service ln -sf /nix/store/n16vlkkvvi5ybvvhsq6wjiv3m4snwf27-non-nixos-gpu/resources/non-nixos-gpu.service "$unit_path" ln -sf "$unit_path" "/nix/var/nix"/gcroots/non-nixos-gpu.service systemctl daemon-reload systemctl enable non-nixos-gpu.service systemctl restart non-nixos-gpu.service ``` `/nix/store/n16vlkkvvi5ybvvhsq6wjiv3m4snwf27-non-nixos-gpu/resources/non-nixos-gpu.service` is: ``` [Unit] Description=GPU driver setup for Nix on non-NixOS Linux systems [Install] WantedBy=multi-user.target [Service] Type=oneshot ExecStart=ln -nsf /nix/store/jm2dpkn1hhh2pb6srkx48livmsn9rgcf-non-nixos-gpu /run/opengl-driver RemainAfterExit=yes ``` The script fails at the `systemctl enable` step with a SELinux violation: ``` Nov 23 21:16:17 audit[1]: AVC avc: denied { read } for pid=1 comm="systemd" name="068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf" dev="dm-0" ino=57046418 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=file permissive=0 Nov 23 21:16:17 sudo[116512]: pam_unix(sudo:session): session closed for user root Nov 23 21:16:17 audit[116512]: AUDIT1106 pid=116512 uid=1000 auid=1000 ses=3 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=wodan addr=? terminal=/dev/pts/1 res=success' Nov 23 21:16:17 audit[116512]: AUDIT1104 pid=116512 uid=1000 auid=1000 ses=3 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" hostname=wodan addr=? terminal=/dev/pts/1 res=success' Nov 23 21:16:17 systemd[1]: Starting foomaticrip-upgrade.service - Allowing already installed printers for foomatic-rip... Nov 23 21:16:17 systemd[1]: session-c17.scope: Deactivated successfully. Nov 23 21:16:17 systemd-logind[1714]: Session c17 logged out. Waiting for processes to exit. Nov 23 21:16:17 systemd-logind[1714]: Removed session c17. Nov 23 21:16:17 systemd[1]: foomaticrip-upgrade.service: Deactivated successfully. Nov 23 21:16:17 systemd[1]: Finished foomaticrip-upgrade.service - Allowing already installed printers for foomatic-rip. Nov 23 21:16:17 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=foomaticrip-upgrade comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 23 21:16:17 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=foomaticrip-upgrade comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 23 21:16:19 systemd[1]: Starting setroubleshootd.service - SETroubleshoot daemon for processing new SELinux denial logs... Nov 23 21:16:19 systemd[1]: Started setroubleshootd.service - SETroubleshoot daemon for processing new SELinux denial logs. Nov 23 21:16:19 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=setroubleshootd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 23 21:16:20 setroubleshoot[117009]: failed to retrieve rpm info for path '/nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf': Nov 23 21:16:20 systemd[1]: Started dbus-:1.3-org.fedoraproject.SetroubleshootPrivileged. Nov 23 21:16:20 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dbus-:1.3-org.fedoraproject.SetroubleshootPrivileged@6 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 23 21:16:20 systemd[1]: fprintd.service: Deactivated successfully. Nov 23 21:16:20 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fprintd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 23 21:16:20 audit: BPF prog-id=437 op=UNLOAD Nov 23 21:16:21 setroubleshoot[117009]: SELinux is preventing systemd from read access on the file /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf. For complete SELinux messages run: sealert -l d3a486b5-ac3e-4a8c-94a7-88d653da5cdd Nov 23 21:16:21 setroubleshoot[117009]: SELinux is preventing systemd from read access on the file /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf. ***** Plugin catchall_labels (83.8 confidence) suggests ******************* If you want to allow systemd to have read access on the 068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf file Then you need to change the label on /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf Do # semanage fcontext -a -t FILE_TYPE '/nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf' where FILE_TYPE is one of the following: NetworkManager_dispatcher_console_var_run_t, NetworkManager_dispatcher_exec_t, NetworkManager_etc_rw_t, [TRUNCATED] Then execute: restorecon -v '/nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf' ***** Plugin catchall (17.1 confidence) suggests ************************** If you believe that systemd should be allowed read access on the 068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf file by default. Then you should report this as a bug. You can generate a local policy module to allow this access. Do allow this access for now by executing: # ausearch -c 'systemd' --raw | audit2allow -M my-systemd # semodule -X 300 -i my-systemd.pp Nov 23 21:16:21 systemd[2605]: Started dbus-:1.2-org.freedesktop.Notifications. Nov 23 21:16:30 systemd[1]: dbus-:1.3-org.fedoraproject.SetroubleshootPrivileged: Deactivated successfully. Nov 23 21:16:30 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dbus-:1.3-org.fedoraproject.SetroubleshootPrivileged@6 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 23 21:16:31 systemd[1]: setroubleshootd.service: Deactivated successfully. Nov 23 21:16:31 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=setroubleshootd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Nov 23 21:16:31 systemd[1]: setroubleshootd.service: Consumed 1.041s CPU time, 71.4M memory peak. ``` `/nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf` is identical (same inode) with `/nix/store/n16vlkkvvi5ybvvhsq6wjiv3m4snwf27-non-nixos-gpu/resources/non-nixos-gpu.service`, i.e. the file the NixOS / Home Manager script symlinked. determinate.systems' "Nix Installer" appears to install a SELinux policy that might cover this case: https://github.com/DeterminateSystems/nix-installer/blob/91e0774401ea84ab8e0dcfdcfbc750ff7036435e/src/planner/linux.rs#L89-L102 Their `nix.pp` is generated from https://github.com/DeterminateSystems/nix-installer/blob/91e0774401ea84ab8e0dcfdcfbc750ff7036435e/src/action/linux/selinux/nix.fc : ``` /nix/store/[^/]+/s?bin(/.*)? system_u:object_r:bin_t:s0 /nix/store/[^/]+/lib/systemd/system(/.*)? system_u:object_r:systemd_unit_file_t:s0 /nix/store/[^/]+/lib(/.*)? system_u:object_r:lib_t:s0 /nix/store/[^/]+/man(/.*)? system_u:object_r:man_t:s0 /nix/store/[^/]+/etc(/.*)? system_u:object_r:etc_t:s0 /nix/store/[^/]+/share(/.*)? system_u:object_r:usr_t:s0 /nix/var/nix/daemon-socket(/.*)? system_u:object_r:var_run_t:s0 /nix/var/nix/profiles(/per-user/[^/]+)?/[^/]+ system_u:object_r:usr_t:s0 ``` However, `semodule -X 300 -i nix.pp` (where `nix.pp` is https://github.com/DeterminateSystems/nix-installer/blob/91e0774401ea84ab8e0dcfdcfbc750ff7036435e/src/action/linux/selinux/nix.pp) followed by `sudo restorecon -rF /nix/store` did not allow me to run the script, either (same SELinux violation). Following the instructions from my own system's logs and running `sudo semanage fcontext -a -t systemd_unit_file_t /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf` followed by `sudo restorecon -v /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf` allows the script to succeed. But I expect this to only be a temporary success, because new versions of the systemd unit will be stored in new files, which will again be lacking the correct context. Reproducible: Always Actual Results: SELinux violations when enabling systemd services. Expected Results: Some way to permit Nix / Home Manager to enable systemd services.
P.S. The GPU access problem I am trying to fix affects Nix applications like `rio` or `gg-jj`. (Mentioning this, in case there is a way to fix the GPU access problem in the Fedora package, that is easier / safer than executing the `non-nixos-gpu` script Home Manager proposes to run.)
I haven't actually tried any systemd services via nix. I wonder if this should go into fedora's selinux-policy?
(In reply to Dennis Schridde from comment #0) > Their `nix.pp` is generated from > https://github.com/DeterminateSystems/nix-installer/blob/ > 91e0774401ea84ab8e0dcfdcfbc750ff7036435e/src/action/linux/selinux/nix.fc : > ``` > [...] > /nix/store/[^/]+/lib/systemd/system(/.*)? > system_u:object_r:systemd_unit_file_t:s0 > [...] > ``` > > However, `semodule -X 300 -i nix.pp` (where `nix.pp` is > https://github.com/DeterminateSystems/nix-installer/blob/ > 91e0774401ea84ab8e0dcfdcfbc750ff7036435e/src/action/linux/selinux/nix.pp) > followed by `sudo restorecon -rF /nix/store` did not allow me to run the > script, either (same SELinux violation). > > Following the instructions from my own system's logs and running `sudo > semanage fcontext -a -t systemd_unit_file_t > /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf` > followed by `sudo restorecon -v > /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf` > allows the script to succeed. P.S. I don't know enough about how SELinux work to explain why determinate.system's `nix.pp` did not work but my manual invocation of `semanage` did. Could this be because the rule is applied based on file name / path and the context is not attached to the inode? So `/nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf` being hardlinked as `/nix/store/n16vlkkvvi5ybvvhsq6wjiv3m4snwf27-non-nixos-gpu/resources/non-nixos-gpu.service`, and systemd for whatever reason trying to access it via the former name instead of the latter, would make SELinux see a different file context?
Just curious, what happens if you try to use/do with the nix systemd package?
(In reply to Jens Petersen from comment #4) > Just curious, what happens if you try to use/do with the nix systemd package? You mean replace Fedora's systemd installation with NixOS'? Is that even possible?
I just mean from inside a nix shell say: e.g. inside `nix-shell -p systemd` or `nix shell nixpkgs#systemd` or you could even add systemd to your nix profile. May not help at all... specially with sudo: so perhaps it should be `sudo nix-shell ...`?
Does not work: ``` ❯ sudo semanage fcontext -d /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf ValueError: File context for /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf is not defined ❯ sudo restorecon -v /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf Relabeled /nix/store/.links/068hxxavszgi3zc7pp6vw1gm90ly5rrqm51j1b43103fxl7z2hbf from system_u:object_r:systemd_unit_file_t:s0 to system_u:object_r:default_t:s0 ❯ nix shell nixpkgs#systemd $ sudo /nix/store/n16vlkkvvi5ybvvhsq6wjiv3m4snwf27-non-nixos-gpu/bin/non-nixos-gpu-setup Failed to enable unit: Access denied ```
Okay, not surprised hm Any different with `sudo nix shell nixpkgs#systemd` ?
Anyway I opened https://github.com/fedora-selinux/selinux-policy/issues/2963
(In reply to Jens Petersen from comment #8) > Any different with `sudo nix shell nixpkgs#systemd` ? No ``` ❯ sudo nix shell nixpkgs#systemd # ~REDACTED/.nix-profile/bin/non-nixos-gpu-setup Failed to enable unit: Access denied ```
(In reply to Jens Petersen from comment #9) > Anyway I opened https://github.com/fedora-selinux/selinux-policy/issues/2963 Thanks! I might be doing things wrongly, but installing the `nix.pp` SELinux module from nix-community/nix-installers or determinate.systems does not appear to solve my problem: ``` $ git clone git:nix-community/nix-installers.git $ cd nix-installers/selinux $ make $ sudo semodule -X 300 -i nix.pp $ sudo restorecon -rF /nix/store $ sudo ~REDACTED/.nix-profile/bin/non-nixos-gpu-setup Failed to enable unit: Access denied ``` `~REDACTED/.nix-profile/bin/non-nixos-gpu-setup` is the script installed via instructions from https://nix-community.github.io/home-manager/index.xhtml#sec-usage-gpu-non-nixos Could there be an additional step I'd have to take?
Okay thanks for trying - I am don't really have any deep ideas. So it seems to give a different error message? Perhaps it is worth stepping back to see what if anything works at all? I think GPU enablement is a harder problem perhaps. Is it possible get any simple systemd units or service to work via /nix? I was trying to think of some such test-case. Not sure what else to test: does the setup work if you install nix with the indeterminant nix installer? You might get better help from the nix community perhaps. You could try booting a nixos live iso image too perhaps to compare?
BTW different problem but for example I also cannot run EGL programs from toolbox: ⬢ fedora43~$ nix run nixpkgs/nixos-25.11#kitty [0.124] [glfw error 65542]: EGL: Failed to initialize EGL: An EGLDisplay argument does not name a valid EGL display connection display_present: 0 egl_platform_present: 0 Segmentation fault (core dumped) nix run nixpkgs/nixos-25.11#kitty
(In reply to Jens Petersen from comment #13) > BTW different problem but for example I also cannot run EGL programs from > toolbox: > > ⬢ fedora43~$ nix run nixpkgs/nixos-25.11#kitty > [0.124] [glfw error 65542]: EGL: Failed to initialize EGL: An EGLDisplay > argument does not name a valid EGL display connection display_present: 0 > egl_platform_present: 0 > Segmentation fault (core dumped) nix run nixpkgs/nixos-25.11#kitty This looks a lot like the problem I tried to solve by following https://nix-community.github.io/home-manager/index.xhtml#sec-usage-gpu-non-nixos -- i.e. what originally brought me onto the path that eventually led me to create this issue report, see https://bugzilla.redhat.com/show_bug.cgi?id=2416675#c0 .
(In reply to Jens Petersen from comment #13) > BTW different problem but for example I also cannot run EGL programs from > toolbox: > > ⬢ fedora43~$ nix run nixpkgs/nixos-25.11#kitty > [0.124] [glfw error 65542]: EGL: Failed to initialize EGL: An EGLDisplay > argument does not name a valid EGL display connection display_present: 0 > egl_platform_present: 0 > Segmentation fault (core dumped) nix run nixpkgs/nixos-25.11#kitty Also testing more: it seems unrelated to toolbox - I see same in a F43 VM. So probably happens without a gpu too. (Of course we have kitty in fedora)
Did you try to "setenforce 0" btw?
No, never tried that, because `sudo semanage fcontext -d /nix/store/.links/...` followed by `sudo restorecon -v /nix/store/.links/...` allows the script to execute successfully. So it seems related to SELinux (which I assume `setenforce 0` would confirm, right?), particularly the policy for a specific file.