Bug 1962593

Code that sets up capabilities:
...
  capng_clear(CAPNG_SELECT_BOTH);
  capng_updatev(CAPNG_ADD, (capng_type_t)(CAPNG_EFFECTIVE|CAPNG_PERMITTED),
    CAP_SYS_ADMIN, CAP_MKNOD, CAP_SYS_RAWIO, -1);
  capng_apply(CAPNG_SELECT_BOTH);
...
I will call that upstream capabilities for now.

The issue has more parts:
1) smartd has "drop your own script" notification system /etc/smartmontools/smartd_warning.d/<your scripts>
that means we can never know what capabilities are required to run those scripts, but as that requires explicit configuration, dropping --capabilities can be part of that configuration, so it's not really issue here

2) default out of the box behavior that's there forever and that has two parts
a) sending message to all logged in users (terminals)
b) sending an email to admin

2a - we use code that writes message directly to every terminal /dev/pts/* those have 0600 permissions, which would probably require CAP_DAC_OVERRIDE, there's also 'wall' command that could be used. I don't know what mechanism it uses and whether it would work without that capability

2b - this part is a bit more difficult as smartd runs notification script, that runs 'mail' command which is symlink based on 'alternatives' which runs 'sendmail' that is symlink to some mail server - that could be exim, sendmail, postfix and it's possible that different server could need different capabilities to run.

I have exim mail set up on my server that I used for testing. Other servers could give different results.

When used with upstream capabilities, it does not send an email and produces an error: 
smartd[11355]: exim: setgroups() failed: Operation not permitted

same error is present if setuid and setgid capabilities are added to smartd:
  capng_updatev(CAPNG_ADD, (capng_type_t)(CAPNG_EFFECTIVE|CAPNG_PERMITTED),
    CAP_SYS_ADMIN, CAP_MKNOD, CAP_SYS_RAWIO, CAP_SETUID, CAP_SETGID, -1);

After adding strace to notification script and CAP_SYS_PTRACE + CAPNG_INHERITABLE, it confirms that setgroups() is failing:
getuid()                                = 0
getgid()                                = 0
setgid(0)                               = 0
setuid(0)                               = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=4*1024}) = 0
getrlimit(RLIMIT_NPROC, {rlim_cur=3882, rlim_max=3882}) = 0
getgroups(65536, [])                    = 0
setgroups(0, NULL)                      = -1 EPERM (Operation not permitted)
setgroups(1, [0])                       = -1 EPERM (Operation not permitted)
write(2, "exim: setgroups() failed: Operat"..., 50) = 50

Michal, were you able to try that setup with wall?

Steve, what is the recommended approach for writing to every terminal?

The mailing problem has multiple issues.

This is a RHEL bugzilla and RHEL does not ship exim, so on RHEL we'd need to have behaviour with postfix and sendmail tested.

However, for exim, the issues (and EPERM) seems to stem from the code

  https://github.com/Exim/exim/blob/master/src/src/exim.c#L3741-L3746

So exim assumes that when it is running as privileged, it has the CAP_SETGID capability to run setgroups. One possibility to enable such behaviour would be to add the CAP_SETGID to smartd.

Another possibility might be to add the capability to the exim binary. Michal, could you try that with filecap?

However, looking at the code

  if ((group_count = getgroups(nelem(group_list), group_list)) < 0)
    exim_fail("exim: getgroups() failed: %s\n", strerror(errno));

  if (  !unprivileged
  #ifndef OS_SETGROUPS_ZERO_DROPS_ALL
     && setgroups(0, NULL) != 0
  #endif
     && setgroups(1, group_list) != 0)
    exim_fail("exim: setgroups() failed: %s\n", strerror(errno));

and the strace output

getgroups(65536, [])                    = 0
setgroups(0, NULL)                      = -1 EPERM (Operation not permitted)
setgroups(1, [0])                       = -1 EPERM (Operation not permitted)

maybe the easiest fix would be to change exim to just not try to do those setgroups at all when getgroups returned 0 -- the process has no supplementary groups, so there are not groups to drop.

Steve, would that conclusion be a reasonable one?

Michal, I also wonder what your alternatives are for the mail command. On my Fedora setup with exim, exim is actually not involved on the mail sending:

# ls -la /usr/bin/mail
lrwxrwxrwx. 1 root root 22 Nov 13  2020 /usr/bin/mail -> /etc/alternatives/mail
# ls -la /etc/alternatives/mail
lrwxrwxrwx. 1 root root 20 Apr 28 12:17 /etc/alternatives/mail -> /usr/bin/mailx.mailx
# ls -la /usr/bin/mailx.mailx
-rwxr-xr-x. 1 root root 422912 Mar 16 20:02 /usr/bin/mailx.mailx
# rpm -qf /usr/bin/mailx.mailx
mailx-12.5-37.fc34.x86_64

re #2
wall - did just quick run when testing the other part and it seems it worked (with the CAPs set as described below)

re #3
that part of the trace is from run with (CAPNG_EFFECTIVE|CAPNG_PERMITTED|CAPNG_INHERITABLE) CAP_SYS_ADMIN, CAP_MKNOD, CAP_SYS_RAWIO, CAP_SETUID, CAP_SETGID, CAP_SYS_PTRACE
I thought that setgroups should work with this, but it obviously didn't


re #4
this was tested on RHEL 7 machine

what you have in your output is only first part that goes through alternatives, you are missing the second part where mailx execve "sendmail"
$ alternatives --display mta
 link currently points to /usr/sbin/sendmail.exim
/usr/sbin/sendmail.exim - priority 10
 slave mta-pam: /etc/pam.d/exim
 slave mta-mailq: /usr/bin/mailq.exim
 slave mta-newaliases: /usr/bin/newaliases.exim
 slave mta-rmail: /usr/bin/rmail.exim
 slave mta-rsmtp: /usr/bin/rsmtp.exim
 slave mta-runq: /usr/bin/runq.exim
 slave mta-sendmail: /usr/lib/sendmail.exim
 slave mta-mailqman: /usr/share/man/man8/exim.8.gz
 slave mta-sendmailman: (null)

I tried with 
filecap /usr/sbin/exim setgid

and it did not work at all:
execve("/usr/sbin/sendmail", ["send-mail", "-i", "--", "masp01"], 0x7ffe3040d730 /* 21 vars */) = -1 EPERM (Operation not permitted)

when I reverted that filecap /usr/sbin/exim none, it gave me setgroups error as before

note: All tests are run with SELinux in permissive mode

that /usr/sbin/sendmail = exim

# ll /usr/sbin/sendmail
lrwxrwxrwx. 1 root root 21 May 19 12:01 /usr/sbin/sendmail -> /etc/alternatives/mta

# ll /etc/alternatives/mta
lrwxrwxrwx. 1 root root 23 May 19 12:01 /etc/alternatives/mta -> /usr/sbin/sendmail.exim

# ll /usr/sbin/sendmail.exim
lrwxrwxrwx. 1 root root  4 May 19 12:01 /usr/sbin/sendmail.exim -> exim

I've now tried with

echo test | setpriv --inh-caps -all --ambient-caps -all \
  --bounding-set -all,+setgid,+setuid,+chown,+fowner,+dac_override mail -v user

that exim needs at least the above capabilities in the bounding set.

Sending email is done with a client which is normally called sendmail, but is always a symlink to a postfix or sendmail client. A server should not be needed. (It appears to be mailx on Fedora.) If the client needed to call setgroups, the setgid capability should be all that's needed. (kernel/groups.c#L172). There may be a selinux interaction here, too.

You can also look into /etc/smartmontools/smartd_warning.d/ and if it's empty drop capabilities. Otherwise, stay root since you have no idea what they need.

Adding either DAC_OVERRIDE/SYS_ADMIN are about the same as staying root. If either has to be added, then its best to add a comment somewhere so that next code review people see it.

The general question is -- should it be the smard code that manipulates the capabilities (which obviously makes much harder to tweak the capabilities set), or should it be done with systemd unit configuration, where the admin would have more opportunities for tuning the setup with snippets to match their use-case?

I think it's better for the daemon to do it. Systemd has no ability to grant capabilities without resorting to ambient capabilities. This means that the daemon has to immediately undo the ambient capabilities on startup so that it doesn't leak them everywhere.

The systemd.exec(5) man page says that CapabilityBoundingSet= can be used to manipulate the bounding set. And since smartd runs as root, I don't think ambient capabilities are needed / useful.

I think it's better for the daemon. While I did not ask upstream about this, we should not forget that there are still linux systems that don't use systemd

For RHEL 9, the exim use-case is not really that relevant -- we need things tested with sendmail and postfix.

Note that some of the privileged exim needs is because it tries to make the delivery of the spooled message (even if no MTA is running), rather than putting things to the mail queue directory and leaving it up to the different process with different privileges to do the delivery part. My hope is that postfix (and hopefully sendmail) don't do that.

Fix used in Fedora seems to work fine, lets use it here

Don't know if this is the right place to comment, but I'm on Fedora 35 with smartmontools 7.2-11 (so supposedly this is fixed IIUC), but I'm still unable to set smartd up to send me notifications or emails.

When I use the "-m" option in smartd.conf it says that "mail" cannot be found, so the redirection to "sendmail" seems to not be working.
When trying to circumvent that by using "-M exec" and pointing to my own script SELinux smacks it down (doesn't want to let smartd execute my script).
When moving my code to one of the smartmontools example scripts in /usr/share/doc[...] it allows executing the existing Example (after chmod) but smacks down the attempt to execute the commands within.
The last thing I tried was to comment everything in one of the Example scripts and just put
   
   notify-send 'test'

there and then also added a new policy to SELinux so it allows executing the "notify-send" command, but I still don't get a notification when running with "-M test" upon restart even though in "systemctl status smartd" it says that the test was executed successfully (and no SELinux action since I introduced the automatically recommended policy).
When I run that same Example script manually, even without root, it gives me the notification just fine.

As an aside, I for some reason now can't remove the custom policy from SELinux, either, ("sudo semodule -r my-notifysend.pp"), as it says "No such file or directory", even though "sudo semodule -l" does list the module...
This has been quite a frustrating process...it shouldn't be so hard to have smartd notify me about problems.

Has this actually been fixed? Apologies if this is the wrong place for comment; if it is, should I open a new issue?

This is a RHEL bz. The Fedora bz is 1954015. The question would be, what mail client do you use?

Thanks for letting me know! Continuing on the correct thread.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: smartmontools), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:4062