Bug 784921 - Support for chrooted services is not good
Summary: Support for chrooted services is not good
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 17
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-01-26 16:27 UTC by Adam Tkac
Modified: 2013-04-30 23:51 UTC (History)
9 users (show)

Fixed In Version: systemd-37-19.fc16
Clone Of:
Environment:
Last Closed: 2012-03-21 19:10:09 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Adam Tkac 2012-01-26 16:27:27 UTC
Description of problem:
Support for chroot(2)-ed services is not so good. Currently I'm trying to create named-chroot.service unit file for chrooted named service. This is current state of the unit:

# cat named-chroot.service 
[Unit]
Description=Berkeley Internet Name Domain (DNS)
Wants=nss-lookup.target
Before=nss-lookup.target
After=network.target

[Service]
Type=forking
EnvironmentFile=-/etc/sysconfig/named
Environment=KRB5_KTNAME=/etc/named.keytab
PIDFile=/var/run/named/named.pid
RootDirectory=/var/named/chroot
RootDirectoryStartOnly=true

ExecStartPre=/usr/libexec/setup-named-chroot.sh $ROOTDIR on
ExecStartPre=/usr/sbin/named-checkconf -t $ROOTDIR -z /etc/named.conf
ExecStart=/usr/sbin/named -u named $OPTIONS

ExecReload=/bin/sh -c '/usr/sbin/rndc reload > /dev/null 2>&1 || /bin/kill -HUP $MAINPID'

ExecStop=/bin/sh -c '/usr/sbin/rndc stop > /dev/null 2>&1 || /bin/kill -TERM $MAINPID'
ExecStopPost=/usr/libexec/setup-named-chroot.sh $ROOTDIR off

PrivateTmp=true
TimeoutSec=25

[Install]
WantedBy=multi-user.target

Macro ROOTDIR=/var/named/chroot is declared in /etc/sysconfig/named.

When I try to `systemctl start named-chroot.service`, I end with the following errors in system log:

Jan 26 16:52:29 rawhide named[2852]: Failed at step FDS spawning /usr/sbin/named: No such file or directory
Jan 26 16:52:29 rawhide systemd[1]: named-chroot.service: control process exited, code=exited status=202
Jan 26 16:52:29 rawhide systemd[1]: Unit named-chroot.service entered failed state.

Version-Release number of selected component (if applicable):
[root@rawhide system]# rpm -q systemd
systemd-39-1.fc17.x86_64

How reproducible:
Try to start named-chroot.service unit as written above. Note you must have installed bind and bind-chroot packages.

Steps to Reproduce:
1. install bind and bind-chroot
2. put service file into /lib/systemd/system/named-chroot.service
3. systemctl --system daemon-reload
4. systemctl start named-chroot.service
  
Actual results:
service doesn't run

Expected results:
running service

Additional info:
I quickly looked into systemd source and it seems the main problem is that src/execute.c:exec_spawn() function calls chroot(2) and then exec*(path). However after chroot(2) call the path is obviously no longer valid and executable is not found.

I though about possible solutions for this and the only one is to `mount --bind` the executable itself with all it's dynamic dependencies into chroot. I'm not sure if this is indented but in my opinion there is no scenario when RootDirectory (and RootDirectoryStartOnly) can be used to start service in chroot(2).

I recommend to remove those options from systemd. Or at least document behavior in systemd.exec manpage to avoid confusion how to create unit file for chrooted service.

Comment 1 Michal Schmidt 2012-01-27 08:20:25 UTC
(In reply to comment #0)
> ExecStartPre=/usr/libexec/setup-named-chroot.sh $ROOTDIR on

BTW, I cannot find setup-named-chroot.sh in any of the packages.

> Jan 26 16:52:29 rawhide named[2852]: Failed at step FDS spawning
> /usr/sbin/named: No such file or directory

In this message it's not /usr/sbin/named that the "No such file or directory" is about.
"step FDS" is not too obvious at first sight, but it means it failed when attempting to close all file descriptors. close_all_fds() is called after chroot() and it wants to iterate over /proc/self/fd.

Does it help if you have /proc mounted in the chroot?

Maybe we can stop requiring that by just changing the order in exec_spawn() - first close all fds, then chroot.

Comment 2 Michal Schmidt 2012-01-27 08:36:15 UTC
(In reply to comment #0)
> I though about possible solutions for this and the only one is to `mount
> --bind` the executable itself with all it's dynamic dependencies into chroot.

I believe that's what this sentence in the manpage is saying:

RootDirectory=
  [...] it must be ensured that the process and all its auxiliary files are
  available in the chroot() jail.

The question is whether systemd should provide a helper utility to be called in ExecStartPre to have the executable and its libraries bind-mounted into the chroot automatically. Does there already exists a utility that would do something like that?:
bind-mount-executable-and-its-libraries /path/to/elf/binary /path/to/chroot

Comment 3 Michal Schmidt 2012-01-27 08:54:53 UTC
Here's a more general thought about chrooting services.

Either the daemon itself does the chroot(), or you let systemd do it. There's a definitely a trade-off:
Pros(+) and cons(-) for letting systemd do it (using RootDirectory=):
 + it can work for programs that do not know how to chroot themselves
 + the program does not even get a chance to screw it up
 - the binary and the libraries have to be present in the chroot directory

It's conceivable that in the bind case the pros can be so insignificant and the con so annoying that you may decide to depend on bind to chroot itself.

Comment 4 Jóhann B. Guðmundsson 2012-01-27 09:20:55 UTC
I think the rule of thumb simply should be if the daemon does the chroot() himself he should be the one to do it not systemd... (In reply to comment #2)
> (In reply to comment #0)
> > I though about possible solutions for this and the only one is to `mount
> > --bind` the executable itself with all it's dynamic dependencies into chroot.
> 
> I believe that's what this sentence in the manpage is saying:
> 
> RootDirectory=
>   [...] it must be ensured that the process and all its auxiliary files are
>   available in the chroot() jail.
> 
> The question is whether systemd should provide a helper utility to be called in
> ExecStartPre to have the executable and its libraries bind-mounted into the
> chroot automatically. Does there already exists a utility that would do
> something like that?:
> bind-mount-executable-and-its-libraries /path/to/elf/binary /path/to/chroot

Question if that should not be added to mount units?

Comment 5 Jóhann B. Guðmundsson 2012-01-27 09:21:46 UTC
(In reply to comment #3)
> Here's a more general thought about chrooting services.
> 
> Either the daemon itself does the chroot(), or you let systemd do it. There's a
> definitely a trade-off:
> Pros(+) and cons(-) for letting systemd do it (using RootDirectory=):
>  + it can work for programs that do not know how to chroot themselves
>  + the program does not even get a chance to screw it up
>  - the binary and the libraries have to be present in the chroot directory
> 
> It's conceivable that in the bind case the pros can be so insignificant and the
> con so annoying that you may decide to depend on bind to chroot itself.

I think the rule of thumb simply should be if the daemon does the chroot() himself he should be the one to do it not the systemd unit...

Comment 6 Lennart Poettering 2012-02-09 14:16:06 UTC
We probably should fix systemd to not require /proc to close all fds. We should just fall back to the classic loop until the rlimit in this case.

Comment 7 Lennart Poettering 2012-03-13 01:30:49 UTC
Fixed in git which I well upload to F17, soon.

Comment 8 Fedora Update System 2012-03-16 02:01:43 UTC
systemd-44-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/systemd-44-1.fc17

Comment 9 Fedora Update System 2012-03-16 19:02:15 UTC
Package systemd-44-1.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-44-1.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-3918/systemd-44-1.fc17
then log in and leave karma (feedback).

Comment 10 Fedora Update System 2012-03-21 19:10:09 UTC
systemd-44-1.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 11 Fedora Update System 2012-04-17 08:16:04 UTC
systemd-37-19.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/systemd-37-19.fc16

Comment 12 Fedora Update System 2012-04-26 03:31:31 UTC
systemd-37-19.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.