Bug 997806

Summary: fail to start systemd container with guest-bind mount
Product: Red Hat Enterprise Linux 7 Reporter: Wayne Sun <gsun>
Component: libvirt-sandboxAssignee: Daniel Berrangé <berrange>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0CC: ajia, dyuan, weizhan, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-09-22 05:22:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Wayne Sun 2013-08-16 08:50:24 UTC
Description of problem:
create systemd container with guest-bind mount, start it will fail

Version-Release number of selected component (if applicable):
libvirt-1.1.1-2.el7.x86_64
libvirt-sandbox-0.5.0-2.el7.x86_64
systemd-206-4.el7.x86_64
kernel-3.10.0-3.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. create
# ll /tmp/test/
total 0

# virt-sandbox-service create -C -u httpd.service -s static,label=system_u:system_r:svirt_lxc_net_t:s0:c3,c131 -m guest-bind:/home=/tmp/test -N dhcp,source=default mountdir8
Created sandbox container dir /var/lib/libvirt/filesystems/mountdir8
Created unit file /etc/systemd/system/mountdir8_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/mountdir8/config/sandbox.cfg

# cat /etc/libvirt-sandbox/services/mountdir8/config/sandbox.cfg
...
[mount.17]
type=GVirSandboxConfigMountGuestBind
target=/home
source=/tmp/test
...

2. start
# systemctl start mountdir8_sandbox

# systemctl status mountdir8_sandbox
mountdir8_sandbox.service - Secure Sandbox Container mountdir8
   Loaded: loaded (/etc/systemd/system/mountdir8_sandbox.service; disabled)
   Active: failed (Result: exit-code) since Fri 2013-08-16 16:45:01 CST; 5s ago
  Process: 873 ExecStop=/usr/bin/virsh -c lxc:/// destroy mountdir8 (code=exited, status=1/FAILURE)
  Process: 827 ExecStart=/usr/libexec/virt-sandbox-service-util -c lxc:/// -s mountdir8 (code=exited, status=1/FAILURE)
 Main PID: 827 (code=exited, status=1/FAILURE)

Aug 16 16:45:00 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com systemd[1]: Starting Secure Sandbox Container mountdir8...
Aug 16 16:45:00 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com systemd[1]: Started Secure Sandbox Container mountdir8.
Aug 16 16:45:01 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com systemd[1]: mountdir8_sandbox.service: main process exited, code=exited, status=1/FAILURE
Aug 16 16:45:01 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virsh[873]: error: Failed to destroy domain mountdir8
Aug 16 16:45:01 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virsh[873]: error: Requested operation is not valid: Domain is not running
Aug 16 16:45:01 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com systemd[1]: mountdir8_sandbox.service: control process exited, code=exited status=1
Aug 16 16:45:01 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com systemd[1]: Unit mountdir8_sandbox.service entered failed state.

# vim /var/log/libvirt/lxc/mountdir8.log
...
2013-08-16 07:26:57.887+0000: 1: debug : lxcContainerUnmountSubtree:574 : Umount /home
2013-08-16 07:26:57.887+0000: 1: error : lxcContainerMountFSBind:1044 : Failed to bind mount directory /tmp/test to /home: No such file or directory
...

3.

Actual results:
fail

Expected results:
success

Additional info:

Comment 2 Daniel Berrangé 2013-09-18 12:59:46 UTC
(In reply to Wayne Sun from comment #0)
> Description of problem:
> create systemd container with guest-bind mount, start it will fail
> 
> Version-Release number of selected component (if applicable):
> libvirt-1.1.1-2.el7.x86_64
> libvirt-sandbox-0.5.0-2.el7.x86_64
> systemd-206-4.el7.x86_64
> kernel-3.10.0-3.el7.x86_64
> 
> How reproducible:
> always
> 
> Steps to Reproduce:
> 1. create
> # ll /tmp/test/
> total 0
> 
> # virt-sandbox-service create -C -u httpd.service -s
> static,label=system_u:system_r:svirt_lxc_net_t:s0:c3,c131 -m
> guest-bind:/home=/tmp/test -N dhcp,source=default mountdir8
> Created sandbox container dir /var/lib/libvirt/filesystems/mountdir8
> Created unit file /etc/systemd/system/mountdir8_sandbox.service
> Created sandbox config
> /etc/libvirt-sandbox/services/mountdir8/config/sandbox.cfg

This test scenario is broken.

'guest-bind' means bind a directory in the guest, to another directory in the guest. What you actually want here is 'host-bind' which means bind a directory in the host, to a directory in the guest.

/tmp in the guest is different from /tmp in the host, since the guest is given a private tmpfs.

Comment 3 Wayne Sun 2013-09-22 04:05:23 UTC
(In reply to Daniel Berrange from comment #2)
> (In reply to Wayne Sun from comment #0)
> > Description of problem:
> > create systemd container with guest-bind mount, start it will fail
> > 
> > Version-Release number of selected component (if applicable):
> > libvirt-1.1.1-2.el7.x86_64
> > libvirt-sandbox-0.5.0-2.el7.x86_64
> > systemd-206-4.el7.x86_64
> > kernel-3.10.0-3.el7.x86_64
> > 
> > How reproducible:
> > always
> > 
> > Steps to Reproduce:
> > 1. create
> > # ll /tmp/test/
> > total 0
> > 
> > # virt-sandbox-service create -C -u httpd.service -s
> > static,label=system_u:system_r:svirt_lxc_net_t:s0:c3,c131 -m
> > guest-bind:/home=/tmp/test -N dhcp,source=default mountdir8
> > Created sandbox container dir /var/lib/libvirt/filesystems/mountdir8
> > Created unit file /etc/systemd/system/mountdir8_sandbox.service
> > Created sandbox config
> > /etc/libvirt-sandbox/services/mountdir8/config/sandbox.cfg
> 
> This test scenario is broken.
> 
> 'guest-bind' means bind a directory in the guest, to another directory in
> the guest. What you actually want here is 'host-bind' which means bind a
> directory in the host, to a directory in the guest.
> 
> /tmp in the guest is different from /tmp in the host, since the guest is
> given a private tmpfs.

Yes, I did it wrong here.
After modify the container xml from:
...
    <filesystem type='bind' accessmode='passthrough'>
      <source dir='/tmp/test'/>
      <target dir='/home'/>
    </filesystem>
...

to:
...
    <filesystem type='bind' accessmode='passthrough'>
      <source dir='/etc/httpd'/>
      <target dir='/home'/>
    </filesystem>
...

# systemctl start mountdir8_sandbox

# systemctl status mountdir8_sandbox
mountdir8_sandbox.service - Secure Sandbox Container mountdir8
   Loaded: loaded (/etc/systemd/system/mountdir8_sandbox.service; disabled)
   Active: active (running) since Sun 2013-09-22 11:55:21 CST; 2s ago
  Process: 7442 ExecStop=/usr/bin/virsh -c lxc:/// destroy mountdir8 (code=exited, status=1/FAILURE)
 Main PID: 9788 (virt-sandbox-se)
   CGroup: /system.slice/mountdir8_sandbox.service
           └─9788 /usr/libexec/virt-sandbox-service-util -c lxc:/// -s mountdir8

Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Reached target System Initialization.
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Listening on D-Bus System Message Bus Socket.
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Reached target Sockets.
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Reached target Timers.
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Reached target Basic System.
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: Starting The Apache HTTP Server...
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: Starting Cleanup of Temporary Directories...
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Started Cleanup of Temporary Directories.
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Started The Apache HTTP Server.
Sep 22 11:55:22 ibm-x3850x5-04.qe.lab.eng.nay.redhat.com virt-sandbox-service-util[9788]: [  OK  ] Reached target Sandbox multi-user target.

# virt-sandbox-service connect mountdir8
sh-4.2# ls /home/
conf  conf.d  conf.modules.d  logs  modules  run
sh-4.2# ls /etc/httpd/
conf  conf.d  conf.modules.d  logs  modules  run

Comment 4 Wayne Sun 2013-09-22 05:22:07 UTC
Tested on

libvirt-1.1.1-6.el7.x86_64
libvirt-sandbox-0.5.0-3.el7.x86_64
systemd-206-4.el7.x86_64
kernel-3.10.0-9.el7.x86_64

# virt-sandbox-service create -C -u httpd.service -s static,label=system_u:system_r:svirt_lxc_net_t:s0:c32,c11 -m guest-bind:/home=/etc/httpd -N dhcp,source=default mountdir10
Created sandbox container dir /var/lib/libvirt/filesystems/mountdir10
Created unit file /etc/systemd/system/mountdir10_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/mountdir10/config/sandbox.cfg

This could work, so close this as NOTABUG.