Bug 927125

Summary:	[virt-sandbox-service] sandbox created with -i imagesize will fail to start
Product:	[Community] Virtualization Tools	Reporter:	Wayne Sun <gsun>
Component:	libvirt-sandbox	Assignee:	Libvirt Maintainers <libvirt-maint>
Status:	CLOSED WONTFIX	QA Contact:
Severity:	medium	Docs Contact:
Priority:	medium
Version:	unspecified	CC:	berrange, cwei, dwalsh, dyuan, mzhan, zpeng
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-04-16 17:47:56 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Wayne Sun 2013-03-25 06:22:28 UTC

Description of problem:
sandbox create -i imagesize can't start

Version-Release number of selected component (if applicable):
libvirt-1.0.3-1.el7.x86_64
libvirt-sandbox-0.1.2-1.el7.x86_64
kernel-3.7.0-0.36.el7.x86_64
systemd-198-4.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. create sandbox with -i imagesize
# virt-sandbox-service create -i 1000 -t svirt_lxc_net_t -l s0:c100,c400 -C -u httpd.service apache14
Loaded plugins: product-id
Created sandbox container image /var/lib/libvirt/images/apache14.raw
Created sandbox config /etc/libvirt-sandbox/services/apache14.sandbox
Created unit file /etc/systemd/system/apache14_sandbox.service

# ll -Z /var/lib/libvirt/images/apache14.raw
-rw-r--r--. root root unconfined_u:object_r:virt_image_t:s0 /var/lib/libvirt/images/apache14.raw

2. start the sandbox
# virt-sandbox-service start apache14
Unable to start container: Failed to create domain: internal error guest failed to start: PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=edad96e7-0486-89ae-594a-e8e4c6a03c2d LIBVIRT_LXC_UUID=edad96e7-0486-89ae-594a-e8e4c6a03c2d LIBVIRT_LXC_NAME=apache14 /usr/libexec/libvirt-sandbox-init-lxc
2013-03-25 06:21:39.877+0000: 11049: info : libvirt version: 1.0.3, package: 1.el7 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2013-03-05-04:25:15, x86-019.build.eng.bos.redhat.com)
2013-03-25 06:21:39.877+0000: 11049: error : virLXCControllerRun:1462 : Unable to send container continue message: Broken pipe
2013-03-25 06:21:39.896+0000: 11049: error : virCommandWait:2313 : internal error Child process (ip link del veth2) unexpected exit status 1: Cannot find device "veth2"

3.
  
Actual results:
fail to start

Expected results:
should success

Additional info:

Comment 2 Alex Jia 2013-05-28 06:35:43 UTC

For libvirt-sandbox-0.2.0-1.el7.x86_64 with libvirt-1.0.5-2.el7.x86_64, I got new error message as follows:

[root@amd-9600b-8-1 ~]# virt-sandbox-service create -i 1000 -s static,label=system_u:system_r:svirt_lxc_net_t:s0:c100,c400 -C -u httpd.service apache2
Created sandbox container image /var/lib/libvirt/images/apache2.raw
Created unit file /etc/systemd/system/apache2_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/apache2.sandbox

[root@amd-9600b-8-1 ~]# ll -h /var/lib/libvirt/images/apache2.raw
-rw-r--r--. 1 root root 954M May 28 14:14 /var/lib/libvirt/images/apache2.raw

[root@amd-9600b-8-1 ~]# virt-sandbox-service start apache2
Unable to start container: Failed to create domain: internal error guest failed to start: PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=83d88a72-d21b-47c2-9a90-23a2b6a399e4 LIBVIRT_LXC_UUID=83d88a72-d21b-47c2-9a90-23a2b6a399e4 LIBVIRT_LXC_NAME=apache2 /usr/libexec/libvirt-sandbox-init-lxc
error receiving signal from container: Input/output error


And the following debugger information from libvirtd.log:


2013-05-28 06:30:21.607+0000: 17706: debug : virConnectDomainEventRegisterAny:17919 : dom=(nil), (VM: name=(null), uuid=), conn=0x7fbeb8001a10, eventID=0, cb=0x7fbeff31f4c0, opaque=0x7fbeffd0ffe0, freecb=(nil)
2013-05-28 06:30:21.616+0000: 17710: debug : virDomainCreateXML:1982 : conn=0x7fbeb8001a10, xmlDesc=<domain type="lxc">
  <name>apache2</name>
  <memory unit="KiB">524288</memory>
  <features>
    <privnet/>
  </features>
  <os>
    <type arch="x86_64">exe</type>
    <init>/usr/libexec/libvirt-sandbox-init-lxc</init>
    <cmdline/>
  </os>
  <devices>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/"/>
      <target dir="/"/>
      <readonly/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/run/libvirt-sandbox/apache2/config"/>
      <target dir="/etc/libvirt-sandbox/scratch"/>
      <readonly/>
    </filesystem>
    <filesystem type="file" accessmode="passthrough">
      <source file="/var/lib/libvirt/images/apache2.raw"/>
      <target dir="/var/lib/libvirt/filesystems/apache2"/>
    </filesystem>
    <filesystem type="ram" accessmode="passthrough">
      <source usage="10485760" units="bytes"/>
      <target dir="/run"/>
    </filesystem>
    <filesystem type="ram" accessmode="passthrough">
      <source usage="104857600" units="bytes"/>
      <target dir="/tmp"/>
    </filesystem>
    <filesystem type="ram" accessmode="passthrough">
      <source usage="104857600" units="bytes"/>
      <target dir="/dev/shm"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/var"/>
      <target dir="/var"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/home"/>
      <target dir="/home"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/root"/>
      <target dir="/root"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/etc/systemd/system"/>
      <target dir="/etc/systemd/system"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/etc/rc.d"/>
      <target dir="/etc/rc.d"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/usr/lib/systemd/system/basic.target.wants"/>
      <target dir="/usr/lib/systemd/system/basic.target.wants"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/usr/lib/systemd/system/local-fs.target.wants"/>
      <target dir="/usr/lib/systemd/system/local-fs.target.wants"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/usr/lib/systemd/system/multi-user.target.wants"/>
      <target dir="/usr/lib/systemd/system/multi-user.target.wants"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/usr/lib/systemd/system/sysinit.target.wants"/>
      <target dir="/usr/lib/systemd/system/sysinit.target.wants"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/usr/lib/systemd/system/sockets.target.wants"/>
      <target dir="/usr/lib/systemd/system/sockets.target.wants"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/etc/machine-id"/>
      <target dir="/etc/machine-id"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/etc/fstab"/>
      <target dir="/etc/fstab"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/etc/hostname"/>
      <target dir="/etc/hostname"/>
    </filesystem>
    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/lib/libvirt/filesystems/apache2/etc/httpd"/>
      <target dir="/etc/httpd"/>
    </filesystem>
    <console type="pty"/>
  </devices>
  <seclabel model="selinux" type="static">
    <label>system_u:system_r:svirt_lxc_net_t:s0:c100,c400</label>
  </seclabel>
</domain>, flags=2
2013-05-28 06:30:21.618+0000: 17710: debug : virDomainObjNew:2043 : obj=0x7fbed4000b50
2013-05-28 06:30:21.886+0000: 17705: error : virNetSocketReadWire:1362 : Cannot recv data: Connection reset by peer
2013-05-28 06:30:21.983+0000: 17710: error : virLXCProcessStart:1285 : internal error guest failed to start: PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=83d88a72-d21b-47c2-9a90-23a2b6a399e4 LIBVIRT_LXC_UUID=83d88a72-d21b-47c2-9a90-23a2b6a399e4 LIBVIRT_LXC_NAME=apache2 /usr/libexec/libvirt-sandbox-init-lxc
error receiving signal from container: Input/output error

Comment 3 Daniel Walsh 2013-06-07 19:52:40 UTC

The problem here is the raw image has to be mounted before the bind mounts.

    <filesystem type="mount" accessmode="passthrough">
      <source dir="/var/run/libvirt-sandbox/apache2/config"/>
      <target dir="/etc/libvirt-sandbox/scratch"/>
      <readonly/>
    </filesystem>
    <filesystem type="file" accessmode="passthrough">
      <source file="/var/lib/libvirt/images/apache2.raw"/>
      <target dir="/var/lib/libvirt/filesystems/apache2"/>
    </filesystem>

Comment 4 Daniel Walsh 2013-06-07 19:54:26 UTC

Even though the libvirt-sandbox config has

[mount.0]
type=GVirSandboxConfigMountHostImage
target=/var/lib/libvirt/filesystems/apache2
source=/var/lib/libvirt/images/apache2.raw

As the first mount.

Comment 5 Daniel Berrangé 2013-07-09 14:39:55 UTC

The first problem here is that virt-sandbox-service is generating the wrong confijg. It is specifying "HostBind" for these, which indicates the that source path refers to the host filesystem. When using an image, the source path refers to the guest filesystem, so we must use "GuestBind" instead. This is fixed in

commit 45235084d101774f5e55674457b92073a3d1f352
Author: Daniel P. Berrange <berrange>
Date:   Tue Jul 9 14:20:10 2013 +0100

    Use 'guest bind' for mount overrides with image based containers


There is then another problem you hit.

We're mounting the image file on /var/lib/libvirt/filesystems/apache2, but we're then trying to bind mount the contents of that ove the top of /var, which  makes the image we just mounted at /var/lib/libvirt/filesystems/apache2 inaccessible.

Comment 8 Daniel Berrangé 2020-04-16 17:47:56 UTC

Closing old bug, virt-sandbox-service feature has been deleted.