Bug 981239 - [LXC] missing [Install] section in SERVICE_sandbox.target
[LXC] missing [Install] section in SERVICE_sandbox.target
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt-sandbox (Show other bugs)
7.0
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Daniel Walsh
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-04 05:57 EDT by Monson Shao
Modified: 2014-09-21 18:51 EDT (History)
6 users (show)

See Also:
Fixed In Version: libvirt-sandbox-0.5.0-1.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-06-13 06:50:37 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Monson Shao 2013-07-04 05:57:42 EDT
Description of problem:

Command "systemctl enable httpd_sandbox.target" fails because [Install] setion is missing. It blocks all sandboxes autostart. A simple patch can fix it.

Further more, would it be better if there is an overall sandbox target to group all the service sandbox targets? e.g.

httpd01_sandbox.service \
                        |--> httpd_sandbox.target \
httpd02_sandbox.service /                         |
                                                  |--> sandbox.target --> multi-user.target
mysql01_sandbox.service \                         |
                        |--> mysql_sandbox.target /
mysql02_sandbox.service /


Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
Command "systemctl enable httpd_sandbox.target"

Actual results:
"systemctl enable httpd_sandbox.target" fail

Expected results:
"systemctl enable httpd_sandbox.target" succeed

Additional info:
Comment 2 Alex Jia 2013-07-04 07:11:08 EDT
To run echo -e "[Install]\nWantedBy=multi-user.target"  >> /etc/systemd/system/httpd_sandbox.target then reboot host, the all of containers are successfully started.
Comment 3 Alex Jia 2013-07-05 06:02:40 EDT
(In reply to Alex Jia from comment #2)
> To run echo -e "[Install]\nWantedBy=multi-user.target"  >>
> /etc/systemd/system/httpd_sandbox.target then reboot host, the all of
> containers are successfully started.

I checked it again, sometimes, only have some containers are successfully autostarted, and can't get reason from log for failed to start other containers.
Comment 4 Daniel Berrange 2013-07-09 06:35:02 EDT
I can't reproduce this problem. What are the exact commands you're using

# virt-sandbox-service create -u httpd.service  -C demo 
Created sandbox container dir /var/lib/libvirt/filesystems/demo
Created unit file /etc/systemd/system/demo_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/demo.sandbox

# systemctl enable demo_sandbox.service
# systemctl disable demo_sandbox.service
rm '/etc/systemd/system/httpd_sandbox.target.wants/demo_sandbox.service'
# cat /etc/systemd/system/demo_sandbox.service 

[Unit]
Description=Secure Sandbox Container demo
Requires=libvirtd.service
After=libvirtd.service
ReloadPropagatedFrom=httpd.service

[Service]
Type=simple
ExecStart=/usr/bin/virt-sandbox-service start demo
ExecReload=/usr/bin/virt-sandbox-service reload -u httpd.service demo
ExecStop=/usr/bin/virt-sandbox-service stop demo

[Install]
WantedBy=httpd_sandbox.target


So it already has an "Install" rule set and virt-sandbox-service in fact already runs 'systemctl enable' on the file it creates.
Comment 5 Monson Shao 2013-07-09 06:51:34 EDT
The [Install] section is missing in the target unit file, which grouping all sandbox service files in same application. So you should try

# systemctl enable httpd_sandbox.target
Comment 6 Daniel Berrange 2013-07-09 06:54:30 EDT
(In reply to Monson Shao from comment #5)
> The [Install] section is missing in the target unit file, which grouping all
> sandbox service files in same application. So you should try
> 
> # systemctl enable httpd_sandbox.target

As per my example above, '[install]' *is* present in the /etc/systemd/system/demo_sandbox.service  file created. Can you explain what you did to create your container ? AFAICT, from the code it should already be added correctly.
Comment 7 Monson Shao 2013-07-09 07:11:14 EDT
I create the container exactly the way you did. '[install]' do exist in demo_sandbox.service, so if you run 'systemctl enable demo_sandbox.service' (or it is enabled by default), then 'systemctl start httpd_sandbox.target' will start demo_sandbox.service.

However, what we want is that httpd_sandbox.target starts follow by multi-user.target, which means httpd_sandbox.target can autostart after booting. For now, you have to manually run 'systemctl start httpd_sandbox.target' to start all httpd sandbox containers.
Comment 8 Daniel Walsh 2013-07-09 16:20:20 EDT
So you want the following?

> cat /etc/systemd/system/httpd_sandbox.target

#  This target file is generated by virt-sandbox-service.
#  Use this target to start/stop all "httpd" sandbox services
#
#  virt-sandbox is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=httpd Sandbox Container Target
Documentation=man:virt-sandbox-service(1)

[Install]
WantedBy=multi-user.target
Comment 9 Daniel Berrange 2013-07-09 16:47:16 EDT
(In reply to Daniel Walsh from comment #8)
> So you want the following?
> 
> > cat /etc/systemd/system/httpd_sandbox.target
> 
> #  This target file is generated by virt-sandbox-service.
> #  Use this target to start/stop all "httpd" sandbox services
> #
> #  virt-sandbox is free software; you can redistribute it and/or modify it
> #  under the terms of the GNU Lesser General Public License as published by
> #  the Free Software Foundation; either version 2.1 of the License, or
> #  (at your option) any later version.
> 
> [Unit]
> Description=httpd Sandbox Container Target
> Documentation=man:virt-sandbox-service(1)
> 
> [Install]
> WantedBy=multi-user.target

That would enable all sandboxes at boot, which is not what I think any admin will want.  We need to have the ability to control enablement of individual sandboxes in the normal manner with 'systemdctl enable foo_sandbox.service'. Unfortunately the existence of the WantedBy=httpd_sandbox.target prevents us doing this.  We added that  WantedBy=httpd_sandbox.target to enable you to start all sandboxes at once, however, I'm not sure that this is actually all that useful / desirable in general.  I think admins are more likely to want to have socket based activation of most httpd sandboxes and perhaps manually set a handful of them to start by default in multi-user.target

Thus IMHO we need to remove the existing  WantedBy=httpd_sandbox.target  and replace it with WantedBy=multi-user.target, so we get more normal systemd behaviour here.
Comment 10 Daniel Walsh 2013-07-09 16:53:21 EDT
I thought the WantedBy=multi-user.target would only happen if the user enabeled the httpd_sandbox.target

systemctl enable httpd_sandbox.target

Then the httpd_sandbox.target would happen at boot with this flag.

But I have no problem removing that code.
Comment 13 Daniel Berrange 2013-08-01 10:57:22 EDT
In libvirt-sandbox >= 0.5.0 the created unit file has WantedBy=multi-user.target
Comment 14 Alex Jia 2013-08-07 03:48:19 EDT
On libvirt-sandbox-0.2.1-1.el7.x86_64:

# virt-sandbox-service create -C -u httpd.service -N dhcp myapache1
Created sandbox container dir /var/lib/libvirt/filesystems/myapache1
Created unit file /etc/systemd/system/myapache1_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/myapache1/config/sandbox.cfg

# grep Install -A 1 /etc/systemd/system/myapache1_sandbox.service
[Install]
WantedBy=httpd_sandbox.target


On libvirt-sandbox-0.5.0-1.el7.x86_64:

# virt-sandbox-service create -C -u httpd.service -N dhcp myapache2
Created sandbox container dir /var/lib/libvirt/filesystems/myapache2
Created unit file /etc/systemd/system/myapachel_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/myapache2/config/sandbox.cfg


# grep Install -A 1 /etc/systemd/system/myapache2_sandbox.service
[Install]
WantedBy=multi-user.target

# systemctl is-enabled multi-user.target
enabled

So the bug is verified on libvirt-sandbox-0.5.0-1.el7.x86_64.
Comment 15 Alex Jia 2013-08-14 01:04:47 EDT
Move this bug to VERIFIED status based on Comment 14.
Comment 16 Alex Jia 2013-10-31 03:10:59 EDT
Retest this on libvirt-sandbox-0.5.0-5.el7.x86_64 with libvirt-1.1.1-9.el7.x86_64, systemd-206-7.el7.x86_64 and kernel-3.10.0-0.rc7.64.el7.x86_64.

[root@localhost ~]# virt-sandbox-service create -C -u httpd.service -N dhcp myapache1
Created sandbox container dir /var/lib/libvirt/filesystems/myapache1
Created unit file /etc/systemd/system/myapache1_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/myapache1/config/sandbox.cfg

[root@localhost ~]# grep Install -A 1 /etc/systemd/syste/myapache1_sandbox.service
[Install]
WantedBy=multi-user.target

[root@localhost ~]# virsh -c lxc:// start myapache1
Domain myapache1 started

[root@localhost ~]# virsh -c lxc:// list
 Id    Name                           State
----------------------------------------------------
 1724  myapache1                      running

[root@localhost ~]# systemctl is-enabled httpd.service
disabled

[root@localhost ~]# systemctl enable httpd.service
ln -s '/usr/lib/systemd/system/httpd.service' '/etc/systemd/system/multi-user.target.wants/httpd.service'

[root@localhost ~]# reboot

Notes, to login the host then check the myapache1 state after rebooting, the myapache1 is 'shut off' state not running, maybe, I'm losting some important steps? Monson, could you help check this? thanks in advance.
Comment 17 Alex Jia 2013-10-31 03:27:41 EDT
(In reply to Alex Jia from comment #16)
> [root@localhost ~]# virsh -c lxc:// list
>  Id    Name                           State
> ----------------------------------------------------
>  1724  myapache1                      running
> 
> [root@localhost ~]# systemctl is-enabled httpd.service
> disabled
> 
> [root@localhost ~]# systemctl enable httpd.service
> ln -s '/usr/lib/systemd/system/httpd.service'
> '/etc/systemd/system/multi-user.target.wants/httpd.service'

Hmmm, I lost the following steps, the container myapache1 still is running after rebooting host.
 
[root@localhost ~]# systemctl is-enabled myapache1_sandbox.service
disabled

[root@localhost ~]# systemctl enable myapache1_sandbox.service
ln -s '/etc/systemd/system/myapache1_sandbox.service' '/etc/systemd/system/multi-user.target.wants/myapache1_sandbox.service'

[root@localhost ~]# systemctl is-enabled myapache1_sandbox.service
enabled

> 
> [root@localhost ~]# reboot
Comment 18 Alex Jia 2013-10-31 03:47:12 EDT
Daniel, it's very weird, sometimes, the running apache container hasn't been automatically started based on steps of the Comment16 and Comment17. And I got the following error when I login the host again.

[root@localhost ~]# systemctl status myapache1_sandbox.service
myapache1_sandbox.service - Secure Sandbox Container myapache1
   Loaded: loaded (/etc/systemd/system/myapache1_sandbox.service; enabled)
   Active: failed (Result: exit-code) since Thu 2013-10-31 15:32:33 CST; 2min 51s ago
  Process: 1145 ExecStop=/usr/bin/virsh -c lxc:/// destroy myapache1 (code=exited, status=1/FAILURE)
  Process: 1112 ExecStart=/usr/libexec/virt-sandbox-service-util -c lxc:/// -s myapache1 (code=exited, status=1/FAILURE)
 Main PID: 1112 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/myapache1_sandbox.service

Oct 31 15:32:29 localhost.localdomain systemd[1]: Starting Secure Sandbox Container myapache1...
Oct 31 15:32:29 localhost.localdomain systemd[1]: Started Secure Sandbox Container myapache1.
Oct 31 15:32:29 localhost.localdomain systemd[1]: myapache1_sandbox.service: main process exited, code=exited, status=1/FAILURE
Oct 31 15:32:33 localhost.localdomain systemd[1]: myapache1_sandbox.service: control process exited, code=exited status=1
Oct 31 15:32:33 localhost.localdomain systemd[1]: Unit myapache1_sandbox.service entered failed state.


[root@localhost ~]# grep myapache1 /var/log/messages
Oct 31 15:30:11 localhost systemd-machined[1747]: New machine lxc-myapache1.
Oct 31 15:30:11 localhost systemd[1]: Started Container lxc-myapache1.
Oct 31 15:31:15 localhost systemd[1]: Stopping Container lxc-myapache1.
Oct 31 15:32:29 localhost systemd[1]: Starting Secure Sandbox Container myapache1...
Oct 31 15:32:29 localhost systemd[1]: Started Secure Sandbox Container myapache1.
Oct 31 15:32:29 localhost systemd[1]: myapache1_sandbox.service: main process exited, code=exited, status=1/FAILURE
Oct 31 15:32:33 localhost virsh[1145]: error: Failed to destroy domain myapache1
Oct 31 15:32:33 localhost systemd[1]: myapache1_sandbox.service: control process exited, code=exited status=1
Oct 31 15:32:33 localhost systemd[1]: Unit myapache1_sandbox.service entered failed state.

[root@localhost ~]# getenforce
Enforcing

BTW, no AVC denied in  /var/log/audit/audit.log.
Comment 19 Daniel Walsh 2013-11-04 10:24:13 EST
Does it work if you put the machine into permissive mode on the reboot?
Comment 20 Alex Jia 2013-11-06 05:22:43 EST
(In reply to Daniel Walsh from comment #19)
> Does it work if you put the machine into permissive mode on the reboot?

Daniel, it still doesn't work, please see blow details.

[root@localhost Workspace]# setenforce 0

[root@localhost Workspace]# getenforce
Permissive

[root@localhost Workspace]# grep avc /var/log/audit/audit.log
type=USER_AVC msg=audit(1383732562.986:9869): pid=1 uid=0 auid=4294967295 ses=4294967295  subj=system_u:system_r:init_t:s0 msg='avc:  received setenforce notice (enforcing=0)  exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'

Notes, is it a selinux issue?

[root@localhost Workspace]# virsh -c lxc:/// start myapache1
Domain myapache1 started

[root@localhost Workspace]# virsh -c lxc:/// start myapache2
Domain myapache2 started

[root@localhost Workspace]# virsh -c lxc:/// list
 Id    Name                           State
----------------------------------------------------
 19689 myapache1                      running
 19754 myapache2                      running

[root@localhost Workspace]# systemctl is-enabled myapache1_sandbox.service
enabled

[root@localhost Workspace]# systemctl is-enabled myapache2_sandbox.service
enabled


[root@localhost Workspace]# systemctl status myapache1_sandbox.service
myapache1_sandbox.service - Secure Sandbox Container myapache1
   Loaded: loaded (/etc/systemd/system/myapache1_sandbox.service; enabled)
   Active: failed (Result: exit-code) since Wed 2013-11-06 18:13:07 CST; 3min 54s ago
  Process: 19621 ExecStop=/usr/bin/virsh -c lxc:/// destroy myapache1 (code=exited, status=1/FAILURE)
  Process: 19537 ExecStart=/usr/libexec/virt-sandbox-service-util -c lxc:/// -s myapache1 (code=exited, status=0/SUCCESS)
 Main PID: 19537 (code=exited, status=0/SUCCESS)

Nov 06 18:11:29 localhost.localdomain virt-sandbox-service-util[19537]: [  OK  ] Reached target Basic System.
Nov 06 18:11:29 localhost.localdomain virt-sandbox-service-util[19537]: Starting The Apache HTTP Server...
Nov 06 18:11:29 localhost.localdomain virt-sandbox-service-util[19537]: Starting Cleanup of Temporary Directories...
Nov 06 18:11:29 localhost.localdomain virt-sandbox-service-util[19537]: [  OK  ] Started Cleanup of Temporary Directories.
Nov 06 18:11:29 localhost.localdomain virt-sandbox-service-util[19537]: [  OK  ] Started The Apache HTTP Server.
Nov 06 18:11:29 localhost.localdomain virt-sandbox-service-util[19537]: [  OK  ] Reached target Sandbox multi-user target.
Nov 06 18:13:07 localhost.localdomain virsh[19621]: error: Failed to destroy domain myapache1
Nov 06 18:13:07 localhost.localdomain virsh[19621]: error: Requested operation is not valid: Domain is not running
Nov 06 18:13:07 localhost.localdomain systemd[1]: myapache1_sandbox.service: control process exited, code=exited status=1
Nov 06 18:13:07 localhost.localdomain systemd[1]: Unit myapache1_sandbox.service entered failed state.

[root@localhost Workspace]# systemctl status myapache2_sandbox.service
myapache2_sandbox.service - Secure Sandbox Container myapache2
   Loaded: loaded (/etc/systemd/system/myapache2_sandbox.service; enabled)
   Active: failed (Result: exit-code) since Wed 2013-11-06 18:10:52 CST; 6min ago
  Process: 19402 ExecStop=/usr/bin/virsh -c lxc:/// destroy myapache2 (code=exited, status=1/FAILURE)
  Process: 19388 ExecStart=/usr/libexec/virt-sandbox-service-util -c lxc:/// -s myapache2 (code=exited, status=1/FAILURE)
 Main PID: 19388 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/myapache2_sandbox.service

Nov 06 18:10:49 localhost.localdomain systemd[1]: Starting Secure Sandbox Container myapache2...
Nov 06 18:10:49 localhost.localdomain systemd[1]: Started Secure Sandbox Container myapache2.
Nov 06 18:10:49 localhost.localdomain virt-sandbox-service-util[19388]: Unable to open connection: Unable to open lxc:///: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such f...r directory
Nov 06 18:10:49 localhost.localdomain systemd[1]: myapache2_sandbox.service: main process exited, code=exited, status=1/FAILURE
Nov 06 18:10:52 localhost.localdomain virsh[19402]: error: Failed to destroy domain myapache2
Nov 06 18:10:52 localhost.localdomain virsh[19402]: error: Requested operation is not valid: Domain is not running
Nov 06 18:10:52 localhost.localdomain systemd[1]: myapache2_sandbox.service: control process exited, code=exited status=1
Nov 06 18:10:52 localhost.localdomain systemd[1]: Unit myapache2_sandbox.service entered failed state.

[root@localhost Workspace]# reboot

[root@localhost ~]# virsh -c lxc:/// list --inactive
 Id    Name                           State
----------------------------------------------------
 -     myapache1                      shut off
 -     myapache2                      shut off

[root@localhost ~]# systemctl status myapache1_sandbox.service
myapache1_sandbox.service - Secure Sandbox Container myapache1
   Loaded: loaded (/etc/systemd/system/myapache1_sandbox.service; enabled)
   Active: failed (Result: exit-code) since Wed 2013-11-06 18:18:44 CST; 1min 31s ago
  Process: 1172 ExecStop=/usr/bin/virsh -c lxc:/// destroy myapache1 (code=exited, status=1/FAILURE)
  Process: 1113 ExecStart=/usr/libexec/virt-sandbox-service-util -c lxc:/// -s myapache1 (code=exited, status=1/FAILURE)
 Main PID: 1113 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/myapache1_sandbox.service

Nov 06 18:18:43 localhost.localdomain systemd[1]: Starting Secure Sandbox Container myapache1...
Nov 06 18:18:43 localhost.localdomain systemd[1]: Started Secure Sandbox Container myapache1.
Nov 06 18:18:44 localhost.localdomain systemd[1]: myapache1_sandbox.service: main process exited, code=exited, status=1/FAILURE
Nov 06 18:18:44 localhost.localdomain systemd[1]: myapache1_sandbox.service: control process exited, code=exited status=1
Nov 06 18:18:44 localhost.localdomain systemd[1]: Unit myapache1_sandbox.service entered failed state.
Comment 22 Ludek Smid 2014-06-13 06:50:37 EDT
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Note You need to log in before you can comment on or make changes to this bug.