Bug 988491 - Please add virt-login-shell support for OpenShift [NEEDINFO]
Please add virt-login-shell support for OpenShift
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt (Show other bugs)
7.0
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Daniel Berrange
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-25 13:16 EDT by Daniel Walsh
Modified: 2014-06-17 20:52 EDT (History)
12 users (show)

See Also:
Fixed In Version: libvirt-1.1.1-3.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-06-13 09:23:39 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
ajia: needinfo? (dwalsh)


Attachments (Terms of Use)
Patch to add virt-login-shell (20.32 KB, patch)
2013-07-25 13:16 EDT, Daniel Walsh
no flags Details | Diff
Broken the patch into two sets, one to add virGetUserDirectoryByUID (1.93 KB, patch)
2013-07-30 16:26 EDT, Daniel Walsh
no flags Details | Diff
Updated patch for virt-login-shell (17.01 KB, patch)
2013-07-30 16:27 EDT, Daniel Walsh
no flags Details | Diff

  None (edit)
Description Daniel Walsh 2013-07-25 13:16:00 EDT
Created attachment 778398 [details]
Patch to add virt-login-shell

Openshift wants to have their gears stuck into a container when they login
to the system.  virt-login-shell will join a running gear with the username of
the person running it, or attempt to start the container if it is not running.
(Currently containers do not exist if they are not running, so I can not test
this feature. But the code is there).

This tool needs to be setuid since joining a container (nsjoin) requires privs.
The root user is not allowed to execute this command. When this tool is
run by a normal user it will only join the "users" container.

Only users who are listed as valid_users in /etc/libvirt/virt-login-shell.conf
are allowed to join containers using this tool. By default no users are allowed.
Comment 2 Daniel Berrange 2013-07-30 11:10:59 EDT
Latest iteration of patches upsteam

https://www.redhat.com/archives/libvir-list/2013-July/msg01318.html
Comment 3 Daniel Walsh 2013-07-30 16:26:54 EDT
Created attachment 780832 [details]
Broken the patch into two sets, one to add virGetUserDirectoryByUID
Comment 4 Daniel Walsh 2013-07-30 16:27:36 EDT
Created attachment 780833 [details]
Updated patch for virt-login-shell
Comment 5 Daniel Berrange 2013-08-08 11:47:52 EDT
Merged upstream now

commit 54d69f540c9928da98f10202b3f21b7abb00bac1
Author: Dan Walsh <dwalsh@redhat.com>
Date:   Thu Aug 8 16:36:31 2013 +0100

    Introduce a virt-login-shell binary
    
    Add a virt-login-shell binary that can be set as a user's
    shell, such that when they login, it causes them to enter
    the LXC container with a name matching their user name.
    
    Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Comment 6 Daniel Berrange 2013-08-13 07:19:07 EDT
Also requires a followup patch https://www.redhat.com/archives/libvir-list/2013-August/msg00577.html
Comment 9 Alex Jia 2013-12-06 05:26:57 EST
# yum install -y libvirt-login-shell
# rpm -q libvirt-login-shell
libvirt-login-shell-1.1.1-13.el7.x86_64

# virt-login-shell -h

Usage:
  virt-login-shell [options]

Options:
  -h | --help            Display program help:
  -V | --version         Display program version:

libvirt login shell

# virt-login-shell -V
virt-login-shell (libvirt) 1.1.1

# man virt-login-shell | grep "virt-login-shell.conf"
       to a container that matches their username, if it exists, and they are configured in /etc/libvirt/virt-login-shell.conf.
       /etc/libvirt/virt-login-shell.conf.
       allowed_users variable in /etc/libvirt/virt-login-shell.conf.

# cat << EOF >> /etc/libvirt/virt-login-shell.conf
> allowed_users = ["sandbox"]
> shell = [ "/bin/ls", "-l", "/home/sandbox"]
> EOF

# tail -2 /etc/libvirt/virt-login-shell.conf
allowed_users = ["sandbox"]
shell = [ "/bin/ls", "-l", "/home/sandbox"]

# ll /usr/bin/virt-login-shell 
-rwsr-x---. 1 root virtlogin 827168 Nov 23 00:17 /usr/bin/virt-login-shell

# chmod a+x /usr/bin/virt-login-shell
# ll /usr/bin/virt-login-shell 
-rwsr-x--x. 1 root virtlogin 827168 Nov 23 00:17 /usr/bin/virt-login-shell

# su sandbox
Last login: Fri Dec  6 17:59:20 CST 2013 on pts/2
Last failed login: Fri Dec  6 17:59:43 CST 2013 on pts/2
There was 1 failed login attempt since the last successful login.

$ virt-login-shell 
Failed to initialize libvirt Error Handling

<strace_slice>

open(0x7f4148b87e38, O_RDONLY)          = -1 EACCES (Permission denied)
open(0x7fffb3ddbf80, O_RDONLY|O_CLOEXEC) = 3
fstat(3, {...})                         = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4148af6000
read(3, 0x7f4148af6000, 4096)           = 2502
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x7f4148af6000, 4096)            = 0
open(0x7f4149a9e340, O_RDONLY)          = -1 ENOENT (No such file or directory)
open(0x7f4149a9e3d0, O_RDONLY)          = -1 ENOENT (No such file or directory)
open(0x7f4149a9f380, O_RDONLY)          = -1 ENOENT (No such file or directory)
open(0x7f4149a9f300, O_RDONLY)          = -1 ENOENT (No such file or directory)
open(0x7f4149a9f400, O_RDONLY)          = -1 ENOENT (No such file or directory)
open(0x7f4149a9f490, O_RDONLY)          = -1 ENOENT (No such file or directory)
open(0x7f4149a9f590, O_RDONLY)          = -1 ENOENT (No such file or directory)
open(0x7f4149a9f510, O_RDONLY)          = -1 ENOENT (No such file or directory)
gettid()                                = 22442
write(2, 0x7fffb3dd9d40, 94libvirt:  error : Failed to open file '/etc/libvirt/virt-login-shell.conf': Permission denied
)            = 94
exit_group(1)                           = ?
+++ exited with 1 +++

</strace_slice>

But all of users has read permission for /etc/libvirt/virt-login-shell.conf.

# ll /etc/libvirt/virt-login-shell.conf
-rw-r--r--. 1 root root 1289 Dec  6 17:51 /etc/libvirt/virt-login-shell.conf

# getenforce
Enforcing

And no any AVC denied error in /var/log/audit/audit.log

BTW, I haven't opensift ENV and not sure whether I need to do other testing, could you tell me more? thanks.
Comment 10 Daniel Walsh 2013-12-06 16:12:45 EST
You need to create a container with the same name as the username.

I think you can set these up with virt-sandbox-service --UID and --username.
Comment 11 Alex Jia 2013-12-09 02:54:50 EST
(In reply to Daniel Walsh from comment #10)
> You need to create a container with the same name as the username.
> 
> I think you can set these up with virt-sandbox-service --UID and --username.

# chmod a+x /usr/bin/virt-login-shell
# ll /usr/bin/virt-login-shell -aZ
-rwsr-x--x. root virtlogin system_u:object_r:bin_t:s0       /usr/bin/virt-login-shell

# useradd -u 1001 -g virtlogin sandbox
# grep sandbox /etc/passwd
sandbox:x:1001:988::/home/sandbox:/bin/bash

# virt-sandbox-service create -N dhcp,source=default -U 1001 --username sandbox -u httpd.service myapache
Created sandbox container dir /var/lib/libvirt/filesystems/myapache
Created unit file /etc/systemd/system/myapache_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/myapache/config/sandbox.cfg

# tail -2 /etc/libvirt/virt-login-shell.conf
allowed_users = [ "sandbox" ]
shell = [ "/bin/sh",  "-l" ]

# virsh -c lxc:/// start myapache
Domain myapache started

# virsh -c lxc:/// domstate myapache
running

# su sandbox
Last login: Mon Dec  9 15:14:56 CST 2013 on pts/2

$ virt-login-shell 
Failed to initialize libvirt Error Handling

Notes, got the same issue with Comment 9 ("write(2, 0x7fffb3dd9d40, 94libvirt:  error : Failed to open file '/etc/libvirt/virt-login-shell.conf': Permission denied")
Comment 12 Alex Jia 2013-12-09 04:00:02 EST
(In reply to Alex Jia from comment #11)
> # useradd -u 1001 -g virtlogin sandbox
> # grep sandbox /etc/passwd
> sandbox:x:1001:988::/home/sandbox:/bin/bash

# useradd sandbox
# grep sandbox /etc/passwd
sandbox:x:1001:1001::/home/sandbox:/bin/bash
Comment 13 Alex Jia 2013-12-09 04:47:09 EST
(In reply to Alex Jia from comment #11)

> # virt-sandbox-service create -N dhcp,source=default -U 1001 --username
> sandbox -u httpd.service myapache
> Created sandbox container dir /var/lib/libvirt/filesystems/myapache
> Created unit file /etc/systemd/system/myapache_sandbox.service
> Created sandbox config
> /etc/libvirt-sandbox/services/myapache/config/sandbox.cfg

# virt-sandbox-service create -N dhcp,source=default -U 1001 --username sandbox -u httpd.service sandbox
Created sandbox container dir /var/lib/libvirt/filesystems/sandbox
Created unit file /etc/systemd/system/sandbox_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/sandbox/config/sandbox.cfg

Notes, the container name should be 'sandbox', but still got the same issue.
Comment 14 Alex Jia 2013-12-09 04:58:48 EST
Daniel, I have never successfull run virt-login-shell, did I miss any important steps? in addition, hasn't libvirt access permission for "/etc/libvirt/virt-login-shell.conf"? the bad thing is I can't find any AVC denied error in /var/log/audit/audit.log. thanks.
Comment 15 Daniel Walsh 2013-12-11 17:23:46 EST
virt-sandbox-service itself is blocking you.  Did you add sandbox to /etc/libvirt/virt-login-shell.conf
Comment 16 Alex Jia 2013-12-12 00:43:29 EST
(In reply to Daniel Walsh from comment #15)
> virt-sandbox-service itself is blocking you.  Did you add sandbox to
> /etc/libvirt/virt-login-shell.conf

# tail -2 /etc/libvirt/virt-login-shell.conf
allowed_users = ["sandbox"]
shell = [ "/bin/ls", "-l", "/home/sandbox"]
Comment 17 Daniel Walsh 2013-12-16 11:17:10 EST
Anything in the log files?

Also could you try in permissive mode?
Comment 18 Alex Jia 2013-12-16 22:26:52 EST
(In reply to Daniel Walsh from comment #17)
> Anything in the log files?

Please see Comment 9, no other useful information in log files such as libvirtd.log and audit.log.

# tail -2 /etc/libvirt/libvirtd.conf 
log_filters="3:remote 4:event 1:lxc 1:conf 1:libvirt 1:json 1:util"
log_outputs="1:file:/var/log/libvirt/libvirtd.log"


> 
> Also could you try in permissive mode?

# su sandbox
Last login: Mon Dec  9 17:44:14 CST 2013 on pts/7

$ getenforce
Permissive

$ virt-login-shell 
Failed to initialize libvirt Error Handling

Notes, got the same issue with 'Permissive' mode.

I think it should be a selinux issue, but as I said, it doesn't work with 'Permissive' mode, and can't find any AVC error in audit.log.

# chmod 777 /etc/libvirt/virt-login-shell.conf
# ll -aZ /etc/libvirt/virt-login-shell.conf
-rwxrwxrwx. root root system_u:object_r:virt_etc_t:s0  /etc/libvirt/virt-login-shell.conf

# virt-login-shell 
libvirt:  error : virt-login-shell must be run by non root users: Operation not permitted

# su sandbox
Last login: Tue Dec 17 11:15:03 CST 2013 on pts/42

$ virt-login-shell
Failed to initialize libvirt Error Handling
Comment 19 zhengqin 2014-01-14 03:26:50 EST
I could reproduce the issue commented by Alex when setting selinux as Permissive mode on current latest builds.


Related package version:
---------------------------
libvirt-1.1.1-18.el7.x86_64
libvirt-login-shell-1.1.1-18.el7.x86_64
libvirt-sandbox-0.5.0-8.el7.x86_64



Test steps to reproduce issue:
1. Set selinux as permissive mode.
    [root@localhost libvirt]# getenforce 
    Permissive


2. Create a group named 'virtlogin' 
    [root@localhost libvirt]#groupadd virtlogin

3. Create a 'sandbox' user account which belongs to group virtlogin.
    [root@localhost libvirt]#useradd -u 1001 -g virtlogin sandbox
    
    [root@localhost libvirt]#passwd sandbox
    
    [root@localhost libvirt]# grep sandbox /etc/passwd
    sandbox:x:1001:1000::/home/sandbox:/bin/bash
    
    [root@localhost libvirt]# grep virtlogin /etc/group
    virtlogin:x:1000:

4. editing /etc/libvirt/virt-login-shell.conf as follows:

 # tail -2 /etc/libvirt/virt-login-shell.conf
 allowed_users = ["sandbox"]
 shell = [ "/bin/ls", "-l"]


5. start a lxc container named as "sandbox"

 [root@localhost libvirt]#virt-sandbox -c lxc:/// /bin/sh -n sandbox

6. switch to 'sandbox' account and run 'virt-login-shell'

 # su sandbox
 $ virt-login-shell

Result:
Failed to run virt-login-shell, and
Error shows: Failed to initialize libvirt Error Handling.

Expected result:
The output for command "/bin/ls -l" should be displayed.
Comment 20 dyuan 2014-01-23 02:30:30 EST
No error "Failed to initialize libvirt Error Handling" show up after the bug 1015247 fixed.
Comment 21 Alex Jia 2014-01-23 03:56:27 EST
(In reply to dyuan from comment #20)
> No error "Failed to initialize libvirt Error Handling" show up after the bug
> 1015247 fixed.

In fact, the error still exists, but the cmd return value is 0.

$ virt-login-shell 
$ echo $?
0
$ strace virt-login-shell 
execve("/usr/bin/virt-login-shell", ["virt-login-shell"], [/* 25 vars */]) = 0

<ignore .../>

write(2, "libvirt:  error : Failed to open"..., 94libvirt:  error : Failed to open file '/etc/libvirt/virt-login-shell.conf': Permission denied
) = 94
exit_group(1)                           = ?
+++ exited with 1 +++

$ ll /etc/libvirt
ls: cannot open directory /etc/libvirt: Permission denied

# ll -Z /etc/libvirt/virt-login-shell.conf 
-rw-r--r--. root root system_u:object_r:virt_etc_t:s0  /etc/libvirt/virt-login-shell.conf

# ll -dZ /etc/libvirt
drwx------. root root system_u:object_r:virt_etc_t:s0  /etc/libvirt

Notes, although everyone has read permission for the '/etc/libvirt/virt-login-shell.conf', but the directory '/etc/libvirt' only can be accessed by root user,
could we assign read permission for the '/etc/libvirt' direcotry in libvirt? or allowing admin to change it manually? I'm not sure if it's safe to assign read permission for the '/etc/libvirt'.
Comment 22 Eric Blake 2014-02-26 18:42:13 EST
(In reply to dyuan from comment #20)
> No error "Failed to initialize libvirt Error Handling" show up after the bug
> 1015247 fixed.

This error is a symptom of the second half of bug 1015247; make sure you are testing with libvirt-1.1.1-19 or later, as 1.1.1-11 through 1.1.1-18 managed to completely cripple the login shell.
Comment 23 Alex Jia 2014-02-27 02:30:53 EST
The bug has been verified on libvirt-login-shell-1.1.1-25.el7.x86_64 with libvirt-1.1.1-25.el7.x86_64.

# virsh -c lxc:/// dumpxml ajia
<domain type='lxc' id='6153'>
  <name>ajia</name>
  <uuid>ff3fa87a-2fc9-434f-b411-8ec7f1f5d0f0</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/bin/sh</init>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/'/>
      <target dir='/'/>
    </filesystem>
    <interface type='network'>
      <mac address='52:54:00:a5:33:78'/>
      <source network='default'/>
      <target dev='vnet1'/>
    </interface>
    <console type='pty' tty='/dev/pts/7'>
      <source path='/dev/pts/7'/>
      <target type='lxc' port='0'/>
      <alias name='console0'/>
    </console>
  </devices>
  <seclabel type='none' model='selinux'/>
</domain>

# virsh -c lxc:/// start ajia
Domain ajia started

# ll /usr/bin/virt-login-shell
-rwsr-x--x. 1 root virtlogin 843696 Feb 26 23:37 /usr/bin/virt-login-shell

# tail -2 /etc/libvirt/virt-login-shell.conf
allowed_users = [ "ajia" ]
shell = [ "/usr/sbin/ip",  "link", "show" ]

# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 00:1e:4f:db:02:5c brd ff:ff:ff:ff:ff:ff
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT 
    link/ether 32:62:da:03:c9:bd brd ff:ff:ff:ff:ff:ff
95: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP mode DEFAULT qlen 1000
    link/ether 32:62:da:03:c9:bd brd ff:ff:ff:ff:ff:ff

# su ajia
$ whoami
ajia
$ virt-login-shell 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
94: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 52:54:00:a5:33:78 brd ff:ff:ff:ff:ff:ff
Comment 24 Ludek Smid 2014-06-13 09:23:39 EDT
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Note You need to log in before you can comment on or make changes to this bug.