Bug 1374126

Summary: using polkit with virsh for non-root access does not work via ssh on a remote host
Product: Red Hat Enterprise Linux 7 Reporter: Fangge Jin <fjin>
Component: libvirtAssignee: John Ferlan <jferlan>
Status: CLOSED ERRATA QA Contact: yafu <yafu>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: dyuan, jferlan, lhuang, mzhan, rbalakri, xuzhang, yafu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-3.2.0-7.el7 Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: 986365 Environment:
Last Closed: 2017-08-01 17:14:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Fangge Jin 2016-09-08 03:09:11 UTC
Description of the problem:
Failed to connect to libvirt system URI on a remote host with non-root user


+++ This bug was initially created as a clone of Bug #986365 +++

--- Additional comment from JinFangge on 2016-07-07 06:15:26 EDT ---

I test on build libvirt-2.0.0-1.el7.x86_64, and meet a problem: failed to connect to libvirt on a remote host with non-root user.

[fjin@localhost ~]$ virsh -c qemu+ssh://10.66.4.152/system
fjin.4.152's password: 
fjin.4.152's password: 

** (pkttyagent:11910): WARNING **: Unable to register authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: An authentication agent already exists for the given subject
Error registering authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: An authentication agent already exists for the given subject (polkit-error-quark, 0)

--- Additional comment from Luyao Huang on 2016-07-12 23:44:14 EDT ---

Hi John,

I think the way libvirt create pkttyagent is not correct in some cases. I have checked what libvirt do when the user use virsh to connect to a remote machine (the same with comment 33). I will show the gdb debug info:

$ /usr/bin/gdb virsh

(gdb) br virshConnect
Breakpoint 1 at 0x266a0: file virsh.c, line 141.
(gdb) r -c qemu+ssh://lhuang@test1/system

Breakpoint 1, virshConnect (ctl=0x7fffffffda70, uri=0x555555804a30 "qemu+ssh://lhuang@test1/system", readonly=false) at virsh.c:141
141	{

(gdb) n
149	    if (ctl->keepalive_interval >= 0) {
(gdb) 
141	{
(gdb) 
151	        keepalive_forced = true;
(gdb) 
149	    if (ctl->keepalive_interval >= 0) {
(gdb) 
145	    bool keepalive_forced = false;
(gdb) 
143	    int interval = 5; /* Default */
(gdb) 
153	    if (ctl->keepalive_count >= 0) {
(gdb) 
144	    int count = 6;    /* Default */
(gdb) 
161	        if ((c = virConnectOpenAuth(uri, virConnectAuthPtrDefault,
(gdb) n
Detaching after fork from child process 27954.
lhuang@test1's password: 

[this is ssh authenticate, but target libvirtd close client connection since it is not pass authenticate, and i can get error like " virPolkitCheckAuth:133 : authentication unavailable: no polkit agent available to authenticate action 'org.libvirt.unix.manage' " in target libvirtd.log]

165	        if (readonly)
(gdb) n
168	        err = virGetLastError();
(gdb) 
169	        if (err && err->domain == VIR_FROM_POLKIT &&
(gdb) 
171	            if (!(pkagent = virPolkitAgentCreate()))

[here virsh create a pkttyagent on *source* machine and i think this doesn't make any sense]

(gdb) 
Detaching after fork from child process 27959.
179	        virResetLastError();
(gdb) 
183	    } while (authfail < 5);
(gdb) p authfail
$1 = 0
(gdb) n
161	        if ((c = virConnectOpenAuth(uri, virConnectAuthPtrDefault,
(gdb) 
Detaching after fork from child process 28110.
lhuang@test1's password: 

[this is ssh authenticate again]

165	        if (readonly)
(gdb) n
168	        err = virGetLastError();
(gdb) 
169	        if (err && err->domain == VIR_FROM_POLKIT &&
(gdb) 
171	            if (!(pkagent = virPolkitAgentCreate()))
(gdb) 
Detaching after fork from child process 28134.

[Here virsh create another pkttyagent again]

** (pkttyagent:28134): WARNING **: Unable to register authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: An authentication agent already exists for the given subject
Error registering authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: An authentication agent already exists for the given subject (polkit-error-quark, 0)

[This is why pkttyagent report we already create a pkttyagent for this process]

179	        virResetLastError();
(gdb) n
183	    } while (authfail < 5);
(gdb) p authfail
$2 = 0

[we won't add authfail, and this make us cannot jump out of this loop]

(gdb) n
161	        if ((c = virConnectOpenAuth(uri, virConnectAuthPtrDefault,
(gdb) 
Detaching after fork from child process 28150.
lhuang@test1's password: 
165	        if (readonly)
(gdb) p authfail
$3 = 0
(gdb) c
Continuing.
Detaching after fork from child process 1550.

** (pkttyagent:1550): WARNING **: Unable to register authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: An authentication agent already exists for the given subject
Error registering authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: An authentication agent already exists for the given subject (polkit-error-quark, 0)
Detaching after fork from child process 1553.
lhuang@test1's password: 


And i can find many pkttyagent process in source machine before i kill the virsh client:

# ps aux|grep pktty
lhaung   14089  0.0  0.0 205184  3028 pts/12   Sl   09:09   0:00 /usr/bin/pkttyagent --process 14082 --notify-fd 7 --fallback
lhaung   14814  0.0  0.0 205184  3028 pts/12   Sl+  09:15   0:00 /usr/bin/pkttyagent --process 14809 --notify-fd 7 --fallback
lhaung   14898  0.0  0.0      0     0 pts/12   Z+   09:15   0:00 [pkttyagent] <defunct>
root     15008  0.0  0.0 112648   956 pts/0    S+   09:16   0:00 grep --color=auto pktty


I think maybe libvirt should not create a pkttyagent when the url is a remote
url.

--- Additional comment from John Ferlan on 2016-09-07 06:44:13 EDT ---

Just make a separate bug.  Luyao did great providing details and a reproducible example. Unfortunately I just didn't have enough time to devote to research those details. Additionally adding +ssh is different than the base issue/problem where polkit authentication just wasn't working for the non-GUI session.

Comment 1 John Ferlan 2017-05-11 15:33:07 UTC
I posted a couple of patches:

https://www.redhat.com/archives/libvir-list/2017-May/msg00320.html

Either of which will avoid the infinite authentication loop. The details are in the referenced cover letter posting. Either of the patches would work, the first would just cause an 'extra' authn challenge/response for something like ssh; whereas, the second one would fail after the first challenge/response failure.

Note that this "failure scenario" only occurs because the username is not root (or whatever name would result in a UID==0). I almost went down the path of well maybe we should allow this non-root path to check if the authn callerUid was in the right group to allow the access; however, that type of processing appears to be "outside" the scope of the problem description.

Comment 2 John Ferlan 2017-05-25 16:40:50 UTC
The following patch has been pushed upstream:

commit 2453501fc82d3b247affb6c9054dc65bf2f669b3
Author: John Ferlan <jferlan>
Date:   Thu May 11 09:17:09 2017 -0400

    virsh: Track when create pkttyagent
    
...
    
    Due to how the processing for authentication using polkit works, the
    virshConnect code must first "attempt" an virConnectOpenAuth and then
    check for a "special" return error code VIR_ERR_AUTH_UNAVAILABLE in
    order to attempt to "retry" the authentication after performing a creation
    of a pkttyagent to handle the challenge/response for the client.
    
    However, if pkttyagent creation is not possible for the authentication
    being attempted (such as perhaps a "qemu+ssh://someuser@localhost/system"),
    then the same failure pattern would be returned and another attempt to
    create a pkttyagent would be done. This would continue "forever" until
    someone forced quit (e.g. ctrl-c) from virsh as the 'authfail' was not
    incremented when creating the pkttyagent.
    
    So add a 'agentCreated' boolean to track if we've attempted to create the
    agent at least once and force a failure if that creation returned the same
    error pattern.
    
    This resolves a possible never ending loop and will generate an error:
    
    error: failed to connect to the hypervisor
    error: authentication unavailable: no polkit agent available to authenticate action 'org.libvirt.unix.manage'
    
    NB: If the authentication was for a sufficiently privileged client, such as
    qemu+ssh://root@localhost/system, then the remoteDispatchAuthList "allows"
    the authentication to use libvirt since @callerUid would be 0.


$ 
$ git describe 2453501fc82d3b247affb6c9054dc65bf2f669b3
v3.3.0-137-g2453501
$


It will have the failure scenario from the description where 2 attempts are made to authenticate, although the second will fail as follows:

$ virsh -c qemu+ssh://localhost/system list --all
someuser@localhost's password: 
someuser@localhost's password: 
error: failed to connect to the hypervisor
error: authentication unavailable: no polkit agent available to authenticate action 'org.libvirt.unix.manage'
$

or

$ virsh -c qemu+ssh://someuser@localhost/system list --all
someuser@localhost's password: 
someuser@localhost's password: 
error: failed to connect to the hypervisor
error: authentication unavailable: no polkit agent available to authenticate action 'org.libvirt.unix.manage'
$

The root access would still be allowed:

$ virsh -c qemu+ssh://root@localhost/system list --all
root@localhost's password: 
 Id    Name                           State
----------------------------------------------------
 -     domain                         shut off
...

Comment 5 yafu 2017-06-12 08:59:25 UTC
Reproduced with libvirt-2.0.0-1.el7.x86_64.

Verified pass with libvirt-3.2.0-9.virtcov.el7.x86_64.
Test steps:
1.Connect without setting user:
$ virsh -c qemu+ssh://localhost/system list
yafu@localhost's password: 
yafu@localhost's password: 
error: failed to connect to the hypervisor
error: authentication unavailable: no polkit agent available to authenticate action 'org.libvirt.unix.manage'

2.Connect with another non-root user:
$virsh -c qemu+ssh://test@localhost/system list
test@localhost's password: 
test@localhost's password: 
error: failed to connect to the hypervisor
error: authentication unavailable: no polkit agent available to authenticate action 'org.libvirt.unix.manage'

3.Connect with a non-exist user:
$ virsh -c qemu+ssh://test-none@localhost/system list
test-none@localhost's password: 
test-none@localhost's password: 
test-none@localhost's password: 
error: failed to connect to the hypervisor
error: Cannot recv data: Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).: Connection reset by peer

4.Connect with root:
$virsh -c qemu+ssh://root@localhost/system --list
root@localhost's password: 
 Id    Name                           State
----------------------------------------------------
 -     full-73                        shut off

Comment 6 errata-xmlrpc 2017-08-01 17:14:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 7 errata-xmlrpc 2017-08-01 23:55:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 8 errata-xmlrpc 2017-08-02 01:27:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846