Bug 737176

Summary: SSH GSSAPI login broken
Product: Red Hat Enterprise Linux 6 Reporter: Dave Allan <dallan>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.2CC: acathrow, ajia, crobinso, dallan, dyuan, mprivozn, mzhan, rwu, weizhan, whuang, witte, xen-maint
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.9.4-12.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 736983 Environment:
Last Closed: 2011-12-06 11:28:51 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 736983    
Bug Blocks: 743047    

Description Dave Allan 2011-09-09 19:26:39 UTC
+++ This bug was initially created as a clone of Bug #736983 +++

Created attachment 522273 [details]
Patch adding KRB5CCNAME to virnetsocket.c

Description of problem:



Version-Release number of selected component (if applicable):

AFAICS this corresponds with the changes in src/rpc since
0.9.2. 


Additional info:

ssh can no longer see KRB5CCNAME and therefore ssh GSSAPI
authentication works no longer.

Adding virCommandAddEnvPass(cmd, "KRB5CCNAME"); to  

libvirt-0.9.5-rc1/src/rpc/virnetsocket.c

resolves the problem.

--- Additional comment from dallan on 2011-09-09 08:37:38 EDT ---

Hi Matthias,

Thanks for the bug report and the patch.  Would you mind submitting it to libvir-list for discussion?

Dave

--- Additional comment from mprivozn on 2011-09-09 10:07:57 EDT ---

Moving to POST:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2011-September/msg00320.html

Comment 4 Huang Wenlong 2011-09-21 09:47:47 UTC
Hi, Michal Privoznik 

I want to verify this bug , but I do not know how to repruduce this bug or verify it , could you tell me how thanks very much .

Wenlong

Comment 5 Matthias Witte 2011-09-21 10:32:23 UTC
To reproduce this bug do this:

you need a client (foo) with virsh installed and a server (bar) in a kerberized environment running the libvirt daemon and an ssh Server with gssapi authentication enabled: "GSSAPIAuthentication yes" in sshd_config. Additionally
set LogLevel to DEBUG in sshd_config and restart the sshd.

Obtain a ticket on the client:

baz@foo:~ kinit

Do an ssh Login on the server:

baz@foo:~ ssh -v bar

If everything is configured correctly you must not be asked for a password.

On the server you will find these lines in the auth.log:

Sep 21 12:11:49 bar sshd[15136]: Authorized to baz, krb5 principal  baz@REALM (krb5_kuserok)
Sep 21 12:11:49 bar sshd[15136]: Accepted gssapi-with-mic for baz from 1.2.3.4 port 36011 ssh2

The debugging output of your client should include these lines:

debug1: Next authentication method: gssapi-with-mic
debug1: Delegating credentials
debug1: Delegating credentials
debug1: Authentication succeeded (gssapi-with-mic).

Then try to access the libvirt daemon from an xterm with:

baz@foo:~ virsh qemu+ssh://bar/system

Passwordless Login will not succeed.

If you 'strace -f' the virsh process you should see lines like these:

baz@foo:~ grep -A 1 'gssapi' virsh.log
[pid 26698] write(2, "debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
[pid 26698] write(2, "debug1: Next authentication method: gssapi-keyex\r\n", 50) = 50
[pid 26698] write(2, "debug1: No valid Key exchange context\r\n", 39) = 39
[pid 26698] write(2, "debug1: Next authentication method: gssapi-with-mic\r\n", 53) = 53
[pid 26698] write(2, "debug1: Next authentication method: publickey\r\n", 47) = 47
--
[pid 26698] write(2, "debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
[pid 26698] write(2, "debug1: Offering public key: /home/baz/.ssh/id_dsa\r\n", 52) = 52
--
[pid 26698] write(2, "debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
[pid 26698] write(2, "debug1: Next authentication method: password\r\n", 46) = 46
--
[pid 26698] write(2, "debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
[pid 26698] write(2, "Permission denied, please try again.\r\n", 38) = 38
--
[pid 26698] write(2, "debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
[pid 26698] write(2, "Permission denied, please try again.\r\n", 38) = 38
--
[pid 26698] write(2, "debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
[pid 26698] write(2, "debug1: No more authentication methods to try.\r\n", 48) = 48
[pid 26698] write(2, "Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).\r\n", 70) = 70
[pid 26698] exit_group(255)             = ?

Comment 6 Huang Wenlong 2011-09-22 05:42:48 UTC
(In reply to comment #5)
> To reproduce this bug do this:
> 
> you need a client (foo) with virsh installed and a server (bar) in a kerberized
> environment running the libvirt daemon and an ssh Server with gssapi
> authentication enabled: "GSSAPIAuthentication yes" in sshd_config. Additionally
> set LogLevel to DEBUG in sshd_config and restart the sshd.
> 
> Obtain a ticket on the client:
> 
> baz@foo:~ kinit
> 
> Do an ssh Login on the server:
> 
> baz@foo:~ ssh -v bar
> 
> If everything is configured correctly you must not be asked for a password.
> 
> On the server you will find these lines in the auth.log:
> 
> Sep 21 12:11:49 bar sshd[15136]: Authorized to baz, krb5 principal  baz@REALM
> (krb5_kuserok)
> Sep 21 12:11:49 bar sshd[15136]: Accepted gssapi-with-mic for baz from 1.2.3.4
> port 36011 ssh2
> 
> The debugging output of your client should include these lines:
> 
> debug1: Next authentication method: gssapi-with-mic
> debug1: Delegating credentials
> debug1: Delegating credentials
> debug1: Authentication succeeded (gssapi-with-mic).
> 
> Then try to access the libvirt daemon from an xterm with:
> 
> baz@foo:~ virsh qemu+ssh://bar/system
> 
> Passwordless Login will not succeed.
> 
> If you 'strace -f' the virsh process you should see lines like these:
> 
> baz@foo:~ grep -A 1 'gssapi' virsh.log
> [pid 26698] write(2, "debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
> [pid 26698] write(2, "debug1: Next authentication method: gssapi-keyex\r\n",
> 50) = 50
> [pid 26698] write(2, "debug1: No valid Key exchange context\r\n", 39) = 39
> [pid 26698] write(2, "debug1: Next authentication method: gssapi-with-mic\r\n",
> 53) = 53
> [pid 26698] write(2, "debug1: Next authentication method: publickey\r\n", 47) =
> 47
> --
> [pid 26698] write(2, "debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
> [pid 26698] write(2, "debug1: Offering public key: /home/baz/.ssh/id_dsa\r\n",
> 52) = 52
> --
> [pid 26698] write(2, "debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
> [pid 26698] write(2, "debug1: Next authentication method: password\r\n", 46) =
> 46
> --
> [pid 26698] write(2, "debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
> [pid 26698] write(2, "Permission denied, please try again.\r\n", 38) = 38
> --
> [pid 26698] write(2, "debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
> [pid 26698] write(2, "Permission denied, please try again.\r\n", 38) = 38
> --
> [pid 26698] write(2, "debug1: Authentications that can continue:
> publickey,gssapi-keyex,gssapi-with-mic,password\r\n", 92) = 92
> [pid 26698] write(2, "debug1: No more authentication methods to try.\r\n", 48)
> = 48
> [pid 26698] write(2, "Permission denied
> (publickey,gssapi-keyex,gssapi-with-mic,password).\r\n", 70) = 70
> [pid 26698] exit_group(255)             = ?

Hi, Matthias Witte 

I have not kerberized envirnment , I just use a var to test it , 

test steps : 
1. export KRB5CCNAME=libvirt_krb_test  
2. export LIBVIRT_DEBUG=1
3. virsh -c qemu+ssh://10.66.5.12/system
...
virCommandRunAsync:2042 : About to run LC_ALL=C PATH=/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin HOME=/root USER=root LOGNAME=root KRB5CCNAME=libvirt_krb_test SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass DISPLAY=localhost:11.0 ssh 10.66.5.12 nc -U /var/run/libvirt/libvirt-sock
...

4. check 
the $KRB5CCNAME is correct . 

So I think the path works .
If you have the kerberos envirnment  ,could you help me to test it , thanks very much .

Comment 7 Rita Wu 2011-10-09 10:17:04 UTC
Hi Matthias,

Would you pls help to verify this bug per commet 6?

Thanks,
Rita

Comment 8 Huang Wenlong 2011-10-21 05:57:09 UTC
Verify this bug with libvirt-0.9.4-19.el6.x86_64 

1) setup a KDC server with libvirt on one host  

2) On the client  get the ticket for libvirt 

1. kinit  

check the ticket your get
2. klist   
Default principal: root.REDHAT.COM

Valid starting     Expires            Service principal
10/21/11 13:33:10  10/22/11 13:33:10  krbtgt/WHUANG.NAY.REDHAT.COM.REDHAT.COM
	renew until 10/21/11 13:33:10
10/21/11 13:33:16  10/22/11 13:33:10  host/whuang.nay.redhat.com.REDHAT.COM
	renew until 10/21/11 13:33:10

3. virsh -c qemu+ssh://$libvirt_ip/system  


It get connection without ssh passwd .

Comment 9 Matthias Witte 2011-10-26 13:31:21 UTC
(In reply to comment #7)
> Hi Matthias,
> 
> Would you pls help to verify this bug per commet 6?
> 
> Thanks,
> Rita

I apologise for my late reply.

Here is my verification:

1. Disable passing KRB5CCNAME in src/rpc/virnetsocket.c and compile 0.9.6

2. Connect to VM Host using virsh

$ LIBVIRT_DEBUG=1 virsh -c qemu+ssh://foo/system
[...]
14:09:11.043: 12278: debug : virCommandRunAsync:2043 : About to run LC_ALL=C PATH=/sbin:/usr/sbin:/usr/local/sbin:/home/baz/bin:/usr/local/bin:/usr/bin:/bin HOME=/home/baz USER=baz LOGNAME=baz SSH_AUTH_SOCK=/tmp/keyring-B3FC1f/ssh DISPLAY=:0.0 ssh foo nc -U /var/run/libvirt/libvirt-sock
[...]

Result: I get a password prompt, KRB5CCNAME ist not passed in virCommandRunAsync

3. Reenable passing KRB5CCNAME and GSSAPI Login works

15:26:30.915: 7177: debug : virCommandRunAsync:2043 : About to run LC_ALL=C PATH=/sbin:/usr/sbin:/usr/local/sbin:/home/baz/bin:/usr/local/bin:/usr/bin:/bin HOME=/home/baz USER=baz LOGNAME=baz KRB5CCNAME=FILE:/tmp/krb5cc_1000_esIWWB SSH_AUTH_SOCK=/tmp/keyring-B3FC1f/ssh DISPLAY=:0.0 ssh foo nc -U /var/run/libvirt/libvirt-sock


All in all I think Wenlongs Test is correct.

Comment 11 errata-xmlrpc 2011-12-06 11:28:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html