Bug 953582

Summary: console from rhevm-shell/ovirt-shell using spicec does not work (linux client)
Product: Red Hat Enterprise Virtualization Manager Reporter: Sadique Puthen <sputhenp>
Component: ovirt-engine-cliAssignee: Michael Pasternak <mpastern>
Status: CLOSED CURRENTRELEASE QA Contact: Barak Dagan <bdagan>
Severity: high Docs Contact:
Priority: high    
Version: 3.1.0CC: acathrow, bazulay, dyasny, iheim, oramraz, pstehlik, Rhev-m-bugs, sgrinber, s.kong, ykaul
Target Milestone: ---Keywords: Reopened
Target Release: 3.2.0   
Hardware: All   
OS: Linux   
Whiteboard: virt
Fixed In Version: sf14 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 953340 Environment:
Last Closed: 2013-04-21 07:34:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 953340    
Bug Blocks:    
Attachments:
Description Flags
Working version of spice.py none

Description Sadique Puthen 2013-04-18 14:29:23 UTC 
+++ This bug was initially created as a clone of Bug #953340 +++

Description of problem:

when tying to connect to the console using rhevm-shell/ovirt-shell, the spicec does not want to connect. steps involved was 

rhevm-3.1.0-50.el6ev.noarch
rhevm-cli-3.1.1.2-1.el6ev.noarch

========================
same for ovirt-shell

Steps to Reproduce:
1. rhevm-shell -l https://[ip] --user user --password pass -A [locationOfCA.crt]
2. console [vmname]
3.
  
Actual results:

At This point, the spicec client open and close quickly, looking at the logs .spicec/spicec.log we can see that it looks like an SSL issue? 

1365413387 INFO [28493:28493] Application::main: starting 0.8.3
1365413387 INFO [28493:28493] init_key_map: using evdev mapping
1365413387 INFO [28493:28493] MultyMonScreen::MultyMonScreen: platform_win: 77594625
1365413387 INFO [28493:28493] ForeignMenu::ForeignMenu: Creating a foreign menu connection /tmp/SpiceForeignMenu-28493.uds
1365413387 INFO [28493:28494] RedPeer::connect_unsecure: Connected to 192.168.12.147 5906
1365413387 INFO [28493:28494] RedPeer::connect_secure: Connected to 192.168.12.147 5907
1365413387 ERROR [28493:28494] RedPeer::connect_secure: failed to connect w/SSL, ssl_error error:00000001:lib(0):func(0):reason(1)
1365413387 WARN [28493:28494] RedChannel::run: SSL Error:
1365413388 INFO [28493:28493] main: Spice client terminated (exitcode = 7)

Expected results:

spice console to vm

Additional info:


-----------------------------------------------------------------
how this was fixed,

#modify /usr/lib/python2.6/site-packages/ovirtcli/platform/posix/spice.py

#original file - at the end of the script, the &quot;else&quot; section, is not passing the ca-file and host-subject
    else:
        args.extend(['-h', host])
        args.extend(['-p', str(port)])
        if secport:
            args.extend([ '-s', str(secport) ])
        args.extend(['-w', ticket])
        args.extend(['-t', title])


#fixed file 
    if cmd.startswith('/usr/libexec'):
        args.extend([host])
        args.extend([str(port)])
        if secport:
            args.extend([str(secport)])
            args.extend(['--ssl-channels', 'smain,sinputs'])
            args.extend(['--ca-file', certfile])
            if host_subject and host_subject != '':
                args.extend(['--host-subject', host_subject])
        args.extend(['-p', ticket])
    else:
        args.extend(['-h', host])
        args.extend(['-p', str(port)])
        if secport:
            args.extend([ '-s', str(secport) ])
            args.extend(['--ca-file', certfile])
            if host_subject and host_subject != '':
                args.extend(['--host-subject', host_subject])
        args.extend(['-w', ticket])
        args.extend(['-t', title])
    pid, pstdin = util.spawn(cmd, args, debug
------------------------------------------------------------------

i attached the file with the fix
Comment 2 Michael Pasternak 2013-04-21 07:34:09 UTC 

*** This bug has been marked as a duplicate of bug 812299 ***
Comment 3 Simon Kong Win Chang 2013-04-22 00:09:42 UTC 
No the other bug is different, their case, they were not using SSL and the cli was passing through the -s port. in this case, we ARE using SSL, but the CLI is NOT passing through --ca-file and --host-subject .

please remove this as being a duplicate/closed.
Comment 8 Ilia Meerovich 2013-05-13 09:04:30 UTC 
looks that it is not working with sf15:

[RHEVM shell (connected)]# console restvm_tags_users
send: 'GET /api/vms?search=name%3Drestvm_tags_users HTTP/1.1\r\nHost: leonid_rhevm.qa.lab.tlv.redhat.com\r\nAccept-Encoding: identity\r\nFilter: False\r\ncookie: JSESSIONID=La63Qg8sh4VaAlWAaSKU-7TZ\r\nPrefer: persistent-auth\r\nContent-type: application/xml\r\nAccept: application/xml\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Mon, 13 May 2013 08:52:46 GMT
header: Pragma: No-cache
header: Cache-Control: no-cache
header: Expires: Thu, 01 Jan 1970 02:00:00 IST
header: Content-Type: application/xml
header: Content-Length: 3609
header: Connection: close
body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<vms>
    <vm href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73" id="61d6eaf3-fb9f-4edb-8799-0316d72bec73">
        <actions>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/ticket" rel="ticket"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/migrate" rel="migrate"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/cancelmigration" rel="cancelmigration"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/shutdown" rel="shutdown"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/start" rel="start"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/stop" rel="stop"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/suspend" rel="suspend"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/detach" rel="detach"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/export" rel="export"/>
            <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/move" rel="move"/>
        </actions>
        <name>restvm_tags_users</name>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/disks" rel="disks"/>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/nics" rel="nics"/>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/cdroms" rel="cdroms"/>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/snapshots" rel="snapshots"/>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/tags" rel="tags"/>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/permissions" rel="permissions"/>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/statistics" rel="statistics"/>
        <link href="/api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/reporteddevices" rel="reporteddevices"/>
        <type>desktop</type>
        <status>
            <state>up</state>
        </status>
        <memory>536870912</memory>
        <cpu>
            <topology sockets="1" cores="1"/>
        </cpu>
        <os type="unassigned">
            <boot dev="hd"/>
        </os>
        <high_availability>
            <enabled>false</enabled>
            <priority>0</priority>
        </high_availability>
        <display>
            <type>spice</type>
            <address>10.35.102.103</address>
            <port>5900</port>
            <secure_port>5901</secure_port>
            <monitors>1</monitors>
            <allow_override>false</allow_override>
            <smartcard_enabled>false</smartcard_enabled>
        </display>
        <host href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba" id="7e950e2d-1b27-4c04-af46-82951f5519ba"/>
        <cluster href="/api/clusters/602a9103-7214-48b4-97e0-cf7ff0e0bd17" id="602a9103-7214-48b4-97e0-cf7ff0e0bd17"/>
        <template href="/api/templates/00000000-0000-0000-0000-000000000000" id="00000000-0000-0000-0000-000000000000"/>
        <start_time>2013-05-13T11:49:41.114+03:00</start_time>
        <creation_time>2013-05-13T11:48:17.171+03:00</creation_time>
        <origin>rhev</origin>
        <stateless>false</stateless>
        <delete_protected>false</delete_protected>
        <placement_policy>
            <affinity>migratable</affinity>
        </placement_policy>
        <memory_policy>
            <guaranteed>536870912</guaranteed>
        </memory_policy>
        <quota id="00000000-0000-0000-0000-000000000000"/>
        <usb>
            <enabled>false</enabled>
        </usb>
    </vm>
</vms>

send: 'POST /api/vms/61d6eaf3-fb9f-4edb-8799-0316d72bec73/ticket HTTP/1.1\r\nHost: leonid_rhevm.qa.lab.tlv.redhat.com\r\nAccept-Encoding: identity\r\nContent-Length: 10\r\nFilter: False\r\nPrefer: persistent-auth\r\ncookie: JSESSIONID=La63Qg8sh4VaAlWAaSKU-7TZ\r\nContent-type: application/xml\r\nAccept: application/xml\r\n\r\n'
send: '<action/>\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Mon, 13 May 2013 08:52:46 GMT
header: Content-Type: application/xml
header: Content-Length: 227
header: Connection: close
body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<action>
    <ticket>
        <value>Mci3AwtP8rgL</value>
        <expiry>7200</expiry>
    </ticket>
    <status>
        <state>complete</state>
    </status>
</action>

send: 'GET /api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba HTTP/1.1\r\nHost: leonid_rhevm.qa.lab.tlv.redhat.com\r\nAccept-Encoding: identity\r\nFilter: False\r\ncookie: JSESSIONID=La63Qg8sh4VaAlWAaSKU-7TZ\r\nPrefer: persistent-auth\r\nContent-type: application/xml\r\nAccept: application/xml\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Mon, 13 May 2013 08:52:46 GMT
header: Pragma: No-cache
header: Cache-Control: no-cache
header: Expires: Thu, 01 Jan 1970 02:00:00 IST
header: Content-Type: application/xml
header: Content-Length: 3049
header: Connection: close
body:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<host href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba" id="7e950e2d-1b27-4c04-af46-82951f5519ba">
    <actions>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/deactivate" rel="deactivate"/>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/approve" rel="approve"/>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/iscsilogin" rel="iscsilogin"/>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/iscsidiscover" rel="iscsidiscover"/>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/commitnetconfig" rel="commitnetconfig"/>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/fence" rel="fence"/>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/install" rel="install"/>
        <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/activate" rel="activate"/>
    </actions>
    <name>10.35.102.103</name>
    <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/storage" rel="storage"/>
    <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/nics" rel="nics"/>
    <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/tags" rel="tags"/>
    <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/permissions" rel="permissions"/>
    <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/statistics" rel="statistics"/>
    <link href="/api/hosts/7e950e2d-1b27-4c04-af46-82951f5519ba/hooks" rel="hooks"/>
    <address>10.35.102.103</address>
    <certificate>
        <organization>qa.lab.tlv.redhat.com</organization>
        <subject>O=qa.lab.tlv.redhat.com,CN=10.35.102.103</subject>
    </certificate>
    <status>
        <state>up</state>
    </status>
    <cluster href="/api/clusters/602a9103-7214-48b4-97e0-cf7ff0e0bd17" id="602a9103-7214-48b4-97e0-cf7ff0e0bd17"/>
    <port>54321</port>
    <type>rhel</type>
    <storage_manager priority="5">true</storage_manager>
    <version major="4" minor="10" build="2" revision="0" full_version="vdsm-4.10.2-15.0.el6ev"/>
    <hardware_information/>
    <power_management>
        <enabled>false</enabled>
        <options/>
    </power_management>
    <ksm>
        <enabled>false</enabled>
    </ksm>
    <transparent_hugepages>
        <enabled>true</enabled>
    </transparent_hugepages>
    <iscsi>
        <initiator>iqn.1994-05.com.redhat:398a382f492c</initiator>
    </iscsi>
    <cpu>
        <topology sockets="1" cores="4" threads="2"/>
        <name>Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz</name>
        <speed>2394</speed>
    </cpu>
    <memory>16712204288</memory>
    <max_scheduling_memory>32482787328</max_scheduling_memory>
    <summary>
        <active>1</active>
        <migrating>0</migrating>
        <total>1</total>
    </summary>
    <os type="RHEL">
        <version full_version="6Server - 6.3.0.3.el6"/>
    </os>
    <libvirt_version major="0" minor="10" build="2" revision="0" full_version="libvirt-0.10.2-18.el6_4.3"/>
</host>

Error: failed to connect w/SSL, ssl_error error:00000001:lib(0):func(0):reason(1)
140509620618504:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1063:
Warning: SSL Error:
[RHEVM shell (connected)]# 
[1]+  Stopped                 /usr/bin/rhevm-shell -c --url=https://leonid_rhevm.qa.lab.tlv.redhat.com:443/api/ -u admin@internal -I -d
[imeerovi@imeerovi ART]$ rpm -q rhevm-cli
rhevm-cli-3.2.0.9-1.el6ev.noarch
[imeerovi@imeerovi ART]$ rpm -q rhevm-sdk
rhevm-sdk-3.2.0.10-1.el6ev.noarch
[imeerovi@imeerovi ART]$
Comment 9 Michael Pasternak 2013-05-13 09:54:59 UTC 
(In reply to comment #8)
> looks that it is not working with sf15:
> 
> Error: failed to connect w/SSL, ssl_error
> error:00000001:lib(0):func(0):reason(1)
> 140509620618504:error:14090086:SSL
> routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify
> failed:s3_clnt.c:1063:

note it fails on "certificate verify".

> Warning: SSL Error:
> [RHEVM shell (connected)]# 
> [1]+  Stopped                 /usr/bin/rhevm-shell -c
> --url=https://leonid_rhevm.qa.lab.tlv.redhat.com:443/api/ -u admin@internal
> -I -d
> [imeerovi@imeerovi ART]$ rpm -q rhevm-cli
> rhevm-cli-3.2.0.9-1.el6ev.noarch
> [imeerovi@imeerovi ART]$ rpm -q rhevm-sdk
> rhevm-sdk-3.2.0.10-1.el6ev.noarch
> [imeerovi@imeerovi ART]$
Comment 10 Simon Kong Win Chang 2013-05-13 23:39:54 UTC 
Created attachment 747469 [details]
Working version of spice.py
Comment 11 Simon Kong Win Chang 2013-05-13 23:41:11 UTC 
as stated before, and on https://bugzilla.redhat.com/show_bug.cgi?id=953340

i have already described how i fixed the issue, and uploaded the working version.
Comment 12 Michael Pasternak 2013-05-14 06:45:09 UTC 
(In reply to comment #11)
> as stated before, and on https://bugzilla.redhat.com/show_bug.cgi?id=953340
> 
> i have already described how i fixed the issue, and uploaded the working
> version.

Simon,

- this bug is a clone of upstream 953340 and addresses mentioned issue
- AFAICS illia experiences different issue, he has:

error:00000001:lib(0):func(0):reason(1)
140509620618504:error:14090086:SSL
routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify
failed:s3_clnt.c:1063:

while you had:

1365413387 ERROR [28493:28494] RedPeer::connect_secure: failed to connect w/SSL, ssl_error error:00000001:lib(0):func(0):reason(1)
1365413387 WARN [28493:28494] RedChannel::run: SSL Error:
Comment 13 Simon Kong Win Chang 2013-05-15 00:37:24 UTC 
the error logs that i posted was extracted from /home/USER/.spicec/spicec.log

they might not output the same level of debug info hence why the message might not be the same, not sure.


may i suggest to replace /usr/lib/python2.6/site-packages/ovirtcli/platform/posix/spice.py on your client with the version that i uploaded and check if it works.

the stuff that i added in the code, was to make the script pass through the ca-file and subject which are required for ssl.



p.s also, i am connecting to the manager by also specifying the CA file (if that makes any difference. 

rhevm-shell -l https://[ip] -A [locationOfCA.crt]
Comment 14 Michael Pasternak 2013-05-19 10:02:27 UTC 
(In reply to comment #13)
> the error logs that i posted was extracted from /home/USER/.spicec/spicec.log
> 
> they might not output the same level of debug info hence why the message
> might not be the same, not sure.
> 
> 
> may i suggest to replace
> /usr/lib/python2.6/site-packages/ovirtcli/platform/posix/spice.py on your
> client with the version that i uploaded and check if it works.
> 
> the stuff that i added in the code, was to make the script pass through the
> ca-file and subject which are required for ssl.
> 

mentioned cli release does that.
Comment 15 Michael Pasternak 2013-05-19 10:06:01 UTC 
(In reply to comment #13)
> the error logs that i posted was extracted from /home/USER/.spicec/spicec.log
> 
> they might not output the same level of debug info hence why the message
> might not be the same, not sure.
> 
> 
> may i suggest to replace
> /usr/lib/python2.6/site-packages/ovirtcli/platform/posix/spice.py on your
> client with the version that i uploaded and check if it works.
> 
> the stuff that i added in the code, was to make the script pass through the
> ca-file and subject which are required for ssl.
> 
> 
> 
> p.s also, i am connecting to the manager by also specifying the CA file (if
> that makes any difference. 
> 
> rhevm-shell -l https://[ip] -A [locationOfCA.crt]

here ca used to authenticate with engine backend, not spice, afaik it
should be installed locally, arnon?
Comment 16 Barak Dagan 2013-05-22 07:28:26 UTC 
Verified 
[RHEVM shell (connected)]# info

backend version: 3.2 (SF17)
sdk version    : 3.2.0.11
cli version    : 3.2.0.9

need to perform the following installations first:

  yum install spice-client.x86_64
  yum install rhevm-cli.noarch
  yum groupinstall "X Window System" "GNOME Desktop Environment"
  yum install xclock


[root@localhost ~]# rhevm-shell -c -l "https://[ip]:443/api" -u [user@domain] --insecure
Password: xxxxxxx


[RHEVM shell (connected)]# console [vmname]
Comment 17 Itamar Heim 2013-06-11 08:56:45 UTC 
3.2 has been released
Comment 18 Itamar Heim 2013-06-11 08:56:45 UTC 
3.2 has been released
Comment 19 Itamar Heim 2013-06-11 08:56:53 UTC 
3.2 has been released
Comment 20 Itamar Heim 2013-06-11 08:59:16 UTC 
3.2 has been released
Comment 21 Itamar Heim 2013-06-11 09:28:41 UTC 
3.2 has been released