Bug 904295

Summary: virt-manager console doesn't connect to SPICE with TLS
Product: [Fedora] Fedora Reporter: Anthony Messina <amessina>
Component: virt-managerAssignee: Cole Robinson <crobinso>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: amessina, berrange, crobinso, hbrock, herrold, jforbes, marcandre.lureau, virt-maint
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: virt-manager-0.10.0-4.fc19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-10-10 00:56:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Proof of concept patch (please don't mind my complete ignorance when it comes to Python) none

Description Anthony Messina 2013-01-26 01:08:13 UTC
Created attachment 687779 [details]
Proof of concept patch (please don't mind my complete ignorance when it comes to Python)

As libvirtd doesn't yet seem to support SASL connections for SPICE, I investigated using TLS for my connections only to find that virt-manager's graphical console does not "upgrade" or switch to TLS even though TLS is offered by the libvirtd server.

virt-viewer *does* switch to TLS.

I'll attach a proof-of-concept patch to support connecting over TLS with SPICE in the virt-manager console.  I'm *sure* there's a better way.

Comment 1 Cole Robinson 2013-02-06 22:50:53 UTC
Hi Anthony, thanks a lot for the patch! In general it looks fine, but I'd like to test it, but never used spice tls before (hence the virt-manager brokenness). How are you verifying that TLS is actually working?

I see there's docs about how to do it all:

http://publib.boulder.ibm.com/infocenter/lnxinfo/v3r0m0/index.jsp?topic=%2Fliaat%2Fliaatsecspiceserver.htm

But if you have any other pointers that would be helpful

Comment 2 Anthony Messina 2013-02-15 07:44:34 UTC
Thanks, Cole.  The instructions above are what I used after searching to figure out how to use TLS with Spice.  My original intent was to use SASL/GSSAPI as I already have a Kerberos-enabled network and VM setup which I had used with VNC.  I had looked into spice as it seemed to provide smartcard support, which I was investigating.

I realize this isn't the place for support/questions, etc.  I am just noticing that there are options available for VM domain configuration (http://libvirt.org/formatdomain.html), which are not available in virt-manager.

Is it possible to have a way to add these options available to virt-manager?

Anyway, as far as TLS, I do believe it is working as I have enabled the full certificate verification on the server/clients, etc. and when I try to connect without them, the connection fails.

Again, my patch was supposed to be proof of concept, based on digging through the code and duplicating what I saw for other variables.  I'm certain there is a more robust way to implement this feature, as well as enable SASL support for Spice as it is in VNC.

That said, I'm using my patch on my live VM setup every day.

I'll be happy to help test, should you need a tester.

Comment 3 Anthony Messina 2013-02-21 22:33:31 UTC
I'm not sure how to properly sniff the connection, but I don't think this patch fixes TLS.  I see far too much VNC protocol in the wireshark output, but like I said, this is perhaps proof of concept ;)

Comment 4 Cole Robinson 2013-09-01 20:12:24 UTC
Finally got around to testing this. One thing that helps is telling your guest to mandate TLS for the main spice channel, which means virt-manager will throw an explicit error if TLS isn't set up correctly. Add this to you <graphics type='spice'> XML

<channel name='main' mode='secure'/>

Anyways, I think it's working upstream. it's more or less what your patch was doing, except we defer to the spice-gtk default CA path rather than invent our own location:

commit cdc82e62a236d0e737851c693d6673f65ca278c1
Author: Cole Robinson <crobinso>
Date:   Sun Sep 1 16:08:14 2013 -0400

    console: Fix spice with TLS (bz #904295)
    
    We now use the same CA trust store as virt-viewer's default, which is:
    
    ~/.spicec/spice_truststore.pem

Comment 5 Anthony Messina 2013-09-02 16:40:23 UTC
(In reply to Cole Robinson from comment #4)
> Finally got around to testing this. One thing that helps is telling your
> guest to mandate TLS for the main spice channel, which means virt-manager
> will throw an explicit error if TLS isn't set up correctly. Add this to you
> <graphics type='spice'> XML
> 
> <channel name='main' mode='secure'/>
> 
> Anyways, I think it's working upstream. it's more or less what your patch
> was doing, except we defer to the spice-gtk default CA path rather than
> invent our own location:
> 
> commit cdc82e62a236d0e737851c693d6673f65ca278c1
> Author: Cole Robinson <crobinso>
> Date:   Sun Sep 1 16:08:14 2013 -0400
> 
>     console: Fix spice with TLS (bz #904295)
>     
>     We now use the same CA trust store as virt-viewer's default, which is:
>     
>     ~/.spicec/spice_truststore.pem

Thanks for the follow up.  I'll look forward to the next release, which hopefully has this support included.  Also, is there a way (it would be nice) to have virt-manager enable some options for <channel name='main' mode='secure'/> when creating VMs to ensure that TLS can be *enforced* from the server side, rather than allowing the client to choose?  Ideally, I'd like to see options to secure all spice channels with TLS.

Comment 6 Cole Robinson 2013-09-02 16:50:06 UTC
> 
> Thanks for the follow up.  I'll look forward to the next release, which
> hopefully has this support included.  Also, is there a way (it would be
> nice) to have virt-manager enable some options for <channel name='main'
> mode='secure'/> when creating VMs to ensure that TLS can be *enforced* from
> the server side, rather than allowing the client to choose?  Ideally, I'd
> like to see options to secure all spice channels with TLS.

I've been trying to move away from the 'virt-manager exposes every XML bit in the UI' paradigm recently. Since TLS is something that will never work out of the box, it's not something I feel compelled to enable for creating new VMs. Especially that not many people are likely using it anyways.

Instead I'd rather add support to virt-install --graphics for it. We already know how to parse the XML, just need to wire up --graphics channel_main_mode=secure or similar. If you're interested, please file an upstream virt-install bug for that: http://virt-manager.org/bugs/#report

The other option is that maybe libvirt could add an option in /etc/libvirt/qemu.conf that makes 'secure' the channel default for new VMs, similar to how you can set a default password for all new VMs. In fact I'd argue libvirt should have done that automatically if spice_tls is enabled, but it's probably too late to change that now. http://libvirt.org/bugs.html

Comment 7 Anthony Messina 2013-09-03 17:53:41 UTC
(In reply to Cole Robinson from comment #6)
> I've been trying to move away from the 'virt-manager exposes every XML bit
> in the UI' paradigm recently. Since TLS is something that will never work
> out of the box, it's not something I feel compelled to enable for creating
> new VMs. Especially that not many people are likely using it anyways.

I can understand this, from the developer/coder standpoint.  I guess it would be difficult to choose which XML bits are significant enough to expose.  I am not sure how many folks are using TLS, but it seems as though Red Hat, Fedora or another enterpise-like entity would be using it, but then, perhaps they aren't using virt-manager to manage the machines (what *do* they use then?)

> Instead I'd rather add support to virt-install --graphics for it. We already
> know how to parse the XML, just need to wire up --graphics
> channel_main_mode=secure or similar. If you're interested, please file an
> upstream virt-install bug for that: http://virt-manager.org/bugs/#report

According to http://libvirt.org/formatdomain.html#elementsGraphics for the "spice" graphics channel,

"The defaultMode attribute sets the default channel security policy, valid values are secure, insecure and the default any (which is secure if possible, but falls back to insecure rather than erroring out if no secure path is available)."

Perhaps a simple way would be to add a dropdown for "defaultMode" with those selections to the "Display Spice" configuration, with an "information hover" point (similar to the SELinux choices in "Overview"), stating that support for TLS-secured channels must be configured on the hypervisor, etc. Maybe that's too much to ask, but from the small-time sysadmin perspective, the ability to create and manage VMs so quickly with virt-manager has been a blessing!  Not only has virt-manage been a good ongoing management tool, but it was a portal for me to begin using VMs in the first place.
 
> The other option is that maybe libvirt could add an option in
> /etc/libvirt/qemu.conf that makes 'secure' the channel default for new VMs,
> similar to how you can set a default password for all new VMs. In fact I'd
> argue libvirt should have done that automatically if spice_tls is enabled,
> but it's probably too late to change that now. http://libvirt.org/bugs.html

I agree that it would be nice to have this in libvirt.  But what do you mean "too late to change that now"?

Personally, I never would have even started this complicated TLS stuff if the libvirt builds enabled SASL for Spice.  http://spice-space.org/page/Features/SASL suggests that the support may be there, but I can't tell for sure.

...

Back on track now to the support introduced in cdc82e62a236d0e737851c693d6673f65ca278c1.  So once that filters through to the next Fedora (19) build of virt-manager, I should be able to use virsh-edit to edit my domain as you described, and I should be good-to-go.

Thank you.

Comment 8 Cole Robinson 2013-09-03 18:11:42 UTC
> 
> I can understand this, from the developer/coder standpoint.  I guess it
> would be difficult to choose which XML bits are significant enough to
> expose.  I am not sure how many folks are using TLS, but it seems as though
> Red Hat, Fedora or another enterpise-like entity would be using it, but
> then, perhaps they aren't using virt-manager to manage the machines (what
> *do* they use then?)
> 

People using advanced spice features are RHEV/ovirt consumers doing virtual desktop stuff. In those cases RHEV is automatically setting up all the security necessary because it owns the whole server farm, virt-manager isn't like that obviously.

> > Instead I'd rather add support to virt-install --graphics for it. We already
> > know how to parse the XML, just need to wire up --graphics
> > channel_main_mode=secure or similar. If you're interested, please file an
> > upstream virt-install bug for that: http://virt-manager.org/bugs/#report
> 
> According to http://libvirt.org/formatdomain.html#elementsGraphics for the
> "spice" graphics channel,
> 
> "The defaultMode attribute sets the default channel security policy, valid
> values are secure, insecure and the default any (which is secure if
> possible, but falls back to insecure rather than erroring out if no secure
> path is available)."
> 
> Perhaps a simple way would be to add a dropdown for "defaultMode" with those
> selections to the "Display Spice" configuration, with an "information hover"
> point (similar to the SELinux choices in "Overview"), stating that support
> for TLS-secured channels must be configured on the hypervisor, etc. Maybe
> that's too much to ask, but from the small-time sysadmin perspective, the
> ability to create and manage VMs so quickly with virt-manager has been a
> blessing!  Not only has virt-manage been a good ongoing management tool, but
> it was a portal for me to begin using VMs in the first place.
>  

I didn't know about defaultMode, that would make the UI simpler certainly, but it's also kind of perilous. Sticking that in the UI with a dropdown with 'secure', 'insecure', 'default', a user will see it say 'why wouldn't I want security', turn it on, and suddenly they can't connect. That's not doing users any favors, and virt-manager will never be able to automagically configure the whole TLS setup. Someone who can follow instructions to set up certificates, etc, can tweak something manually in the libvirt XML as well.

> > The other option is that maybe libvirt could add an option in
> > /etc/libvirt/qemu.conf that makes 'secure' the channel default for new VMs,
> > similar to how you can set a default password for all new VMs. In fact I'd
> > argue libvirt should have done that automatically if spice_tls is enabled,
> > but it's probably too late to change that now. http://libvirt.org/bugs.html
> 
> I agree that it would be nice to have this in libvirt.  But what do you mean
> "too late to change that now"?
> 

I meant, too late to change the behavior of the existing spice_tls option to imply defaultMode=secure for all new VMs. If I was implementing that feature from scratch I'd make it do that, but we can't change that behavior now. We can however add a new qemu.conf option 'spice_channel_default_mode' which maps to defaultMode if none is explicitly specified in the XML. So in addition to setting spice_tls=1, you also set spice_channel_default_mode=secure, and get the behavior you want. But someone needs to file a bug for that (hint hint :) )

> Personally, I never would have even started this complicated TLS stuff if
> the libvirt builds enabled SASL for Spice. 
> http://spice-space.org/page/Features/SASL suggests that the support may be
> there, but I can't tell for sure.
> 

Hmm, looking at the libvirt code, it doesn't appear to have ever been wired up for spice, though it is supported for VNC. elmarco, what ever happened to libvirt + spice + sasl support?

Comment 9 Marc-Andre Lureau 2013-09-03 18:19:01 UTC
(In reply to Cole Robinson from comment #8)
> > Personally, I never would have even started this complicated TLS stuff if
> > the libvirt builds enabled SASL for Spice. 
> > http://spice-space.org/page/Features/SASL suggests that the support may be
> > there, but I can't tell for sure.
> > 
> 
> Hmm, looking at the libvirt code, it doesn't appear to have ever been wired
> up for spice, though it is supported for VNC. elmarco, what ever happened to
> libvirt + spice + sasl support?

At them time I added support for spice, I don't think I ever touched libvirt yet;) It's so quite long ago, and wasn't driven by any particular need but me wanted to learn. So I would say it's never has been really officilly supported, perhaps it just works for some people, but I never heard anyone asking about it ever :)

Comment 10 Anthony Messina 2013-09-03 18:44:23 UTC
(In reply to Cole Robinson from comment #8)
> > > The other option is that maybe libvirt could add an option in
> > > /etc/libvirt/qemu.conf that makes 'secure' the channel default for new VMs,
> > > similar to how you can set a default password for all new VMs. In fact I'd
> > > argue libvirt should have done that automatically if spice_tls is enabled,
> > > but it's probably too late to change that now. http://libvirt.org/bugs.html
> > 
> > I agree that it would be nice to have this in libvirt.  But what do you mean
> > "too late to change that now"?
> 
> I meant, too late to change the behavior of the existing spice_tls option to
> imply defaultMode=secure for all new VMs. If I was implementing that feature
> from scratch I'd make it do that, but we can't change that behavior now. We
> can however add a new qemu.conf option 'spice_channel_default_mode' which
> maps to defaultMode if none is explicitly specified in the XML. So in
> addition to setting spice_tls=1, you also set
> spice_channel_default_mode=secure, and get the behavior you want. But
> someone needs to file a bug for that (hint hint :) )

Got the hint ;) https://bugzilla.redhat.com/show_bug.cgi?id=1004037

Comment 11 Anthony Messina 2013-09-03 19:17:00 UTC
(In reply to Marc-Andre Lureau from comment #9)
> (In reply to Cole Robinson from comment #8)
> > > Personally, I never would have even started this complicated TLS stuff if
> > > the libvirt builds enabled SASL for Spice. 
> > > http://spice-space.org/page/Features/SASL suggests that the support may be
> > > there, but I can't tell for sure.
> > > 
> > 
> > Hmm, looking at the libvirt code, it doesn't appear to have ever been wired
> > up for spice, though it is supported for VNC. elmarco, what ever happened to
> > libvirt + spice + sasl support?
> 
> At them time I added support for spice, I don't think I ever touched libvirt
> yet;) It's so quite long ago, and wasn't driven by any particular need but
> me wanted to learn. So I would say it's never has been really officilly
> supported, perhaps it just works for some people, but I never heard anyone
> asking about it ever :)

For me, the Kerberos/SASL issue began with Bug #847417, where I was using VNC (no TLS at all) and a completely Kerberized environment with libvirt, qemu, and virt-manager.  Since that seemed to be unresolvable, and wanting to ensure some sort of security, I investigated and implemented Spice with TLS, only then to file this bug (Bug #904295).

In between, I implemented FreeIPA to help with the certificate issues, since I never managed any of my own and got the upgraded Kerberos infrastructure as well.

I'd love to see Kerberos/SASL/GSSAPI support for Spice.  While Kerberos has a reputation for being difficult and mysterious to set up, configuring a TLS infrastructure with the proper certs and keys spread all over the place has been the one time I'd say Kerberos was truly *easy*, in comarison.

SASL support was my original goal as it provided authentication, authorization, and encryption support (at least with VNC).

Right now, I'm in this place in the middle, where Spice isn't supporting SASL encryption, and libvirt isn't enforcing TLS *or* SASL.

I am at least am able to use SASL authentication with auth_tls = "sasl" in libvirt.

Comment 12 Marc-Andre Lureau 2013-09-03 19:29:37 UTC
(In reply to Anthony Messina from comment #11)
> SASL support was my original goal as it provided authentication,
> authorization, and encryption support (at least with VNC).
> 
> Right now, I'm in this place in the middle, where Spice isn't supporting
> SASL encryption, and libvirt isn't enforcing TLS *or* SASL.

Why do you say "Spice isn't supporting SASL encryption"?

Isn't all SPICE traffic channeld through SASL and encryption supposed to be done by some mechanism backend already?

> 
> I am at least am able to use SASL authentication with auth_tls = "sasl" in
> libvirt.

Is that enough to also enable spice server sasl support?

Comment 13 Marc-Andre Lureau 2013-09-03 19:31:42 UTC
oh, and the wiki page and implementation suggests that we can layer sasl on top of spice tls (you could encrypt two times!).

Comment 14 Anthony Messina 2013-09-03 19:55:43 UTC
(In reply to Marc-Andre Lureau from comment #12)
> (In reply to Anthony Messina from comment #11)
> > SASL support was my original goal as it provided authentication,
> > authorization, and encryption support (at least with VNC).
> > 
> > Right now, I'm in this place in the middle, where Spice isn't supporting
> > SASL encryption, and libvirt isn't enforcing TLS *or* SASL.
> 
> Why do you say "Spice isn't supporting SASL encryption"?

Because my qemu-kvm instance is showing

-spice port=5900,tls-port=5901,addr=0.0.0.0,x509-dir=/etc/pki/libvirt-spice,seamless-migration=on

When according to http://spice-space.org/page/Features/SASL, it needs the 'sasl' flag:

"Add the ',sasl' flag when launching QEMU with a Spice server."


> Isn't all SPICE traffic channeld through SASL and encryption supposed to be
> done by some mechanism backend already?
> 
> > 
> > I am at least am able to use SASL authentication with auth_tls = "sasl" in
> > libvirt.
> 
> Is that enough to also enable spice server sasl support?

As far as I can tell, from using virt-manager & TLS, the connection from virt-manager to the hypervisor is made on port 16514.  But when opening a virtual machine's console via virt-manager, it's connecting on port 5900, rather than 5901 (that's what cdc82e62a236d0e737851c693d6673f65ca278c1 is supposed to help fix).  Even so, Spice connections are not currently authenticated via SASL, but via the 'passwd' entry (only the connection to libvirt is SASL-ized).

<graphics type='spice' autoport='yes' passwd='superdupersecret'/>

Comment 15 Daniel Berrangé 2013-09-04 09:19:48 UTC
(In reply to Marc-Andre Lureau from comment #12)
> (In reply to Anthony Messina from comment #11)
> > SASL support was my original goal as it provided authentication,
> > authorization, and encryption support (at least with VNC).
> > 
> > Right now, I'm in this place in the middle, where Spice isn't supporting
> > SASL encryption, and libvirt isn't enforcing TLS *or* SASL.
> 
> Why do you say "Spice isn't supporting SASL encryption"?
> 
> Isn't all SPICE traffic channeld through SASL and encryption supposed to be
> done by some mechanism backend already?

SASL has support for doing data encryption, and spice-gtk looks like it is using this correctly - the code is basically cribbed from the VNC SASL code by the looks of it. It of course depends on the mechanism used - commonly only the gssapi/kerberos and digest-md5 mechanisms in SASL do encryption. 

> > I am at least am able to use SASL authentication with auth_tls = "sasl" in
> > libvirt.
> 
> Is that enough to also enable spice server sasl support?

No, that config option only controls SASL for libvirtd.

There is a separate vnc_sasl  option for controling VNC. We are lacking support for a spice_sasl option in libvirt to enable this per VM I believe.

(In reply to Marc-Andre Lureau from comment #13)
> oh, and the wiki page and implementation suggests that we can layer sasl on
> top of spice tls (you could encrypt two times!).

Actually double-encryption shouldn't be happening.  When TLS is used, spice-gtk is turning off SASL's encryption layer to avoid double-encryption.

Comment 16 Anthony Messina 2013-09-04 09:25:11 UTC
(In reply to Daniel Berrange from comment #15)

Thank you for the clarification and confirmation, Daniel.

Comment 17 Fedora Update System 2013-09-24 15:48:45 UTC
virt-manager-0.10.0-2.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/virt-manager-0.10.0-2.fc19

Comment 18 Fedora Update System 2013-09-26 06:23:42 UTC
Package virt-manager-0.10.0-2.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing virt-manager-0.10.0-2.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-17656/virt-manager-0.10.0-2.fc19
then log in and leave karma (feedback).

Comment 19 Anthony Messina 2013-09-28 03:54:59 UTC
Using virt-manager-0.10.0-3.fc19.noarch, this seems to be mostly resolved.  I can update the following per-channel configuration via virsh edit and confirm that I am connecting via tls-port 5901 (rather than the regular port 5900).

<graphics type='spice' autoport='yes' passwd='password'>
  <channel name='main' mode='secure'/>
  <channel name='inputs' mode='secure'/>
  <channel name='cursor' mode='secure'/>
  <channel name='playback' mode='secure'/>
  <channel name='record' mode='secure'/>
  <channel name='smartcard' mode='secure'/>
  <channel name='usbredir' mode='secure'/>
</graphics>

But I am not able to use the following (which is much easier), or the virt-manager will only say that the VM's graphical console is not yet running.

<graphics type='spice' autoport='yes' passwd='password' defaultMode='secure'/>

Even with the "mostly working" config, I still show one connection to the unsecured port 5900:

qemu]# netstat -anlpW|grep 590[01].*ESTABLISHED
tcp        0      0 10.1.2.5:60317        10.1.2.5:5901         ESTABLISHED 13006/python        
tcp        0      0 10.1.2.5:5900         10.1.2.5:47266        ESTABLISHED 14074/qemu-system-x 
tcp        0      0 10.1.2.5:60321        10.1.2.5:5901         ESTABLISHED 13006/python        
tcp        0      0 10.1.2.5:47266        10.1.2.5:5900         ESTABLISHED 13006/python        
tcp        0      0 10.1.2.5:5901         10.1.2.5:60321        ESTABLISHED 14074/qemu-system-x 
tcp        0      0 10.1.2.5:60322        10.1.2.5:5901         ESTABLISHED 13006/python        
tcp        0      0 10.1.2.5:5901         10.1.2.5:60317        ESTABLISHED 14074/qemu-system-x 
tcp        0      0 10.1.2.5:5901         10.1.2.5:60322        ESTABLISHED 14074/qemu-system-x

Comment 20 Cole Robinson 2013-10-03 19:57:21 UTC
Hmm, defaultMode='secure' is working for me on F20 with virt-manager git. Can you try virt-manager git on F19?

git clone git://fedorahosted.org/virt-manager.git
cd virt-manager
./virt-manager --debug

And post the output here if it still isn't working.

Comment 21 Anthony Messina 2013-10-05 16:10:39 UTC
(In reply to Cole Robinson from comment #20)
> Hmm, defaultMode='secure' is working for me on F20 with virt-manager git.
> Can you try virt-manager git on F19?
> 
> git clone git://fedorahosted.org/virt-manager.git
> cd virt-manager
> ./virt-manager --debug
> 
> And post the output here if it still isn't working.

I can connect, but it connects to the non-TLS ports:
~]# netstat -anlpW|grep 590[01].*ESTABLISHED
tcp        0      0 10.1.2.5:45536        10.1.2.102:5900       ESTABLISHED 22110/python        
tcp        0      0 10.1.2.5:45533        10.1.2.102:5900       ESTABLISHED 22110/python        
tcp        0      0 10.1.2.5:45535        10.1.2.102:5900       ESTABLISHED 22110/python        
tcp        0      0 10.1.2.5:45534        10.1.2.102:5900       ESTABLISHED 22110/python

Here's the debug output as requested.

[Sat, 05 Oct 2013 11:04:36 virt-manager 22110] DEBUG (connection:169) Using libvirt API for netdev enumeration
[Sat, 05 Oct 2013 11:04:36 virt-manager 22110] DEBUG (connection:191) Using libvirt API for mediadev enumeration
[Sat, 05 Oct 2013 11:04:39 virt-manager 22110] DEBUG (details:583) Showing VM details: <vmmDomain object at 0x2edaeb0 (virtManager+domain+vmmDomain at 0x7f2c581de8c0)>
[Sat, 05 Oct 2013 11:04:39 virt-manager 22110] DEBUG (engine:340) window counter incremented to 2
[Sat, 05 Oct 2013 11:04:39 virt-manager 22110] DEBUG (console:1333) Starting connect process for proto=spice trans=tls connhost=linux-ws2.example.com connuser=amessina connport=None gaddr=0.0.0.0 gport=None gtlsport=5900 gsocket=None
[Sat, 05 Oct 2013 11:04:39 virt-manager 22110] DEBUG (keyring:61) Using keyring session /org/freedesktop/secrets/session/s4
[Sat, 05 Oct 2013 11:04:43 virt-manager 22110] DEBUG (console:1240) Viewer connected

(virt-manager:22110): GSpice-WARNING **: Warning no automount-inhibiting implementation available
[Sat, 05 Oct 2013 11:07:48 virt-manager 22110] DEBUG (details:600) Closing VM details: <vmmDomain object at 0x2edaeb0 (virtManager+domain+vmmDomain at 0x7f2c581de8c0)>
[Sat, 05 Oct 2013 11:07:48 virt-manager 22110] DEBUG (engine:344) window counter decremented to 1
[Sat, 05 Oct 2013 11:07:49 virt-manager 22110] DEBUG (console:1213) Viewer disconnected
[Sat, 05 Oct 2013 11:08:04 virt-manager 22110] DEBUG (details:600) Closing VM details: <vmmDomain object at 0x2edaeb0 (virtManager+domain+vmmDomain at 0x7f2c581de8c0)>
[Sat, 05 Oct 2013 11:08:06 virt-manager 22110] DEBUG (manager:219) Closing manager
[Sat, 05 Oct 2013 11:08:06 virt-manager 22110] DEBUG (engine:344) window counter decremented to 0
[Sat, 05 Oct 2013 11:08:06 virt-manager 22110] DEBUG (engine:426) Exiting app normally.

Comment 22 Cole Robinson 2013-10-05 16:22:38 UTC
(In reply to Anthony Messina from comment #21)
> (In reply to Cole Robinson from comment #20)
> > Hmm, defaultMode='secure' is working for me on F20 with virt-manager git.
> > Can you try virt-manager git on F19?
> > 
> > git clone git://fedorahosted.org/virt-manager.git
> > cd virt-manager
> > ./virt-manager --debug
> > 
> > And post the output here if it still isn't working.
> 
> I can connect, but it connects to the non-TLS ports:
> ~]# netstat -anlpW|grep 590[01].*ESTABLISHED
> tcp        0      0 10.1.2.5:45536        10.1.2.102:5900       ESTABLISHED
> 22110/python        
> tcp        0      0 10.1.2.5:45533        10.1.2.102:5900       ESTABLISHED
> 22110/python        
> tcp        0      0 10.1.2.5:45535        10.1.2.102:5900       ESTABLISHED
> 22110/python        
> tcp        0      0 10.1.2.5:45534        10.1.2.102:5900       ESTABLISHED
> 22110/python
> 
> Here's the debug output as requested.
> 
> [Sat, 05 Oct 2013 11:04:36 virt-manager 22110] DEBUG (connection:169) Using
> libvirt API for netdev enumeration
> [Sat, 05 Oct 2013 11:04:36 virt-manager 22110] DEBUG (connection:191) Using
> libvirt API for mediadev enumeration
> [Sat, 05 Oct 2013 11:04:39 virt-manager 22110] DEBUG (details:583) Showing
> VM details: <vmmDomain object at 0x2edaeb0 (virtManager+domain+vmmDomain at
> 0x7f2c581de8c0)>
> [Sat, 05 Oct 2013 11:04:39 virt-manager 22110] DEBUG (engine:340) window
> counter incremented to 2
> [Sat, 05 Oct 2013 11:04:39 virt-manager 22110] DEBUG (console:1333) Starting
> connect process for proto=spice trans=tls connhost=linux-ws2.example.com
> connuser=amessina connport=None gaddr=0.0.0.0 gport=None gtlsport=5900
> gsocket=None

This bit here is saying that libvirt allocated port 5900 as the tlsport. And all those connections from netstat are showing port 5900. So if I'm understanding things correctly, it's all working here.

You can use remote-viewer as a separate client to try and verify, something like

  remote-viewer spice://$HOST:5900

Should fail, but

  remote-viewer spice://$HOST?tls-port=5900

Should work (it does for me, make sure you use the same hostname as used in your ssl setup)

Comment 23 Anthony Messina 2013-10-05 16:55:00 UTC
(In reply to Cole Robinson from comment #20)
> Hmm, defaultMode='secure' is working for me on F20 with virt-manager git.
> Can you try virt-manager git on F19?
> 
> git clone git://fedorahosted.org/virt-manager.git
> cd virt-manager
> ./virt-manager --debug
> 
> And post the output here if it still isn't working.

Nevermind Comment #21.  I see that using your instructions in Comment #20, there is no 'gport=5900' but only a 'gport=None' and 'gtlsport=5900'.  This works in the latest version from Git, but not the latest release for F19 (virt-manager-0.10.0-3.fc19.noarch).

At this point, I'd like to verify the TLS traffic with tcpdump.  What parameters do you use to verify TLS traffic with tcpdump.

Comment 24 Cole Robinson 2013-10-05 16:56:42 UTC
(In reply to Anthony Messina from comment #23)
> (In reply to Cole Robinson from comment #20)
> > Hmm, defaultMode='secure' is working for me on F20 with virt-manager git.
> > Can you try virt-manager git on F19?
> > 
> > git clone git://fedorahosted.org/virt-manager.git
> > cd virt-manager
> > ./virt-manager --debug
> > 
> > And post the output here if it still isn't working.
> 
> Nevermind Comment #21.  I see that using your instructions in Comment #20,
> there is no 'gport=5900' but only a 'gport=None' and 'gtlsport=5900'.  This
> works in the latest version from Git, but not the latest release for F19
> (virt-manager-0.10.0-3.fc19.noarch).
> 

When I get some more time I'll try and figure out what's wrong with the f19 version

> At this point, I'd like to verify the TLS traffic with tcpdump.  What
> parameters do you use to verify TLS traffic with tcpdump.

No idea, sorry

Comment 25 Anthony Messina 2013-10-05 17:12:05 UTC
(In reply to Cole Robinson from comment #22)
> This bit here is saying that libvirt allocated port 5900 as the tlsport. And
> all those connections from netstat are showing port 5900. So if I'm
> understanding things correctly, it's all working here.
> 
> You can use remote-viewer as a separate client to try and verify, something
> like
> 
>   remote-viewer spice://$HOST:5900
> 
> Should fail, but
> 
>   remote-viewer spice://$HOST?tls-port=5900
> 
> Should work (it does for me, make sure you use the same hostname as used in
> your ssl setup)

I can confirm that this works, and now I just need to figure out how to verify that TLS is in fact encrypting the connections.

Comment 26 Anthony Messina 2013-10-05 17:24:34 UTC
(In reply to Cole Robinson from comment #24)
> (In reply to Anthony Messina from comment #23)
> > (In reply to Cole Robinson from comment #20)
> > > Hmm, defaultMode='secure' is working for me on F20 with virt-manager git.
> > > Can you try virt-manager git on F19?
> > > 
> > > git clone git://fedorahosted.org/virt-manager.git
> > > cd virt-manager
> > > ./virt-manager --debug
> > > 
> > > And post the output here if it still isn't working.
> > 
> > Nevermind Comment #21.  I see that using your instructions in Comment #20,
> > there is no 'gport=5900' but only a 'gport=None' and 'gtlsport=5900'.  This
> > works in the latest version from Git, but not the latest release for F19
> > (virt-manager-0.10.0-3.fc19.noarch).
> > 
> 
> When I get some more time I'll try and figure out what's wrong with the f19
> version

Ok, I can confirm TLS verification by mucking around with my spice_truststore.pem.  Also, I can confirm that the Git master (v0.10.0-502-g8e460dc) works whereas virt-manager-0.10.0-3.fc19 just sits there saying "graphical console is not yet active for guest".  remote-viewer from virt-viewer-0.5.6-1.fc19.x86_64 works properly with TLS.

Comment 27 Fedora Update System 2013-10-06 19:40:09 UTC
virt-manager-0.10.0-4.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/virt-manager-0.10.0-4.fc19

Comment 28 Anthony Messina 2013-10-07 10:25:03 UTC
I can confirm that this issue is resolved with https://admin.fedoraproject.org/updates/virt-manager-0.10.0-4.fc19:  I can use

<graphics type='spice' autoport='yes' passwd='password' defaultMode='secure'/>

and qemu starts with

-spice tls-port=5902,addr=0.0.0.0,x509-dir=/etc/pki/libvirt-spice,tls-channel=default,seamless-migration=on

and virt-manager's console connects properly.

Thank you very much for resolving this issue.

Comment 29 Fedora Update System 2013-10-08 11:35:36 UTC
Package virt-manager-0.10.0-4.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing virt-manager-0.10.0-4.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-18499/virt-manager-0.10.0-4.fc19
then log in and leave karma (feedback).

Comment 30 Fedora Update System 2013-10-10 00:56:22 UTC
virt-manager-0.10.0-4.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.