RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 726174 - Impossible libvirt remote administration via qemu+ssh
Summary: Impossible libvirt remote administration via qemu+ssh
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Eric Blake
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-07-27 18:15 UTC by Eric Blake
Modified: 2012-06-20 06:29 UTC (History)
11 users (show)

Fixed In Version: libvirt-0.9.9-1.el6
Doc Type: Bug Fix
Doc Text:
Differences between the Fedora and Debian implementations of the 'nc' command, such as the presence or absence of a '-q' option, meant that attempts to use a remote connection from a client expecting one behavior to a server providing another behavior would hang or cause other failures on a reconnection attempt. Libvirt has now been patched to probe the capabilities of nc, and use the appropriate options of the server even if they differ from the nc on the client, in order to allow interaction between either type of machine without issue.
Clone Of: 562176
Environment:
Last Closed: 2012-06-20 06:29:35 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2012:0748 0 normal SHIPPED_LIVE Low: libvirt security, bug fix, and enhancement update 2012-06-19 19:31:38 UTC

Comment 1 Eric Blake 2011-07-27 18:17:57 UTC
The patch https://bugzilla.redhat.com/attachment.cgi?id=399858 from bug 562176 would need modernizing, but serves as an example of how to change from a hard-coded nc execution into a shell script that auto-detects whether nc -q is supported (debian-based remote, in which case -q must be used) or missing (redhat-based remote, in which case -q is not needed).

Comment 3 Dave Allan 2011-07-27 23:58:04 UTC
Eric, why does this depend on bz 562176?  It looks to me like this is a workaround in case libvirt is talking to a system without that fix, but maybe I read it too quickly, or is the depends just an artifact of the clone?

Comment 4 Eric Blake 2011-07-28 02:06:43 UTC
(In reply to comment #3)
> is the depends just an artifact of the clone?

The latter was correct; I've dropped the dependency.

Comment 5 Eric Blake 2011-09-21 18:13:03 UTC
There was an upstream attempt here, but it also needs work before it is worth including:
https://www.redhat.com/archives/libvir-list/2011-July/msg02041.html
https://www.redhat.com/archives/libvir-list/2011-August/msg00050.html

Comment 7 Eric Blake 2011-10-13 23:11:39 UTC
This is now fixed upstream:

commit 6ac6238de33fc74e7545b245ae273d1bfd658808
Author: Guido Günther <agx>
Date:   Thu Oct 13 21:49:01 2011 +0200

    Use virBufferEscapeShell in virNetSocketNewConnectSSH
    
    to escape the netcat command since it's passed to the shell. Adjust
    expected test case output accordingly.

commit 920487b36d2bf38ebad89207e64f57a7ffa2afb1
Author: Guido Günther <agx>
Date:   Thu Jul 28 15:25:00 2011 +0200

    Add virBufferEscapeShell
    
    Escape strings so they're safe to pass to the shell. It's based on
    virsh's cmdEcho.

commit a2b5c57db83559d4fe32ee90fbb6685555388e06
Author: Guido Günther <agx>
Date:   Fri Jul 8 21:07:29 2011 +0200

    Autodetect if the remote nc command supports the -q option
    
    Based on a patch by Marc Deslauriers <marc.deslauriers>
    
    RH: https://bugzilla.redhat.com/show_bug.cgi?id=562176
    Ubuntu: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/517478
    Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=573172

Comment 10 Huang Wenlong 2011-11-24 03:10:46 UTC
Hi, Eric Black 

I can not reporduce this bug .

I setup a debian-6.0.2.1(libvirt-0.8.3-5+squeeze2)   and ubuntu-10.04 (libvirt-0.8.8-1ubuntu6.5)  and rhel6.2(libvirt-0.9.4-23.el6_2.1.x86_64)

But I can connect rhel server via "sudo virsh -c qemu+ssh://$server_ip/system" with client debian and ubuntu.
Is there anything wrong with my environment ?

Wenlong

Comment 11 Eric Blake 2011-11-24 12:56:03 UTC
(In reply to comment #10)
> I setup a debian-6.0.2.1(libvirt-0.8.3-5+squeeze2)   and ubuntu-10.04
> (libvirt-0.8.8-1ubuntu6.5)  and rhel6.2(libvirt-0.9.4-23.el6_2.1.x86_64)
> 
> But I can connect rhel server via "sudo virsh -c qemu+ssh://$server_ip/system"
> with client debian and ubuntu.
> Is there anything wrong with my environment ?

Debian client and Red Hat server is not the problem (the old libvirt was hard-coded to Red Hat nc semantics, and nc is used on the server).  Rather, the problem is evident with Red Hat client and Debian server (where the Debian nc requires the use of -q).  The fix also has to be careful that even though -q must be used on Debian, it does not exist on Red Hat nc, so fixing things to talk to a Debian server must not break talking to a Red Hat server.

Comment 12 Huang Wenlong 2011-11-25 03:11:58 UTC
(In reply to comment #11)
> (In reply to comment #10)
> > I setup a debian-6.0.2.1(libvirt-0.8.3-5+squeeze2)   and ubuntu-10.04
> > (libvirt-0.8.8-1ubuntu6.5)  and rhel6.2(libvirt-0.9.4-23.el6_2.1.x86_64)
> > 
> > But I can connect rhel server via "sudo virsh -c qemu+ssh://$server_ip/system"
> > with client debian and ubuntu.
> > Is there anything wrong with my environment ?
> 
> Debian client and Red Hat server is not the problem (the old libvirt was
> hard-coded to Red Hat nc semantics, and nc is used on the server).  Rather, the
> problem is evident with Red Hat client and Debian server (where the Debian nc
> requires the use of -q).  The fix also has to be careful that even though -q
> must be used on Debian, it does not exist on Red Hat nc, so fixing things to
> talk to a Debian server must not break talking to a Red Hat server.

Hi,Eric

Thanks for your information, but I  also can connect debian Server with RHEL Client ,is that right? 

 Debian-6.0.2.1(libvirt-0.8.3-5+squeeze2  netcat-openbsd-1.89-4)  
 Rhel-6.2  (libvirt-0.9.4-23.el6_2.1.x86_64 nc-1.84-22.el6.x86_64)  

Wenlong

Comment 13 Eric Blake 2011-11-28 15:12:16 UTC
(In reply to comment #12)
> > Debian client and Red Hat server is not the problem (the old libvirt was
> > hard-coded to Red Hat nc semantics, and nc is used on the server).  Rather, the
> > problem is evident with Red Hat client and Debian server (where the Debian nc
> > requires the use of -q).  The fix also has to be careful that even though -q
> > must be used on Debian, it does not exist on Red Hat nc, so fixing things to
> > talk to a Debian server must not break talking to a Red Hat server.
> 
> Hi,Eric
> 
> Thanks for your information, but I  also can connect debian Server with RHEL
> Client ,is that right? 
> 
>  Debian-6.0.2.1(libvirt-0.8.3-5+squeeze2  netcat-openbsd-1.89-4)  
>  Rhel-6.2  (libvirt-0.9.4-23.el6_2.1.x86_64 nc-1.84-22.el6.x86_64) 

I haven't personally reproduced the problem (since I don't have a debian server handy), rather, I just reviewed the patches that were applied upstream by others who had experienced the problem.  My understanding of the bug report is that you are using correct software versions to expose the issue, but that you won't see the problem until you attempt back-to-back connections.  That is, Debian 'nc' refuses to let go of a port when the client disconnects, unless you use 'nc -q'; so the first connection will succeed, but it is subsequent connections where things hang because the port is still tied up by the prior connection.  Red Had 'nc' lacks -q, but lets go of the port right away, and therefore behaves the same as Debian 'nc -q'.

Comment 14 Huang Wenlong 2011-11-29 07:37:48 UTC
Hi, Eric 

When I add LIBVIRT_DEBUG=1 virsh -c in the Debian client I got some useful infomation :
...
: doRemoteOpen:565 : proceeding with name = qemu:///system
03:27:58.904: debug : virExecWithHook:712 : ssh 10.66.5.12 sh -c 'nc -q 2>&1 | grep -q "requires an argument";if [ $? -eq 0 ] ; then   CMD="nc -q 0 -U /var/run/libvirt/libvirt-sock";else   CMD="nc -U /var/run/libvirt/libvirt-sock";fi;eval "$CMD";'
...

the client will check  server's nc whether has -q then add or del -q option in the CMD . 
And the libvirt-0.9.4-23.el6_2.1.x86_64 also has this check. 
So it is fixed ,right ? 

Wenlong

Comment 18 Huang Wenlong 2012-01-10 02:50:53 UTC
Verify this bug with :
libvirt-0.9.9-1.el6.x86_64



# LIBVIRT_DEBUG=1 virsh -c qemu+ssh://$Debian/system
...
 About to run LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin HOME=/root USER=root LOGNAME=root SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass DISPLAY=localhost:10.0 ssh 10.66.7.102 sh -c 'if 'nc' -q 2>&1 | grep "requires an argument" >/dev/null 2>&1; then ARG=-q0;else ARG=;fi;'nc' $ARG -U /var/run/libvirt/libvirt-sock'

...


 virsh -c qemu+ssh://$Debian/system list
root.7.102's password: 
 Id Name                 State
----------------------------------

Comment 19 Eric Blake 2012-05-03 19:20:29 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Differences between the Fedora and Debian implementations of the 'nc' command, such as the presence or absence of a '-q' option, meant that attempts to use a remote connection from a client expecting one behavior to a server providing another behavior would hang or cause other failures on a reconnection attempt.  Libvirt has now been patched to probe the capabilities of nc, and use the appropriate options of the server even if they differ from the nc on the client, in order to allow interaction between either type of machine without issue.

Comment 21 errata-xmlrpc 2012-06-20 06:29:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0748.html


Note You need to log in before you can comment on or make changes to this bug.