Bug 738439

Summary: missing functions on remote libvirt give wrong error
Product: Red Hat Enterprise Linux 6 Reporter: Eric Blake <eblake>
Component: libvirtAssignee: Eric Blake <eblake>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.2CC: acathrow, dallan, dyuan, mzhan, rwu, weizhan, whuang, yupzhang
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-0.9.4-12.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 781895 (view as bug list) Environment:
Last Closed: 2011-12-06 11:29:26 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 743047, 781895    

Description Eric Blake 2011-09-14 19:08:07 UTC
Description of problem:
Libvirt 0.9.3 introduced a regression where missing RPC calls on the server get presented to the client as VIR_ERR_RPC instead of VIR_ERR_NO_SUPPORT.  This in turn can lead to some odd errors from virsh and other clients trying to use a new API not present on the server, with the change in behavior being viewed as a regression compared to a server running 0.9.2 or earlier.

Version-Release number of selected component (if applicable):
libvirt-0.9.4-11.el6

How reproducible:
100%

Steps to Reproduce:
1. On the server, run a version of libvirt that lacks an API command that will be tickled by the client.  For example, libvirt 0.9.4 lacks virDomainMigrateGetMaxSpeed, while libvirt 0.9.5 has 'virsh migrate-getspeed'.  Other combinations include libvirt 0.9.3, which lacks virDomainSaveImageDefineXML, while libvirt 0.9.4 has 'virsh save-image-edit'.  [Note that these two possibilities imply that one of your two machines can run RHEL 6.2, but the other machine must run something older or newer, rather than the RHEL release; and it is your choice whether to have the server or the client running the orthogonal version]
2. On the client, run the appropriate counterpart version of libvirt, and issue the virsh command that triggers the unknown API [see step 1 for two possibilities, depending on which libvirt version you are using]
3.
  
Actual results:
I tested with server 0.9.4 and client from 0.9.5-rc1, where 'virsh domblkstat dom vda' fails with 'error: unknown procedure: 243'.

Expected results:
For my test case, virsh should see VIR_ERR_NO_SUPPORT and gracefully fall back to virDomainBlockStats instead of the newer virDomainBlockStatsFlags.

Additional info:
Upstream patch proposed:
https://www.redhat.com/archives/libvir-list/2011-September/msg00547.html

Comment 2 Eric Blake 2011-09-16 14:57:16 UTC
In POST:
https://www.redhat.com/archives/libvir-list/2011-September/msg00547.html

There is no need to adjust the server side - the bug is only present when talking older server to newer client.  Since an older client talking to a newer server should never be sending unknown RPC in the first place, it will never trigger the error message, so the newer server does not have to bend over backwards to preserve the older-style errors.  That is, 0.9.4 is the only release where this problem hits, and only when talking to an 0.9.3 server with one of the APIs added in 0.9.4.  Therefore, the fix is only on the client side.

Since this is a client side fix, the only way to test that RHEL has the fix is to use a server running 0.9.3 and a fixed RHEL 0.9.4 client that triggers one of the APIs present only in 0.9.4.

Comment 5 Huang Wenlong 2011-09-21 09:17:08 UTC
Test this bug :

Server: 
libvirt-0.9.3-2.el6.x86_64 

Client: 
libvirt-0.9.4-12.el6.x86_64


In client : 
virsh -c qemu+ssh://10.66.85.207/system 

virsh # domblkstat t hda

domblkstat t hda
hda rd_req 42882
hda rd_bytes 130320896
hda wr_req 196
hda wr_bytes 1790976
hda errs 0


virsh # save-image-edit help
error: unknown procedure: 235

Comment 6 Huang Wenlong 2011-09-21 09:21:43 UTC
hi , Eric Blake 

I try to verify this bug with  Comment 5  , but I am not sure whether my way is orrect , so  please help me confirm it ,thanks . 
If my way is wrong ,please tell me the correct verify method  thanks very much .

Wenlong

Comment 7 Eric Blake 2011-09-21 16:34:27 UTC
In virsh, save-image-edit doesn't make any decisions based on the error given for a failed rpc function, and it has a custom error handler that does not display full error information to the user.  So I think we have to write a simple program (C or python) rather than relying on virsh, to properly verify this.  This program should do the trick when compiled against 0.9.4 (or newer), and targetting a server of 0.9.3:

$ cat foo.c
#include <stdio.h>
#include <libvirt/libvirt.h>
#include <libvirt/virterror.h>
int main(int argc, char **argv) {
    if (argc < 3) {
        fprintf(stderr, "usage: %s uri file\n", argv[0]);
        return 1;
    }
    virConnectPtr conn = virConnectOpenAuth(argv[1], virConnectAuthPtrDefault,
                                            0);
    if (!conn) {
        fprintf(stderr, "unable to open connection to %s\n", argv[1]);
        return 1;
    }
    char *xml = virDomainSaveImageGetXMLDesc(conn, argv[2], 0);
    if (xml) {
        fprintf(stderr, "server is too new\n");
        return 1;
    }
    virErrorPtr err = virGetLastError();
    if (!err) {
        fprintf(stderr, "missing error\n");
        return 1;
    }
    if (err->code != VIR_ERR_NO_SUPPORT) {
        fprintf(stderr, "wrong error code %d\n", err->code);
        return 1;
    }
    puts("success");
    virConnectClose(conn);
    return 0;
}
$ gcc -o foo foo.c -lvirt
$ ./foo qemu+ssh://remote/system /path/to/save/file
libvirt: RPC error : unknown procedure: 235
success

Note the expected error message mentions "RPC error", rather than "Remote error"

Comment 8 Huang Wenlong 2011-09-22 03:24:30 UTC
Verify  this bug in libvirt-0.9.4-12.el6.x86_64 :

Server: 
libvirt-0.9.3-2.el6.x86_64 

Client: 
libvirt-0.9.4-12.el6.x86_64



# cat foo.c
#include <stdio.h>
#include <libvirt/libvirt.h>
#include <libvirt/virterror.h>
int main(int argc, char **argv) {
    if (argc < 3) {
        fprintf(stderr, "usage: %s uri file\n", argv[0]);
        return 1;
    }
    virConnectPtr conn = virConnectOpenAuth(argv[1], virConnectAuthPtrDefault,
                                            0);
    if (!conn) {
        fprintf(stderr, "unable to open connection to %s\n", argv[1]);
        return 1;
    }
    char *xml = virDomainSaveImageGetXMLDesc(conn, argv[2], 0);
    if (xml) {
        fprintf(stderr, "server is too new\n");
        return 1;
    }
    virErrorPtr err = virGetLastError();
    if (!err) {
        fprintf(stderr, "missing error\n");
        return 1;
    }
    if (err->code != VIR_ERR_NO_SUPPORT) {
        fprintf(stderr, "wrong error code %d\n", err->code);
        return 1;
    }
    puts("success");
    virConnectClose(conn);
    return 0;
}

# ./foo qemu+ssh://10.66.85.207/system /tmp/foo.log
The authenticity of host '10.66.85.207 (10.66.85.207)' can't be established.
RSA key fingerprint is c9:4d:b4:d9:cf:bd:ca:56:72:4b:42:7d:86:38:7b:fd.
Are you sure you want to continue connecting (yes/no)? yes
root.85.207's password: 
libvir: RPC error : unknown procedure: 235
success

Comment 9 errata-xmlrpc 2011-12-06 11:29:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html