RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1226697 - [virt-v2v] Allow json: filenames in qemu-img
Summary: [virt-v2v] Allow json: filenames in qemu-img
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm
Version: 7.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Hanna Czenczek
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-05-31 19:27 UTC by Richard W.M. Jones
Modified: 2015-11-19 05:03 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-1.5.3-92.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-19 05:03:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2213 0 normal SHIPPED_LIVE qemu-kvm bug fix and enhancement update 2015-11-19 08:16:10 UTC

Description Richard W.M. Jones 2015-05-31 19:27:54 UTC
Description of problem:

For background on this request, please see this virt-devel posting:
https://post-office.corp.redhat.com/mailman/private/virt-devel/2015-May/msg00338.html

Allow json:... to be used to specify filenames and
parameters in qemu-img.  For example:

    $ ./qemu-img create -f qcow2 /tmp/test.qcow2 \
        -b 'json: { "file.driver":"https",
                    "file.url":"https://foo/bar",
                    "file.timeout":1000 }'

As far as I can tell this was added upstream by the following
commit:

commit 4993f7ea7e63f18f18880289d6be8a9ab1591409
Author: Max Reitz <mreitz>
Date:   Thu May 8 20:12:41 2014 +0200

    block: Allow JSON filenames
    
    If the filename given to bdrv_open() is prefixed with "json:", parse the
    rest as a JSON object and merge the result into the options QDict. If
    there are conflicts, the options QDict takes precedence.
    
    Signed-off-by: Max Reitz <mreitz>
    Reviewed-by: Eric Blake <eblake>
    Signed-off-by: Kevin Wolf <kwolf>

and commit 5e5c4f63f4a7f48f571ea5671bf8452fe9655cdd
substantially refactored this code.  I don't know if other
commits are involved.

Version-Release number of selected component (if applicable):

qemu-kvm-1.5.3-89.el7

Comment 2 Hanna Czenczek 2015-06-01 15:05:42 UTC
Hi Richard,

Is there any specific use case this is needed for? (I don't see it mentioned in the mail you referenced.) Backporting it would probably not hurt too bad, but unless there is a specific need I would rather refrain from it.

Max

Comment 3 Richard W.M. Jones 2015-06-01 15:44:33 UTC
This seems pretty simple.  I had to backport only these upstream commits:

commit 9c5268127722087eec88c4c41f3363855bb1234b
Author: Max Reitz <mreitz>
Date:   Thu May 8 20:12:39 2014 +0200

    qdict: Add qdict_join()
    
    This function joins two QDicts by absorbing one into the other.
    
    Signed-off-by: Max Reitz <mreitz>
    Reviewed-by: Benoit Canet <benoit>
    Reviewed-by: Eric Blake <eblake>
    Signed-off-by: Kevin Wolf <kwolf>

commit 4993f7ea7e63f18f18880289d6be8a9ab1591409
Author: Max Reitz <mreitz>
Date:   Thu May 8 20:12:41 2014 +0200

    block: Allow JSON filenames
    
    If the filename given to bdrv_open() is prefixed with "json:", parse the
    rest as a JSON object and merge the result into the options QDict. If
    there are conflicts, the options QDict takes precedence.
    
    Signed-off-by: Max Reitz <mreitz>
    Reviewed-by: Eric Blake <eblake>
    Signed-off-by: Kevin Wolf <kwolf>

I didn't bother about 5e5c4f63f4a7f48f571ea5671bf8452fe9655cdd
since it's just a code refactoring and it references a function
(bdrv_fill_options) that does not exist in this version of qemu.

Comment 4 Richard W.M. Jones 2015-06-01 15:47:48 UTC
(In reply to Max Reitz from comment #2)
> Hi Richard,
> 
> Is there any specific use case this is needed for? (I don't see it mentioned
> in the mail you referenced.) Backporting it would probably not hurt too bad,
> but unless there is a specific need I would rather refrain from it.

Yes, virt-v2v uses json: filenames throughout to avoid the difficult
escaping problems with passing complex parameters to qemu-img.  A fairly
complex example involving up to 6 parameters is:
https://github.com/libguestfs/libguestfs/blob/master/v2v/input_libvirt_vcenter_https.ml#L249

Comment 5 Hanna Czenczek 2015-06-01 16:12:27 UTC
Very well then, thank you Richard. I'll backport it.

Max

Comment 6 Richard W.M. Jones 2015-06-01 18:07:04 UTC
Scratch build containing the backported patches (also
patches for bug 1226684):
http://brewweb.devel.redhat.com/brew/taskinfo?taskID=9288421

Comment 7 Richard W.M. Jones 2015-06-01 20:15:25 UTC
Improved scratch build, this time remembering to enable the
drivers in ./configure:
http://brewweb.devel.redhat.com/brew/taskinfo?taskID=9288785

Comment 8 Richard W.M. Jones 2015-06-02 08:41:10 UTC
Please see the successful results of my testing here:
https://bugzilla.redhat.com/show_bug.cgi?id=1226684#c6
https://bugzilla.redhat.com/show_bug.cgi?id=1226683#c3

Comment 10 Miroslav Rezanina 2015-06-16 18:01:35 UTC
Fix included in qemu-kvm-1.5.3-92.el7

Comment 12 Richard W.M. Jones 2015-06-17 16:19:28 UTC
qemu-kvm-1.5.3-92.el7.x86_64
I've tested this now and it all seems to be working.

Comment 13 mazhang 2015-07-23 07:17:50 UTC
Test this bug on qemu-kvm-1.5.3-97.el7.x86_64.

1. Create image with backing file specified by json which file.driver is https, then check the image's information.
# qemu-img create -f qcow2 /home/img -b 'json: {"file.driver":"https","file.url":"https://10.66.9.236/rhel7.2-64.qcow2","file.sslverify":"off", "file.readahead":"64k","file.timeout":2000}'
Formatting '/home/img', fmt=qcow2 size=21474836480 backing_file='json: {"file.driver":"https","file.url":"https://10.66.9.236/rhel7.2-64.qcow2","file.sslverify":"off", "file.readahead":"64k","file.timeout":2000}' encryption=off cluster_size=65536 lazy_refcounts=off

# qemu-img info /home/img 
image: /home/img
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 196K
cluster_size: 65536
backing file: json: {"file.driver":"https","file.url":"https://10.66.9.236/rhel7.2-64.qcow2","file.sslverify":"off", "file.readahead":"64k","file.timeout":2000}
Format specific information:
    compat: 1.1
    lazy refcounts: false

2. Create image with backing file specified by json which file.driver is ssh, then check the image's information.

# qemu-img create -f qcow2 overlay -b 'json: {"file.driver":"ssh","file.host":"10.66.11.16","file.path":"/home/rhel7.2-64.qcow2","file.host_key_check":"no" }'
Formatting 'overlay', fmt=qcow2 size=21474836480 backing_file='json: {"file.driver":"ssh","file.host":"10.66.11.16","file.path":"/home/rhel7.2-64.qcow2","file.host_key_check":"no" }' encryption=off cluster_size=65536 lazy_refcounts=off 

# qemu-img info overlay 
image: overlay
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 196K
cluster_size: 65536
backing file: json: {"file.driver":"ssh","file.host":"10.66.11.16","file.path":"/home/rhel7.2-64.qcow2","file.host_key_check":"no" }
Format specific information:
    compat: 1.1
    lazy refcounts: false


Just one question for file.driver is http/https.
When file.driver is http, qemu-img not complain url with https.
# qemu-img create -f qcow2 /home/img -b 'json: {"file.driver":"http","file.url":"https://10.66.9.236/rhel7.2-64.qcow2","file.sslverify":"off", "file.readahead":"64k","file.timeout":2000}'
Formatting '/home/img', fmt=qcow2 size=21474836480 backing_file='json: {"file.driver":"http","file.url":"https://10.66.9.236/rhel7.2-64.qcow2","file.sslverify":"off", "file.readahead":"64k","file.timeout":2000}' encryption=off cluster_size=65536 lazy_refcounts=off 

or file.driver is https, url is http.
# qemu-img create -f qcow2 /home/img -b 'json: {"file.driver":"https","file.url":"http://10.66.9.236/rhel7.2-64.qcow2","file.sslverify":"off", "file.readahead":"64k","file.timeout":2000}'
Formatting '/home/img', fmt=qcow2 size=21474836480 backing_file='json: {"file.driver":"https","file.url":"http://10.66.9.236/rhel7.2-64.qcow2","file.sslverify":"off", "file.readahead":"64k","file.timeout":2000}' encryption=off cluster_size=65536 lazy_refcounts=off 

Richard, Could you please have a look at above test, is this expected?

Thanks,
Mazhang.

Comment 14 Richard W.M. Jones 2015-07-23 07:50:46 UTC
That's (kind of) a "back door", since qemu uses file.driver
to select the curl driver, but then doesn't check that the URL
scheme from file.url matches the scheme from file.driver.
I wouldn't worry about it.

Comment 15 mazhang 2015-07-23 08:10:42 UTC
(In reply to Richard W.M. Jones from comment #14)
> That's (kind of) a "back door", since qemu uses file.driver
> to select the curl driver, but then doesn't check that the URL
> scheme from file.url matches the scheme from file.driver.
> I wouldn't worry about it.

Thanks, let's move this bug to verified, if any problem please let me know.

Comment 17 errata-xmlrpc 2015-11-19 05:03:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2213.html


Note You need to log in before you can comment on or make changes to this bug.