Bug 1092583

Summary: guestfish removes query string from URI
Product: [Community] Virtualization Tools Reporter: Matthew Booth <mbooth>
Component: libguestfsAssignee: Pino Toscano <ptoscano>
Status: NEW --- QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: ptoscano, rjones
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Matthew Booth 2014-04-29 13:59:47 UTC
Description of problem:
Executing the following:

LIBGUESTFS_DEBUG=1 ./run fish/guestfish -x --ro -a 'https://devel:password@vsphere.vmware.marston/folder/192.168.50.20_base%2ffdb77464-31f8-4b84-8cd2-4371473170e0/fdb77464-31f8-4b84-8cd2-4371473170e0.1-flat.vmdk?dcPath=Development&dsName=cluster1%252ddatastore1'

does:

libguestfs: trace: add_drive "/folder/192.168.50.20_base/fdb77464-31f8-4b84-8cd2-4371473170e0/fdb77464-31f8-4b84-8cd2-4371473170e0.1-flat.vmdk" "readonly:true" "protocol:https" "server:tcp:vsphere.vmware.marston" "username:devel:password"

(note the missing query string), and ultimately fails with:

libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -o backing_file=https://devel:password@vsphere.vmware.marston/folder/192.168.50.20_base/fdb77464-31f8-4b84-8cd2-4371473170e0/fdb77464-31f8-4b84-8cd2-4371473170e0.1-flat.vmdk
libguestfs: command: run: \ /home/mbooth/src/guestfs/libguestfs/tmp/libguestfssa3Xbi/overlay1
filename: https://devel:password@vsphere.vmware.marston/folder/192.168.50.20_base/fdb77464-31f8-4b84-8cd2-4371473170e0/fdb77464-31f8-4b84-8cd2-4371473170e0.1-flat.vmdk
CURL: Error opening file: The requested URL returned error: 404 Not Found

Version-Release number of selected component (if applicable):
8df792d4a76eb188fcbc68b38c8bc1e757a6f6ff (Version 1.27.7)

Comment 1 Richard W.M. Jones 2014-04-29 14:12:27 UTC
Note the API description:

http://libguestfs.org/guestfs.3.html#guestfs_add_drive_opts

Simple example of this:

$ guestfish -x --ro -a 'http://example.com/test?q=1&s=2' |& grep example
libguestfs: trace: add_drive "/test" "readonly:true" "protocol:http" "server:tcp:example.com"
libguestfs: trace: disk_create "/tmp/libguestfsHD6cEm/overlay1" "qcow2" -1 "backingfile:http://example.com/test"

Notice that the query string "?q=1&s=2" completely disappears.
I think it ought to be added to the filename.  Quoting may be a
problem?

There's another bug here too, which is that guestfish parses
the username and password together.  Note in the output above:

  "username:devel:password"

The guestfs_add_drive_opts API has a "secret" parameter which is
only used by RBD at the moment but is intended for passwords.  We
should probably use this for curl too.

Comment 2 Pino Toscano 2014-05-02 15:47:51 UTC
(In reply to Richard W.M. Jones from comment #1)
> Note the API description:
> 
> http://libguestfs.org/guestfs.3.html#guestfs_add_drive_opts
> 
> Simple example of this:
> 
> $ guestfish -x --ro -a 'http://example.com/test?q=1&s=2' |& grep example
> libguestfs: trace: add_drive "/test" "readonly:true" "protocol:http"
> "server:tcp:example.com"
> libguestfs: trace: disk_create "/tmp/libguestfsHD6cEm/overlay1" "qcow2" -1
> "backingfile:http://example.com/test"
> 
> Notice that the query string "?q=1&s=2" completely disappears.
> I think it ought to be added to the filename.  Quoting may be a
> problem?

Not sure about quoting, but for sure our URI parser discards the query string, so when creating the URL for qemu the result lacks it.

Most probably we need a new optional argument in add-drive for the query string, otherwise the query string does not get past the URL string specified to guestfish and thus not even getting into the drive struct within the library.

> There's another bug here too, which is that guestfish parses
> the username and password together.  Note in the output above:
> 
>   "username:devel:password"
> 
> The guestfs_add_drive_opts API has a "secret" parameter which is
> only used by RBD at the moment but is intended for passwords.  We
> should probably use this for curl too.

Password handling fixed with
  https://www.redhat.com/archives/libguestfs/2014-May/msg00007.html

Comment 3 Pino Toscano 2014-05-07 14:58:09 UTC
(In reply to Pino Toscano from comment #2)
> (In reply to Richard W.M. Jones from comment #1)
> > There's another bug here too, which is that guestfish parses
> > the username and password together.  Note in the output above:
> > 
> >   "username:devel:password"
> > 
> > The guestfs_add_drive_opts API has a "secret" parameter which is
> > only used by RBD at the moment but is intended for passwords.  We
> > should probably use this for curl too.
> 
> Password handling fixed with
>   https://www.redhat.com/archives/libguestfs/2014-May/msg00007.html

... and committed as
https://github.com/libguestfs/libguestfs/commit/3633109ff35f1d0f6bc72ed129911db6c4ae5c8e
which is in libguestfs >= 1.27.9.

Comment 4 Pino Toscano 2015-06-17 07:46:57 UTC
*** Bug 1232476 has been marked as a duplicate of this bug. ***