Bug 1557995 - QAPI schema for RBD storage misses the 'password-secret' option
Summary: QAPI schema for RBD storage misses the 'password-secret' option
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.5
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Markus Armbruster
QA Contact: Longxiang Lyu
URL:
Whiteboard:
Depends On:
Blocks: 760547 1558125 1596511
TreeView+ depends on / blocked
 
Reported: 2018-03-19 12:23 UTC by Peter Krempa
Modified: 2018-11-01 11:07 UTC (History)
10 users (show)

Fixed In Version: qemu-kvm-rhev-2.12.0-5.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1596511 (view as bug list)
Environment:
Last Closed: 2018-11-01 11:07:15 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Peter Krempa 2018-03-19 12:23:27 UTC
Description of problem:
According to the qapi schema, there's no way to pass the secret object name for RBD authentication via -blockdev/blockdev-add. QAPI schema documments following fields:

{ 'struct': 'BlockdevOptionsRbd',
  'data': { 'pool': 'str',
            'image': 'str',
            '*conf': 'str',
            '*snapshot': 'str',
            '*user': 'str',
            '*server': ['InetSocketAddressBase'] } }


Version-Release number of selected component (if applicable):
current upstream (so older versions should be affected too)

Comment 1 Kevin Wolf 2018-04-04 10:57:50 UTC
This was intentionally removed in commit 577d8c9a ('rbd: Revert -blockdev parameter password-secret'), together with commit 464444fc ('rbd: Revert -blockdev and -drive parameter auth-supported'), because of design doubts:


>    rbd: Revert -blockdev parameter password-secret
>
>    This reverts a part of commit 8a47e8e.  We're having second thoughts
>    on the QAPI schema (and thus the external interface), and haven't
>    reached consensus, yet.  Issues include:
>    
>    * BlockdevOptionsRbd member @password-secret isn't actually a
>      password, it's a key generated by Ceph.
>    
>    * We're not sure where member @password-secret belongs (see the
>      previous commit).
>    
>    * How @password-secret interacts with settings from a configuration
>      file specified with @conf is undocumented.
>    
>    Let's avoid painting ourselves into a corner now, and revert the
>    feature for 2.9.
>    
>    Note that users can still configure an authentication key with a
>    configuration file.  They probably do that anyway if they use Ceph
>    outside QEMU as well.


>    rbd: Revert -blockdev and -drive parameter auth-supported
>
>    This reverts half of commit 0a55679.  We're having second thoughts on
>    the QAPI schema (and thus the external interface), and haven't reached
>    consensus, yet.  Issues include:
>    
>    * The implementation uses deprecated rados_conf_set() key
>      "auth_supported".  No biggie.
>    
>    * The implementation makes -drive silently ignore invalid parameters
>      "auth" and "auth-supported.*.X" where X isn't "auth".  Fixable (in
>      fact I'm going to fix similar bugs around parameter server), so
>      again no biggie.
>    
>    * BlockdevOptionsRbd member @password-secret applies only to
>      authentication method cephx.  Should it be a variant member of
>      RbdAuthMethod?
>    
>    * BlockdevOptionsRbd member @user could apply to both methods cephx
>      and none, but I'm not sure it's actually used with none.  If it
>      isn't, should it be a variant member of RbdAuthMethod?
>    
>    * The client offers a *set* of authentication methods, not a list.
>      Should the methods be optional members of BlockdevOptionsRbd instead
>      of members of list @auth-supported?  The latter begs the question
>      what multiple entries for the same method mean.  Trivial question
>      now that RbdAuthMethod contains nothing but @type, but less so when
>      RbdAuthMethod acquires other members, such the ones discussed above.
>    
>    * How BlockdevOptionsRbd member @auth-supported interacts with
>      settings from a configuration file specified with @conf is
>      undocumented.  I suspect it's untested, too.
>
>    Let's avoid painting ourselves into a corner now, and revert the
>    feature for 2.9.
>    
>    Note that users can still configure authentication methods with a
>    configuration file.  They probably do that anyway if they use Ceph
>    outside QEMU as well.


It's probably time to restart the discussion what the options should really look like. At the same time, a workaround is suggested (using a configuration file).

Would using a configuration file be feasible for libvirt? There is another thing that QEMU supports on the command line, but not in QMP, which is passing through arbitrary RBD configuration options. So maybe using a config file would be the most future-proof thing for libvirt anyway.

Comment 2 Peter Krempa 2018-04-06 06:02:31 UTC
The configuration file workaround isn't really a feasible as libvirt tolerated users using the configuration file themselves.

This would mean that libvirt would have to load the exsiting file and add the authentication variables internally, so that all other options are preserved and this is not really feasible as a workaround. More specifically, since it's possible to use the 'password-secret' option from the command line adding such a workaround for -blockdev is really not desired at all.

Comment 3 Markus Armbruster 2018-06-15 17:16:34 UTC
Upstream commit d083f954a95d37b460df0c2fbfe46ad7eb207b10 provides 'key-secret' in -blockdev, blockdev-add and -drive.

The previous commit provides 'auth-client-required'.

Comment 6 Miroslav Rezanina 2018-06-25 14:10:15 UTC
Fix included in qemu-kvm-rhev-2.12.0-5.el7

Comment 8 Longxiang Lyu 2018-07-09 08:50:43 UTC
QE wants to know the exact format to use "key-secret", please help to check the trials below:
1. no explicit secret, use /etc/ceph/ceph.client.admin.keyring.
# qemu-kvm -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,VNC server running on ::1:5900

2. explicitly specify secret for user "admin"
# qemu-kvm --object secret,id=sec0,data=`printf %s "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0
qemu-kvm: -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting: Invalid argument

Comment 9 Markus Armbruster 2018-07-09 09:11:08 UTC
Yes, if you don't specify key-secret, then the key defaults to whatever your configuration files specify.  Refer to Ceph documentation for the exact list of configuration files used.  For me, it's /etc/ceph/ceph.conf ~/.ceph/ceph.conf ceph.conf.

You example 1. got mangled; input and output aren't separated.  I'm guessing you mean

# qemu-kvm -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx

Hint: you should be able to omit "raw", like this:

# qemu-kvm -blockdev node-name=disk,driver=rbd,pool=rbd,image=convert.raw,server.0.host=10.73.196.181,server.0.port=6789,user=admin,auth-client-required.0=cephx

I think your example 2. is missing format=base64 in -object.  Please try:

# qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev node-name=disk,driver=rbd,pool=rbd,image=convert.raw,server.0.host=10.73.196.181,server.0.port=6789,user=admin,auth-client-required.0=cephx,key-secret=sec0

Comment 10 Longxiang Lyu 2018-07-09 09:55:21 UTC
Hi, Markus, here is result with format specified, still error connecting.

# qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0
qemu-kvm: -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting: Invalid argument

Comment 11 Jeff Cody 2018-07-09 12:43:30 UTC
(In reply to Longxiang Lyu from comment #10)
> Hi, Markus, here is result with format specified, still error connecting.
> 
> # qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s
> "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev

Isn't 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' already base64 encoded?  Have you tried just passing 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' as your data?


> driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> file.auth-client-required.0=cephx,file.key-secret=sec0
> qemu-kvm: -blockdev
> driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting:
> Invalid argument

Comment 12 Longxiang Lyu 2018-07-10 02:36:52 UTC
(In reply to Jeff Cody from comment #11)
> (In reply to Longxiang Lyu from comment #10)
> > Hi, Markus, here is result with format specified, still error connecting.
> > 
> > # qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s
> > "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev
> 
> Isn't 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' already base64 encoded? 
> Have you tried just passing 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' as
> your data?
> 
> 
> > driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> > raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> > file.auth-client-required.0=cephx,file.key-secret=sec0
> > qemu-kvm: -blockdev
> > driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> > raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> > file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting:
> > Invalid argument

thanks for your remind, Jeff. Now I could boot the image successfully.

Comment 13 Longxiang Lyu 2018-07-10 02:43:18 UTC
# qemu-kvm --object secret,id=sec0,format=base64,data="AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==" -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0 -monitor stdio
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900

(qemu) info block -n
disk: json:{"driver": "raw", "file": {"pool": "rbd", "image": "convert.raw", "server.0.host": "10.73.196.181", "driver": "rbd", "server.0.port": "6789", "user": "admin", "auth-client-required.0": "cephx", "key-secret": "sec0"}} (raw)
    Cache mode:       writeback

#block002: json:{"pool": "rbd", "image": "convert.raw", "server.0.host": "10.73.196.181", "driver": "rbd", "server.0.port": "6789", "user": "admin", "auth-client-required.0": "cephx", "key-secret": "sec0"} (rbd)
    Cache mode:       writeback

Could boot rbd images with explicit specification of authentication method and key secret.

Thanks for the help received and mark this bug as verified then.

Comment 14 errata-xmlrpc 2018-11-01 11:07:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3443


Note You need to log in before you can comment on or make changes to this bug.