Bug 1557995

This was intentionally removed in commit 577d8c9a ('rbd: Revert -blockdev parameter password-secret'), together with commit 464444fc ('rbd: Revert -blockdev and -drive parameter auth-supported'), because of design doubts:


>    rbd: Revert -blockdev parameter password-secret
>
>    This reverts a part of commit 8a47e8e.  We're having second thoughts
>    on the QAPI schema (and thus the external interface), and haven't
>    reached consensus, yet.  Issues include:
>    
>    * BlockdevOptionsRbd member @password-secret isn't actually a
>      password, it's a key generated by Ceph.
>    
>    * We're not sure where member @password-secret belongs (see the
>      previous commit).
>    
>    * How @password-secret interacts with settings from a configuration
>      file specified with @conf is undocumented.
>    
>    Let's avoid painting ourselves into a corner now, and revert the
>    feature for 2.9.
>    
>    Note that users can still configure an authentication key with a
>    configuration file.  They probably do that anyway if they use Ceph
>    outside QEMU as well.


>    rbd: Revert -blockdev and -drive parameter auth-supported
>
>    This reverts half of commit 0a55679.  We're having second thoughts on
>    the QAPI schema (and thus the external interface), and haven't reached
>    consensus, yet.  Issues include:
>    
>    * The implementation uses deprecated rados_conf_set() key
>      "auth_supported".  No biggie.
>    
>    * The implementation makes -drive silently ignore invalid parameters
>      "auth" and "auth-supported.*.X" where X isn't "auth".  Fixable (in
>      fact I'm going to fix similar bugs around parameter server), so
>      again no biggie.
>    
>    * BlockdevOptionsRbd member @password-secret applies only to
>      authentication method cephx.  Should it be a variant member of
>      RbdAuthMethod?
>    
>    * BlockdevOptionsRbd member @user could apply to both methods cephx
>      and none, but I'm not sure it's actually used with none.  If it
>      isn't, should it be a variant member of RbdAuthMethod?
>    
>    * The client offers a *set* of authentication methods, not a list.
>      Should the methods be optional members of BlockdevOptionsRbd instead
>      of members of list @auth-supported?  The latter begs the question
>      what multiple entries for the same method mean.  Trivial question
>      now that RbdAuthMethod contains nothing but @type, but less so when
>      RbdAuthMethod acquires other members, such the ones discussed above.
>    
>    * How BlockdevOptionsRbd member @auth-supported interacts with
>      settings from a configuration file specified with @conf is
>      undocumented.  I suspect it's untested, too.
>
>    Let's avoid painting ourselves into a corner now, and revert the
>    feature for 2.9.
>    
>    Note that users can still configure authentication methods with a
>    configuration file.  They probably do that anyway if they use Ceph
>    outside QEMU as well.


It's probably time to restart the discussion what the options should really look like. At the same time, a workaround is suggested (using a configuration file).

Would using a configuration file be feasible for libvirt? There is another thing that QEMU supports on the command line, but not in QMP, which is passing through arbitrary RBD configuration options. So maybe using a config file would be the most future-proof thing for libvirt anyway.

The configuration file workaround isn't really a feasible as libvirt tolerated users using the configuration file themselves.

This would mean that libvirt would have to load the exsiting file and add the authentication variables internally, so that all other options are preserved and this is not really feasible as a workaround. More specifically, since it's possible to use the 'password-secret' option from the command line adding such a workaround for -blockdev is really not desired at all.

Upstream commit d083f954a95d37b460df0c2fbfe46ad7eb207b10 provides 'key-secret' in -blockdev, blockdev-add and -drive.

The previous commit provides 'auth-client-required'.

Fix included in qemu-kvm-rhev-2.12.0-5.el7

QE wants to know the exact format to use "key-secret", please help to check the trials below:
1. no explicit secret, use /etc/ceph/ceph.client.admin.keyring.
# qemu-kvm -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,VNC server running on ::1:5900

2. explicitly specify secret for user "admin"
# qemu-kvm --object secret,id=sec0,data=`printf %s "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0
qemu-kvm: -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting: Invalid argument

Yes, if you don't specify key-secret, then the key defaults to whatever your configuration files specify.  Refer to Ceph documentation for the exact list of configuration files used.  For me, it's /etc/ceph/ceph.conf ~/.ceph/ceph.conf ceph.conf.

You example 1. got mangled; input and output aren't separated.  I'm guessing you mean

# qemu-kvm -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx

Hint: you should be able to omit "raw", like this:

# qemu-kvm -blockdev node-name=disk,driver=rbd,pool=rbd,image=convert.raw,server.0.host=10.73.196.181,server.0.port=6789,user=admin,auth-client-required.0=cephx

I think your example 2. is missing format=base64 in -object.  Please try:

# qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev node-name=disk,driver=rbd,pool=rbd,image=convert.raw,server.0.host=10.73.196.181,server.0.port=6789,user=admin,auth-client-required.0=cephx,key-secret=sec0

Hi, Markus, here is result with format specified, still error connecting.

# qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0
qemu-kvm: -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting: Invalid argument

(In reply to Longxiang Lyu from comment #10)
> Hi, Markus, here is result with format specified, still error connecting.
> 
> # qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s
> "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev

Isn't 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' already base64 encoded?  Have you tried just passing 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' as your data?


> driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> file.auth-client-required.0=cephx,file.key-secret=sec0
> qemu-kvm: -blockdev
> driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting:
> Invalid argument

(In reply to Jeff Cody from comment #11)
> (In reply to Longxiang Lyu from comment #10)
> > Hi, Markus, here is result with format specified, still error connecting.
> > 
> > # qemu-kvm --object secret,id=sec0,format=base64,data=`printf %s
> > "AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew=="|base64` -blockdev
> 
> Isn't 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' already base64 encoded? 
> Have you tried just passing 'AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==' as
> your data?
> 
> 
> > driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> > raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> > file.auth-client-required.0=cephx,file.key-secret=sec0
> > qemu-kvm: -blockdev
> > driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.
> > raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,
> > file.auth-client-required.0=cephx,file.key-secret=sec0: error connecting:
> > Invalid argument

thanks for your remind, Jeff. Now I could boot the image successfully.

# qemu-kvm --object secret,id=sec0,format=base64,data="AQBtCyJblJvNDxAAHy8O94K9MaS+vOfe91C/ew==" -blockdev driver=raw,node-name=disk,file.driver=rbd,file.pool=rbd,file.image=convert.raw,file.server.0.host=10.73.196.181,file.server.0.port=6789,file.user=admin,file.auth-client-required.0=cephx,file.key-secret=sec0 -monitor stdio
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900

(qemu) info block -n
disk: json:{"driver": "raw", "file": {"pool": "rbd", "image": "convert.raw", "server.0.host": "10.73.196.181", "driver": "rbd", "server.0.port": "6789", "user": "admin", "auth-client-required.0": "cephx", "key-secret": "sec0"}} (raw)
    Cache mode:       writeback

#block002: json:{"pool": "rbd", "image": "convert.raw", "server.0.host": "10.73.196.181", "driver": "rbd", "server.0.port": "6789", "user": "admin", "auth-client-required.0": "cephx", "key-secret": "sec0"} (rbd)
    Cache mode:       writeback

Could boot rbd images with explicit specification of authentication method and key secret.

Thanks for the help received and mark this bug as verified then.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3443