Is this going to be transparent in terms of libvirt XML, or
do we plan to model this through the XML?  In other words
would we just want to transparently change the implementation of:

  <disk type='network'>
    <source protocol="https" name="url_path">
  ...

or would we model it like:

  <disk type='network'>
    <source protocol="external">
      <program>nbdkit</program>
      <args>...

(or however that would work)

(In reply to Richard W.M. Jones from comment #1)
> Is this going to be transparent in terms of libvirt XML, or
> do we plan to model this through the XML?  In other words
> would we just want to transparently change the implementation of:
> 
>   <disk type='network'>
>     <source protocol="https" name="url_path">
>   ...

If we want to transparently fix the issue of the curl driver being removed from qemu we need to switch to nbdkit when it's not present with identical configuration. Otherwise the higher level APP would need to change their behaviour to ...

 
> or would we model it like:
> 
>   <disk type='network'>
>     <source protocol="external">
>       <program>nbdkit</program>
>       <args>...
> 
> (or however that would work)

... use this new XML.

It would make sense to start nbdkit separately, but generally we try to avoid giving the user the ability to specify arbitrary arguments for a program as it doesn't give us a chance to abstract changes in the helper program. In this instance the only thing the user will get is security isolation.

Sure thing, I agree.  By the way if the nbdkit support is generic enough
then it could be used to enable VDDK support using nbdkit-vddk-plugin, eg:

  <disk type='network'>
    <source protocol="vddk" file="[datastore1] Fedora/Fedora.vmdk">
      <host name="esxi"/>
      <auth username='root'/>
      <libdir>/opt/vmware-vix-disklib-distrib</libdir>
    </source>
   ...
  </disk>

https://libguestfs.org/nbdkit-vddk-plugin.1.html

So, Rich's comment makes me wonder -- are http and https the only protocols we are currently planning to move up to the libvirt level? Or will there be more dependencies removed from qemu-kvm that will cause similar issues for other protocols (e.g. ssh, etc)?

ssh is likely too IMHO.  We've spent less time on nbdkit-ssh-plugin,
but it is still slightly more featureful than the qemu driver, and
because of (low) performance characteristics makes sense to move out
of qemu too.

oh, indeed the ssh plugin was actually removed as a dependency from qemu in rhel 9 as well, I thought it was only curl so far. Updating title.

(In reply to Jonathon Jongsma from comment #4)
> So, Rich's comment makes me wonder -- are http and https the only protocols
> we are currently planning to move up to the libvirt level? Or will there be
> more dependencies removed from qemu-kvm that will cause similar issues for
> other protocols (e.g. ssh, etc)?

Yes, curl and ssh protocols are now optional (not required by qemu-kvm).

(In reply to Richard W.M. Jones from comment #3)
> Sure thing, I agree.  By the way if the nbdkit support is generic enough
> then it could be used to enable VDDK support using nbdkit-vddk-plugin

The more generic support sounds like something that could maybe supported by the future XML syntax that enables using a storage daemon, because nbdkit essentially just functions as a limited storage daemon in this case. Limited because it doesn't have QMP or anything like that, so I suppose certain features like storage migration or taking snapshots might not be possible when it's used.

@pkrempa This sounds like something to discuss in our planned storage daemon meeting.

The direct answer to how to do snapshotting (etc) is that you'd do it
in exactly the same way as snapshotting any NBD device, ie. qemu or q-s-d
does it.  It's entirely conceivable that q-s-d would be involved here.

The point of this change is to deal with 3 or 4 different problems:

(1a) Some drivers in qemu are not really maintained (eg. ssh), and the
nbdkit equivalent is both maintained and far more featureful.

(1b) nbdkit-curl-plugin is also much more featureful, and in particular
can do cookie and header renewal which is essential for accessing
certain non-public webservices like docker registries.

(2) Some drivers cannot be in qemu for licensing reasons (eg. vddk).

(3) Some drivers cannot be in qemu for technical reasons
(eg. vddk because it requires a re-exec(2) to reinitialize the
process, the imageio driver which is written in Python).

(4) We'd like to apply fine-grained SELinux policies per block driver.

(In reply to Richard W.M. Jones from comment #9)
> The direct answer to how to do snapshotting (etc) is that you'd do it
> in exactly the same way as snapshotting any NBD device, ie. qemu or q-s-d
> does it.  It's entirely conceivable that q-s-d would be involved here.

Right, so libvirt may need to support one storage daemon (nbdkit) serving another storage daemon (qsd), and also accessing the external process not for a whole disk image, but just for a backing file. That's a pretty important data point when designing how libvirt manages things.

> (1a) Some drivers in qemu are not really maintained (eg. ssh), and the
> nbdkit equivalent is both maintained and far more featureful.

It's indeed a sad truth that we decided to duplicate some efforts with nbdkit, inevitably leading to one implementation being less actively maintained than the other. But what you're using it for is not really relevant for the point I was making, which is about a generic way to use storage daemons (potentially including nbdkit) from libvirt.

v3 on the list,
https://listman.redhat.com/archives/libvir-list/2022-October/235052.html

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.