Matt, can you take a look at this please?

Can't you configure in containers.conf the plugin that is used for this volume?  Docker watches a directory for sockets and then `discovers` them while Podman requires you to define them in containers.conf, I believe.

Why is looking in a particular directory for sockets coming and going better then having a configuration file the lists where the sockets are?

Hi @Daniel Walsh,

The plugin is configured in containers.conf and that is how it is volumes are created using the plugin on first node. On the second node, even though the containers.conf has the plugin info, it is does not detect the volumes from the plugin.

Ok that would be a bug.

@dwalsh Is there any update on this?

We're looking at how to prioritize this work against other on-going work and priorities. We should be able to provide further details in the next one to two weeks. Thanks for your patience.

@mangesh, we would like to schedule a quick talk with to discuss our proposal on how to deal with this bugzilla.  would you be willing to have this conversation with us?  if so, we can directly email each other to get it set up.

@Brent Sure, we can have a call to discuss the proposals. Please email me at mangesh.panche.

I implemented this feature as podman volume reload command. This will sync the volume plugins with the libpod db volumes.
Will this work for you?

Upstream PR: https://github.com/containers/podman/pull/14713

@mangesh, does this satisfy your requirements?

This will be fixed in RHEL 8.7 and 9.1 with Podman v4.2 or later.  Setting to POST and assigning to Jindrich for any further BZ/packaging needs.

We are on RHEL 8.6 and would like the fix in upcoming release of 8.6 i.e. 8.6.0.2 which I understand is in August.
What are the timelines of 8.7?

8.7 is currently planned to be GA during November.

Regarding the solution proposed,

Is this operation "podman volume reload" idempotent? 

What's the performance impact of running the operation when 100 volumes are configured
- no new volumes discovered and
- 25 new volumes discovered?

@pholzing can you address Mangesh's questions in comment https://bugzilla.redhat.com/show_bug.cgi?id=2093079#c16 please?

> Regarding the solution proposed,
> 
> Is this operation "podman volume reload" idempotent? 

Yes, it will sync the local libpod db against the plugins state, so it will add volumes to the db if new ones
are found in the plugin but it will also remove volumes from the db if they were deleted from the plugin.
So if the plugin volumes do not change podman volume reload does nothing and returns successfully.


> What's the performance impact of running the operation when 100 volumes are
> configured
> - no new volumes discovered and
> - 25 new volumes discovered?

In all cases we have to get the full list of volumes from the configured plugins and iterate through them.
From a quick test with 100 new volumes "podman volume reload" took 0.47 seconds on my laptop and and
then running it again with no new discovered volumes it took 0.18 seconds.

@Paul Thanks for the information. 

We would need the fix in RHEL 8.6 as discussed earlier.

@pholzing please do the backport to the v4.1.1-rhel branch.

PR: https://github.com/containers/podman/pull/14860

Test with podman-4.1.1-6.module+el8.7.0+15895+a6753917.x86_64 and it works as expected.

Set up two host using one volume plugin convoy and use podman volume reload to get the volume created in another host, the volume can be get and used as expected. So set this to verified. More details:

Create from one host:
[root@kvm-06-guest25 ~]# podman volume create --driver convoy test_othernode
test_othernode

Reload and use it from another host:
[root@ibm-x3250m6-06 podman]# podman volume reload
Added:
test_othernode
[root@ibm-x3250m6-06 podman]# podman run -it -v test_othernode:/test quay.io/libpod/busybox 
Trying to pull quay.io/libpod/busybox:latest...
Getting image source signatures
Copying blob 9758c28807f2 done  
Copying config f0b02e9d09 done  
Writing manifest to image destination
Storing signatures
/ # ls 
bin   dev   etc   home  proc  root  run   sys   test  tmp   usr   var
/ # cd test
/test # ls
/test # echo "hello world" > test
/test # cat test
hello world
/test # exit
[root@ibm-x3250m6-06 podman]# podman rm -fa
602c019f8820148cd5f4106eb2de37a39f93e4de1ef738e8a06a605857d62c09
[root@ibm-x3250m6-06 podman]# podman volume rm test_othernode
test_othernode


Use reload to check if the volume is also deleted in the first host.
[root@kvm-06-guest25 ~]# podman volume reload
Removed:
test_othernode

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7457