Bug 2228223

Summary: libvirt storage pool goes inactive
Product: Red Hat Enterprise Linux 9 Reporter: schandle
Component: libvirtAssignee: Peter Krempa <pkrempa>
libvirt sub component: Storage QA Contact: Meina Li <meili>
Status: POST --- Docs Contact:
Severity: high    
Priority: unspecified CC: hreitz, lmen, pkrempa, vgoyal, virt-maint
Version: 9.2Keywords: Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description schandle 2023-08-01 18:40:31 UTC 
Description of problem:
When configuring a logical storage pool virsh, the storage pool will automatically go into an "inactive" state after a short period of time (5-10 minutes).

Version-Release number of selected component (if applicable):
kernel-5.14.0-284.11.1.el9_2.x86_64                         Thu Jun 22 07:07:42 2023
libvirt-9.0.0-10.2.el9_2.x86_64
lvm2-2.03.17-7.el9.x86_64


How reproducible:
100% 

Steps to Reproduce:
1. install qemu-kvm libvirt virt-install virt-viewer
2. virsh pool-define-as storage-pool logical --source-dev /dev/sda --target=/dev/storage-pool
3. virsh pool-build storage-pool
4  virsh pool-start storage-pool
5. virsh pool-autostart storage-pool

Actual results:

# date ; virsh pool-list --all
Tue Aug  1 07:49:35 AM EDT 2023
 Name                            State    Autostart
-----------------------------------------------------
 storage-pool                    active   yes

# date ; virsh pool-list --all
Tue Aug  1 07:55:03 AM EDT 2023
 Name                            State      Autostart
-------------------------------------------------------
 storage-pool                    inactive   yes


Expected results:
For the storage pool to stay active or autostart if virtstoraged is restarted.  

Additional info:
It appears that virt*d.service are being restarted   
~~~
Aug  1 07:55:03 hostname systemd[1]: Starting Virtualization qemu daemon...
Aug  1 07:55:03 hostname systemd[1]: Started Virtualization qemu daemon.
Aug  1 07:55:03 hostname systemd[1]: Starting Virtualization storage daemon...
Aug  1 07:55:03 hostname systemd[1]: Started Virtualization storage daemon.
Aug  1 07:55:26 hostname systemd[1]: Starting Virtualization network daemon...
Aug  1 07:55:26 hostname systemd[1]: Started Virtualization network daemon.
Aug  1 07:55:27 hostname systemd[1]: Starting Virtualization nwfilter daemon...
Aug  1 07:55:27 hostname systemd[1]: Started Virtualization nwfilter daemon.
Aug  1 07:55:27 hostname systemd[1]: Starting Virtualization nodedev daemon...
Aug  1 07:55:27 hostname systemd[1]: Started Virtualization nodedev daemon.
Aug  1 07:55:37 hostname systemd[89815]: Starting Virtual filesystem service...
Aug  1 07:55:37 hostname systemd[89815]: Started Virtual filesystem service.
~~~
Comment 2 Hanna Czenczek 2023-08-02 08:38:17 UTC 
Peter, can you take a look at the logs and see what this might be about (or who might have an idea)?
Comment 3 Meina Li 2023-08-03 06:50:51 UTC 
Can reproduce this bug on:

libvirt-9.0.0-10.2.el9_2.x86_64
qemu-kvm-7.2.0-14.el9_2.3.x86_64
and
libvirt-9.5.0-4.el9.x86_64
qemu-kvm-8.0.0-10.el9.x86_64


Test Steps:
Just like the steps in description.

Additional info:
1. Restart libvirtd/virtstoraged, the status of pool will become inactive.
2. Restart virtqemud, the status will not change.
Comment 4 Peter Krempa 2023-08-08 14:01:08 UTC 
The issue is that the convenience directory for LVs of a VG in '/dev/' is not created for an empty VG. Since libvirt was first checking the directory it assumed the pool does not exist.

I've posted a patch:

https://listman.redhat.com/archives/libvir-list/2023-August/241150.html
Comment 5 Peter Krempa 2023-08-17 11:58:53 UTC 
Fixed upstream:

commit fa1a54baa59d244289ce666f9dc52d9eabca47f1
Author: Peter Krempa <pkrempa>
Date:   Tue Aug 8 15:53:53 2023 +0200

    virStorageBackendLogicalCheckPool: Properly mark empty logical pools as active
    
    The '/dev' filesystem convenience directory for a LVM volume group is
    not created when the volume group is empty.
    
    The logic in 'virStorageBackendLogicalCheckPool' which is used to see
    whether a pool is active was first checking presence of the directory,
    which failed for an empty VG.
    
    Since the second step is virStorageBackendLogicalMatchPoolSource which
    is checking mapping between configured PVs and the VG, we can simply
    rely on the function to also check presence of the pool.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2228223
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

v9.6.0-26-gfa1a54baa5