Bug 2026537

Summary: Can't vol-download volume with --sparse option in split daemon mode
Product: Red Hat Enterprise Linux 9 Reporter: Meina Li <meili>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
libvirt sub component: General QA Contact: Meina Li <meili>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: jdenemar, lmen, mprivozn, virt-maint, xuzhang
Version: 9.0Keywords: Automation, Triaged, Upstream
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-8.0.0-0rc1.1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-17 12:45:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version: 8.0.0
Embargoed:

Description Meina Li 2021-11-25 02:15:57 UTC
Description of problem:
Can't vol-download volume with --sparse option in split daemon mode

Version-Release number of selected component (if applicable):
libvirt-7.9.0-1.el9.x86_64
qemu-kvm-6.1.0-6.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Create a pool and a volume for it.
# virsh pool-create-as --name dir-pool --type dir --target /var/tmp/dir-pool
Pool dir-pool created
# virsh vol-create-as --pool dir-pool dir-vol --capacity 10485760 --allocation 10485760 --format qcow2
Vol dir-vol created
# virsh pool-refresh dir-pool
Pool dir-pool refreshed

2. Prepare a disk image and attach it to the guest.
# qemu-img create -f raw /var/tmp/dir-pool/test.img 1G
Formatting '/var/tmp/dir-pool/test.img', fmt=raw size=1073741824
# cat /tmp/test.xml 
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/tmp/dir-pool/test.img'/>
      <target dev='vdb' bus='virtio'/>
    </disk>
# virsh attach-device avocado-vt-vm1 /tmp/test.xml --config
Device attached successfully

3. Write some data to the disk.
# virsh start avocado-vt-vm1
Domain 'avocado-vt-vm1' started
[in guest] #mkfs.ext4 -F /dev/vdb && mkdir -p test && mount /dev/vdb test && dd if=/dev/urandom of=test/file bs=1M count=100 && sync

4. Vol-download the volume with --sparse option.
# virsh pool-refresh dir-pool
Pool dir-pool refreshed
# virsh vol-download test.img /var/tmp/download-sparse.raw --pool dir-pool --sparse
error: cannot receive data from volume test.img
error: this function is not supported by the connection driver: virStreamInData

Actual results:
Can't vol-download the volume with --sparse option with unsupported error

Expected results:
Can vol-download the volume with --sparse option

Additional info:
Can run successfully with virsh -c storage:///system:
# virsh -c storage:///system vol-download test.img /var/tmp/download-sparse.raw --pool dir-pool --sparse

Comment 1 Meina Li 2021-11-25 02:24:58 UTC
Enable split daemon mode:
# cat split-daemon.sh 
systemctl stop libvirtd.service
systemctl stop libvirtd{,-ro,-admin,-tcp,-tls}.socket
systemctl start virtqemud
systemctl start virtsecretd
systemctl start virtstoraged
systemctl start virtproxyd

Comment 2 Michal Privoznik 2021-11-25 10:11:19 UTC
This is because remote driver isn't prepared to call virStreamInData(). I mean, when designing sparse streams, virStreamInData() was meant to be local only, there's no RPC representation. Let me see if we need one or we can go with something different.

Comment 3 Michal Privoznik 2021-12-07 15:37:07 UTC
Patches posted on the list:

https://listman.redhat.com/archives/libvir-list/2021-December/msg00211.html

Comment 4 Michal Privoznik 2021-12-13 13:50:36 UTC
Merged upstream as:

c1b06f5cf0 remote_driver: Implement virStreamInData() callback
2981d1c95e rpc: Introduce virNetClientStreamInData()
dd75e2e464 virStreamInData: Allow callback to not rewind the stream

v7.10.0-227-gc1b06f5cf0

Comment 5 Meina Li 2021-12-15 03:36:09 UTC
Pre-verified Version:
libvirt-8.0.0-1.fc35.x86_64
qemu-kvm-6.1.0-13.fc35.x86_64

Pre-verified Steps:
1.Prepare a split daemon environment:
systemctl stop libvirtd.service
systemctl stop libvirtd{,-ro,-admin,-tcp,-tls}.socket
systemctl start virtqemud
systemctl start virtsecretd
systemctl start virtstoraged
systemctl start virtproxyd
systemctl start virtnetworkd

2.Run vol-download and vol-upload according the following test case.
RHEL-115530 - [vol-download][vol-upload][sparse file] download and upload volume with --sparse param

Comment 6 Meina Li 2021-12-15 07:20:14 UTC
Add another scenario: 
RHEL-189537 - [storage][vol-download][vol-upload][sparse file] Download and upload volume with --sparse param when source/target is block device

Comment 10 Meina Li 2022-01-24 03:57:42 UTC
Verified Version:
libvirt-8.0.0-1.el9.x86_64
qemu-kvm-6.2.0-3.el9.x86_64

Verified Steps:
For basic test steps:
1. Create a pool and a volume for it.
# virsh pool-create-as --name dir-pool --type dir --target /var/tmp/dir-pool
Pool dir-pool created
# virsh vol-create-as --pool dir-pool dir-vol --capacity 10485760 --allocation 10485760 --format qcow2
Vol dir-vol created
# virsh pool-refresh dir-pool
Pool dir-pool refreshed

2. Prepare a disk image and attach it to the guest.
# qemu-img create -f raw /var/tmp/dir-pool/test.img 1G
Formatting '/var/tmp/dir-pool/test.img', fmt=raw size=1073741824
# cat /tmp/test.xml 
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/tmp/dir-pool/test.img'/>
      <target dev='vdb' bus='virtio'/>
    </disk>
# virsh attach-device avocado-vt-vm1 /tmp/test.xml --config
Device attached successfully

3. Write some data to the disk.
# virsh start avocado-vt-vm1
Domain 'avocado-vt-vm1' started
[in guest] #mkfs.ext4 -F /dev/vdb && mkdir -p test && mount /dev/vdb额日发ied test && dd if=/dev/urandom of=test/file bs=1M count=100 && sync

4. Vol-download the volume with --sparse option.
# virsh pool-refresh dir-pool
Pool dir-pool refreshed
# virsh vol-download test.img /var/tmp/download-sparse.raw --pool dir-pool --sparse
---Can vol-download with --sparse successfully
5. For complicated test steps, can refer to RHEL-115530 and RHEL-189537 in Links.

Comment 12 errata-xmlrpc 2022-05-17 12:45:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: libvirt), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2390