Bug 1460623

There was a lot of movement in this area lately. Although not until 3.2.0. What's the output of "ls -ls vol_dir_pool.img vol_test".

# ls -ls vol_dir_pool.img vol_test
    0 -rw-------. 1 root root 52428800 Jun 13 04:54 vol_dir_pool.img
50684 -rw-r--r--. 1 root root 51899760 Jun 13 04:54 vol_test

Looks like the downloaded volume is getting truncated.

This situation is existed pool that type is logical. And above two xml files contents are as follows.

# vgcreate logical_pool /dev/sdk1

storage.xml

<pool type="logical">
  <name>logical_pool</name>
  <source>
    <name>logical_pool</name>
    <format type="lvm2"/>
    <device path="/dev/sdk1"/>
  </source>
  <target>
    <path>/dev/logical_pool</path>
  </target>
</pool>

volume.xml

<volume>
  <name>vol_logical_pool</name>
  <capacity unit="M">50</capacity>
  <allocation>0</allocation>
  <target>
    <path>
      /dev/logical_pool/vol_logical_pool
    </path>
  </target>
</volume>

The contents are different between vol_logical_pool and vol_test.

------- Comment From shivapbh.com 2017-07-07 08:37 EDT-------
On directory based pool after vol-download I don't see any difference between the original and downloaded volumes.

This is my host
# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Pegas)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Pegas)"
...
Libvirt - libvirt-3.2.0-15.el7a.ppc64le

On Comment 5, you mention the pool type is logical. Are you saying the issue was observed on pool backed on LVM ? What is the correlation to Comment 1 & 5? Can you reiterate the steps to reproduce as the steps mentioned in Comment 1 is not leading to the problem?

Thanks,
Shivaprasad

------- Comment From lagarcia.com 2017-08-07 12:24 EDT-------
Hello Red Hat,

Would you be able to answer the question from previous comment?

Created attachment 1311647 [details]
auto script

Created attachment 1311649 [details]
dir_pool incorrect log

Created attachment 1311650 [details]
logical_pool incorrect log

(In reply to IBM Bug Proxy from comment #6)
> ------- Comment From shivapbh.com 2017-07-07 08:37 EDT-------
> On directory based pool after vol-download I don't see any difference
> between the original and downloaded volumes.
> 
> This is my host
> # cat /etc/os-release
> NAME="Red Hat Enterprise Linux Server"
> VERSION="7.4 (Pegas)"
> ID="rhel"
> ID_LIKE="fedora"
> VERSION_ID="7.4"
> PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Pegas)"
> ...
> Libvirt - libvirt-3.2.0-15.el7a.ppc64le
> 
> On Comment 5, you mention the pool type is logical. Are you saying the issue
> was observed on pool backed on LVM ? What is the correlation to Comment 1 &
> 5? Can you reiterate the steps to reproduce as the steps mentioned in
> Comment 1 is not leading to the problem?
> 
> Thanks,
> Shivaprasad

This is my host
# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4"

# rpm -q libvirt qemu-kvm-rhev
libvirt-3.2.0-14.virtcov.el7_4.2.ppc64le
qemu-kvm-rhev-2.9.0-16.el7_4.3.ppc64le

And I feel so sorry about the reproducible is not correct.
I reproduce it by script, and dir_pool reproducible is about 10%, logical_pool reproducible is about 30%.
Finally, I add 3 attachments.
1. auto1460623.py is the script
2. dir_pool.log is the dir_pool incorrect log
3. logical_pool.log is the logical_pool incorrect log

------- Comment From niteshkonkar.com 2017-11-16 11:08 EDT-------
Hello,

I figuered it out that this is the commit that is causing the bug.

commit b7d44f450c06803df7df3ad380f7a5c97425c1e6
Author: John Ferlan <jferlan>
Date:   Fri Mar 24 09:26:17 2017 -0400

storage: Fix capacity value for LUKS encrypted volumes

https://bugzilla.redhat.com/show_bug.cgi?id=1371892

The 'capacity' value (e.g. guest logical size) for a LUKS volume is
smaller than the 'physical' value of the file in the file system, so
we need to account for that.

When peeking at the encryption information about the volume add a fetch
of the payload_offset which is described as the offset to the start of
the volume data (in 512 byte sectors) in QEMU's QCryptoBlockLUKSHeader.

Then adjust the ->capacity appropriately when we determine that the
volume target encryption has a payload_offset value.

The following series seems to fix the issue.

https://www.redhat.com/archives/libvir-list/2016-April/msg01869.html

Thanks,
Nitesh Konkar.

The sparse stream changes you mention have been merged a while ago.

Does the issue still reproduce with a recent libvirt?

------- Comment From niteshkonkar.com 2017-11-17 02:10 EDT-------
Hi,

I do not see the issue in the latest upstream libvirt.

1. virsh --version
3.10.0

2. cat /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Maipo)"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:beta:server"

REDHAT_SUPPORT_PRODUCT_VERSION="7.4 Beta"

4.hexdump  /var/lib/libvirt/images/dir_pool/vol_dir_pool.img
22600000

5. hexdump /var/lib/libvirt/images/dir_pool/vol_test
22600000

oops ... the previous comment has to read:
"
Hi,

I do not see the issue in the latest upstream libvirt. 

1. virsh --version
3.10.0

2. cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:beta:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4 Beta"

3. virsh vol-download /var/lib/libvirt/images/dir_pool/vol_dir_pool.img /var/lib/libvirt/images/dir_pool/vol_test

4.hexdump  /var/lib/libvirt/images/dir_pool/vol_dir_pool.img 
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
22600000

5. hexdump /var/lib/libvirt/images/dir_pool/vol_test
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
22600000
"
...

Andrea, Wenli, Can this one be closed?

Yup, moving it to POST now.

Reproduce:
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.4 (Maipo)
# rpm -q kernel libvirt qemu-kvm-rhev
kernel-3.10.0-693.17.1.el7.ppc64le
libvirt-3.2.0-14.el7_4.9.ppc64le
qemu-kvm-rhev-2.9.0-16.el7_4.14.ppc64le
Steps to Reproduce:
1. virsh pool-create storage.xml [as follows]
<pool type='dir'>
  <name>dir_pool</name>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images/dir_pool</path>
  </target>
</pool>

2. virsh vol-create dir_pool volume.xml [as follows]
<volume>
  <name>vol_dir_pool.img</name>
  <capacity unit="M">50</capacity>
  <allocation>0</allocation>
  <target>
    <path>/var/lib/libvirt/images/dir_pool/vol_dir_pool.img</path>
    <format type="raw"/>
  </target>
</volume>
3. Use the scripts(https://bugzilla.redhat.com/attachment.cgi?id=1311647)
4. The bug could be reproduce


Verify:
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.5 Beta (Maipo)
# rpm -q qemu-kvm-rhev libvirt kernel
qemu-kvm-rhev-2.10.0-18.el7.ppc64le
libvirt-3.9.0-9.virtcov.el7.ppc64le
kernel-3.10.0-837.el7.ppc64le

Steps to Reproduce:
1. virsh pool-create storage.xml [as follows]
<pool type='dir'>
  <name>dir_pool</name>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images/dir_pool</path>
  </target>
</pool>

2. virsh vol-create dir_pool volume.xml [as follows]
<volume>
  <name>vol_dir_pool.img</name>
  <capacity unit="M">50</capacity>
  <allocation>0</allocation>
  <target>
    <path>/var/lib/libvirt/images/dir_pool/vol_dir_pool.img</path>
    <format type="raw"/>
  </target>
</volume>
3. Use the scripts(https://bugzilla.redhat.com/attachment.cgi?id=1311647)
4. Try 100000 times, and could not reproduce the bug.

According that, the bug could set to verified

According to comment 18, set to verified

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704