RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1460623 - There are differences between downloaded data from the given volume and the original volume content
Summary: There are differences between downloaded data from the given volume and the o...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.4
Hardware: ppc64le
OS: Linux
high
high
Target Milestone: rc
: 7.5
Assignee: Andrea Bolognani
QA Contact: Junxiang Li
URL:
Whiteboard:
Depends On:
Blocks: 1399177
TreeView+ depends on / blocked
 
Reported: 2017-06-12 08:56 UTC by wenli
Modified: 2018-04-10 10:50 UTC (History)
19 users (show)

Fixed In Version: libvirt-3.9.0-4.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-10 10:48:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
auto script (639 bytes, text/plain)
2017-08-10 09:55 UTC, Junxiang Li 		no flags Details
dir_pool incorrect log (164.79 KB, text/plain)
2017-08-10 09:57 UTC, Junxiang Li 		no flags Details
logical_pool incorrect log (175.14 KB, text/plain)
2017-08-10 09:58 UTC, Junxiang Li 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
IBM Linux Technology Center 155953 0 None None None 2019-03-22 07:28:08 UTC
Red Hat Product Errata RHEA-2018:0704 0 None None None 2018-04-10 10:50:02 UTC

Description wenli 2017-06-12 08:56:39 UTC 
Description of problem:
There are difference between downloaded data from the given volume and the original volume content. And this question is occurred on ppc machine only.

Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux Server release 7.4 Beta (Maipo)
Linux ibm-p8-garrison-02.rhts.eng.bos.redhat.com 3.10.0-675.el7.ppc64le #1 SMP Mon May 29 23:22
libvirt-3.2.0-9.el7.ppc64le

How reproducible:
80%

Steps to Reproduce:
1. virsh pool-create storage.xml [as follows]
<pool type='dir'>
  <name>dir_pool</name>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images/dir_pool</path>
  </target>
</pool>

2. virsh vol-create dir_pool volume.xml [as follows]
<volume>
  <name>vol_dir_pool.img</name>
  <capacity unit="M">50</capacity>
  <allocation>0</allocation>
  <target>
    <path>/var/lib/libvirt/images/dir_pool/vol_dir_pool.img</path>
    <format type="raw"/>
  </target>
</volume>

3. virsh vol-download /var/lib/libvirt/images/dir_pool/vol_dir_pool.img /var/lib/libvirt/images/dir_pool/vol_test

4. cd /var/lib/libvirt/images/dir_pool && diff vol_dir_pool.img vol_test

Actual results:
Binary files vol_dir_pool.img and vol_test are difference, so I hexdump both file, and get follow value.

# hexdump vol_dir_pool.img

0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
3200000

# hexdump vol_test

0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
317ed70

Expected results:
There are the same content and hex value, no different between them.
Comment 2 Michal Privoznik 2017-06-13 08:44:06 UTC 
There was a lot of movement in this area lately. Although not until 3.2.0. What's the output of "ls -ls vol_dir_pool.img vol_test".
Comment 3 wenli 2017-06-13 08:56:25 UTC 
# ls -ls vol_dir_pool.img vol_test
    0 -rw-------. 1 root root 52428800 Jun 13 04:54 vol_dir_pool.img
50684 -rw-r--r--. 1 root root 51899760 Jun 13 04:54 vol_test
Comment 4 David Gibson 2017-06-14 01:55:25 UTC 
Looks like the downloaded volume is getting truncated.
Comment 5 wenli 2017-06-14 04:09:07 UTC 
This situation is existed pool that type is logical. And above two xml files contents are as follows.

# vgcreate logical_pool /dev/sdk1

storage.xml

<pool type="logical">
  <name>logical_pool</name>
  <source>
    <name>logical_pool</name>
    <format type="lvm2"/>
    <device path="/dev/sdk1"/>
  </source>
  <target>
    <path>/dev/logical_pool</path>
  </target>
</pool>

volume.xml

<volume>
  <name>vol_logical_pool</name>
  <capacity unit="M">50</capacity>
  <allocation>0</allocation>
  <target>
    <path>
      /dev/logical_pool/vol_logical_pool
    </path>
  </target>
</volume>

The contents are different between vol_logical_pool and vol_test.
Comment 6 IBM Bug Proxy 2017-07-07 12:40:24 UTC 
------- Comment From shivapbh.com 2017-07-07 08:37 EDT-------
On directory based pool after vol-download I don't see any difference between the original and downloaded volumes.

This is my host
# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Pegas)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Pegas)"
...
Libvirt - libvirt-3.2.0-15.el7a.ppc64le

On Comment 5, you mention the pool type is logical. Are you saying the issue was observed on pool backed on LVM ? What is the correlation to Comment 1 & 5? Can you reiterate the steps to reproduce as the steps mentioned in Comment 1 is not leading to the problem?

Thanks,
Shivaprasad
Comment 7 IBM Bug Proxy 2017-08-07 16:30:52 UTC 
------- Comment From lagarcia.com 2017-08-07 12:24 EDT-------
Hello Red Hat,

Would you be able to answer the question from previous comment?
Comment 8 Junxiang Li 2017-08-10 09:55:31 UTC 
Created attachment 1311647 [details]
auto script
Comment 9 Junxiang Li 2017-08-10 09:57:17 UTC 
Created attachment 1311649 [details]
dir_pool incorrect log
Comment 10 Junxiang Li 2017-08-10 09:58:33 UTC 
Created attachment 1311650 [details]
logical_pool incorrect log
Comment 11 Junxiang Li 2017-08-10 09:59:59 UTC 
(In reply to IBM Bug Proxy from comment #6)
> ------- Comment From shivapbh.com 2017-07-07 08:37 EDT-------
> On directory based pool after vol-download I don't see any difference
> between the original and downloaded volumes.
> 
> This is my host
> # cat /etc/os-release
> NAME="Red Hat Enterprise Linux Server"
> VERSION="7.4 (Pegas)"
> ID="rhel"
> ID_LIKE="fedora"
> VERSION_ID="7.4"
> PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Pegas)"
> ...
> Libvirt - libvirt-3.2.0-15.el7a.ppc64le
> 
> On Comment 5, you mention the pool type is logical. Are you saying the issue
> was observed on pool backed on LVM ? What is the correlation to Comment 1 &
> 5? Can you reiterate the steps to reproduce as the steps mentioned in
> Comment 1 is not leading to the problem?
> 
> Thanks,
> Shivaprasad

This is my host
# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4"

# rpm -q libvirt qemu-kvm-rhev
libvirt-3.2.0-14.virtcov.el7_4.2.ppc64le
qemu-kvm-rhev-2.9.0-16.el7_4.3.ppc64le

And I feel so sorry about the reproducible is not correct.
I reproduce it by script, and dir_pool reproducible is about 10%, logical_pool reproducible is about 30%.
Finally, I add 3 attachments.
1. auto1460623.py is the script
2. dir_pool.log is the dir_pool incorrect log
3. logical_pool.log is the logical_pool incorrect log
Comment 12 IBM Bug Proxy 2017-11-16 16:15:12 UTC 
------- Comment From niteshkonkar.com 2017-11-16 11:08 EDT-------
Hello,

I figuered it out that this is the commit that is causing the bug.

commit b7d44f450c06803df7df3ad380f7a5c97425c1e6
Author: John Ferlan <jferlan>
Date:   Fri Mar 24 09:26:17 2017 -0400

storage: Fix capacity value for LUKS encrypted volumes

https://bugzilla.redhat.com/show_bug.cgi?id=1371892

The 'capacity' value (e.g. guest logical size) for a LUKS volume is
smaller than the 'physical' value of the file in the file system, so
we need to account for that.

When peeking at the encryption information about the volume add a fetch
of the payload_offset which is described as the offset to the start of
the volume data (in 512 byte sectors) in QEMU's QCryptoBlockLUKSHeader.

Then adjust the ->capacity appropriately when we determine that the
volume target encryption has a payload_offset value.

The following series seems to fix the issue.

https://www.redhat.com/archives/libvir-list/2016-April/msg01869.html

Thanks,
Nitesh Konkar.
Comment 13 Andrea Bolognani 2017-11-16 16:44:47 UTC 
The sparse stream changes you mention have been merged a while ago.

Does the issue still reproduce with a recent libvirt?
Comment 14 IBM Bug Proxy 2017-11-17 07:20:30 UTC 
------- Comment From niteshkonkar.com 2017-11-17 02:10 EDT-------
Hi,

I do not see the issue in the latest upstream libvirt.

1. virsh --version
3.10.0

2. cat /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Maipo)"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:beta:server"

REDHAT_SUPPORT_PRODUCT_VERSION="7.4 Beta"

4.hexdump  /var/lib/libvirt/images/dir_pool/vol_dir_pool.img
22600000

5. hexdump /var/lib/libvirt/images/dir_pool/vol_test
22600000
Comment 15 Hanns-Joachim Uhl 2017-11-17 08:48:28 UTC 
oops ... the previous comment has to read:
"
Hi,

I do not see the issue in the latest upstream libvirt. 

1. virsh --version
3.10.0

2. cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 Beta (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:beta:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4 Beta"

3. virsh vol-download /var/lib/libvirt/images/dir_pool/vol_dir_pool.img /var/lib/libvirt/images/dir_pool/vol_test

4.hexdump  /var/lib/libvirt/images/dir_pool/vol_dir_pool.img 
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
22600000

5. hexdump /var/lib/libvirt/images/dir_pool/vol_test
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
22600000
"
...
Comment 16 Karen Noel 2017-11-30 01:14:38 UTC 
Andrea, Wenli, Can this one be closed?
Comment 17 Andrea Bolognani 2017-12-04 09:52:11 UTC 
Yup, moving it to POST now.
Comment 18 Junxiang Li 2018-01-25 09:47:34 UTC 
Reproduce:
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.4 (Maipo)
# rpm -q kernel libvirt qemu-kvm-rhev
kernel-3.10.0-693.17.1.el7.ppc64le
libvirt-3.2.0-14.el7_4.9.ppc64le
qemu-kvm-rhev-2.9.0-16.el7_4.14.ppc64le
Steps to Reproduce:
1. virsh pool-create storage.xml [as follows]
<pool type='dir'>
  <name>dir_pool</name>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images/dir_pool</path>
  </target>
</pool>

2. virsh vol-create dir_pool volume.xml [as follows]
<volume>
  <name>vol_dir_pool.img</name>
  <capacity unit="M">50</capacity>
  <allocation>0</allocation>
  <target>
    <path>/var/lib/libvirt/images/dir_pool/vol_dir_pool.img</path>
    <format type="raw"/>
  </target>
</volume>
3. Use the scripts(https://bugzilla.redhat.com/attachment.cgi?id=1311647)
4. The bug could be reproduce


Verify:
# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.5 Beta (Maipo)
# rpm -q qemu-kvm-rhev libvirt kernel
qemu-kvm-rhev-2.10.0-18.el7.ppc64le
libvirt-3.9.0-9.virtcov.el7.ppc64le
kernel-3.10.0-837.el7.ppc64le

Steps to Reproduce:
1. virsh pool-create storage.xml [as follows]
<pool type='dir'>
  <name>dir_pool</name>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images/dir_pool</path>
  </target>
</pool>

2. virsh vol-create dir_pool volume.xml [as follows]
<volume>
  <name>vol_dir_pool.img</name>
  <capacity unit="M">50</capacity>
  <allocation>0</allocation>
  <target>
    <path>/var/lib/libvirt/images/dir_pool/vol_dir_pool.img</path>
    <format type="raw"/>
  </target>
</volume>
3. Use the scripts(https://bugzilla.redhat.com/attachment.cgi?id=1311647)
4. Try 100000 times, and could not reproduce the bug.

According that, the bug could set to verified
Comment 20 Junxiang Li 2018-01-26 00:58:48 UTC 
According to comment 18, set to verified
Comment 26 errata-xmlrpc 2018-04-10 10:48:37 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704
Note You need to log in before you can comment on or make changes to this bug.