Bug 1039791

Summary: qemu-img creates truncated VMDK image with subformat=twoGbMaxExtentFlat
Product: Red Hat Enterprise Linux 7 Reporter: Fam Zheng <famz>
Component: qemu-kvmAssignee: Fam Zheng <famz>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0CC: juzhang, knoel, rbalakri, sluo, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-1.5.3-67.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1088788 (view as bug list) Environment:
Last Closed: 2015-03-05 08:02:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1088788    

Description Fam Zheng 2013-12-10 02:05:24 UTC
Description of problem:
The command:
qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 1000G

will create a test.vmdk with truncated content, because of vmdk driver has a fixed size buffer.

Expected results:
A correct image should be created.

=====================================
$ cat test.vmdk 
# Disk DescriptorFile
version=1
CID=52a672ff
parentCID=ffffffff
createType="twoGbMaxExtentFlat"

# Extent description
RW 4194304 FLAT "test-f001.vmdk" 0
RW 4194304 FLAT "test-f002.vmdk" 0
RW 4194304 FLAT "test-f003.vmdk" 0
RW 4194304 FLAT "test-f004.vmdk" 0
RW 4194304 FLAT "test-f005.vmdk" 0
RW 4194304 FLAT "test-f006.vmdk" 0
RW 4194304 FLAT "test-f007.vmdk" 0
RW 4194304 FLAT "test-f008.vmdk" 0
RW 4194304 FLAT "test-f009.vmdk" 0
RW 4194304 FLAT "test-f010.vmdk" 0
RW 4194304 FLAT "test-f011.vmdk" 0
RW 4194304 FLAT "test-f012.vmdk" 0
RW 4194304 FLAT "test-f013.vmdk" 0
RW 4194304 FLAT "test-f014.vmdk" 0
RW 4194304 FLAT "test-f015.vmdk" 0
RW 4194304 FLAT "test-f016.vmdk" 0
RW 4194304 FLAT "test-f017.vmdk" 0
RW 4194304 FLAT "test-f018.vmdk" 0
RW 4194304 FLAT "test-f019.vmdk" 0
RW 4194304 FLAT "test-f020.vmdk" 0
RW 4194304 FLAT "test-f021.vmdk" 0
RW 4194304 FLAT "test-f022.vmdk" 0
RW 4194304 FLAT "test-f023.vmdk" 0
RW 4194304 FLAT "test-f024.vmdk" 0
RW 4194304 FLAT "test-f025.vmdk" 0
RW 4194304 FLAT "test-f026.vmdk" 0
RW 4194304 FLAT "test-f027.vmdk" 0
RW 4194304 FLAT "test-f028.vmdk" 0
RW 4194304 FLAT "test-f029.vmdk" 0
RW 4194304 FLAT "test-f030.vmdk" 0
RW 4194304 FLAT "test-f031.vmdk" 0
RW 4194304 FLAT "test-f032.vmdk" 0
RW 4194304 FLAT "test-f033.vmdk" 0
RW 4194304 FLAT "test-f034.vmdk" 0
RW 4194304 FLAT "test-f035.vmdk" 0
RW 4194304 FLAT "test-f036.vmdk" 0
RW 4194304 FLAT "test-f037.vmdk" 0
RW 4194304 FLAT "test-f038.vmdk" 0
RW 4194304 FLAT "test-f039.vmdk" 0
RW 4194304 FLAT "test-f040.vmdk" 0
RW 4194304 FLAT "test-f041.vmdk" 0
RW 4194304 FLAT "test-f042.vmdk" 0
RW 4194304 FLAT "test-f043.vmdk" 0
RW 4194304 FLAT "test-f044.vmdk" 0
RW 4194304 FLAT "test-f045.vmdk" 0
RW 4194304 FLAT "test-f046.vmdk" 0
RW 4194304 FLAT "test-f047.vmdk" 0
RW 4194304 FLAT "test-f048.vmdk" 0
RW 4194304 FLAT "test-f049.vmdk" 0
RW 4194304 FLAT "test-f050.vmdk" 0
RW 4194304 FLAT "test-f051.vmdk" 0
RW 4194304 FLAT "test-f052.vmdk" 0
RW 4194304 FLAT "test-f053.vmdk" 0
RW 4194304 FLAT "test-f054.vmdk" 0
RW 4194304 FLAT "test-f055.vmdk" 0
RW 4194304 FLAT "test-f056.vmdk" 0
RW 4194304 FLAT "test-f057.vmdk" 0
RW 4194304 FLAT "test-f058.vmdk" 0
RW 4194304 FLAT "test-f059.vmdk" 0
RW 4194304 FLAT "test-f060.vmdk" 0
RW 4194304 FLAT "test-f061.vmdk" 0
RW 4194304 FLAT "test-f062.vmdk" 0
RW 4194304 FLAT "test-f063.vmdk" 0
RW 4194304 FLAT "test-f064.vmdk" 0
RW 4194304 FLAT "test-f065.vmdk" 0
RW 4194304 FLAT "test-f066.vmdk" 0
RW 4194304 FLAT "test-f067.vmdk" 0
RW 4194304 FLAT "test-f068.vmdk" 0
RW 4194304 FLAT "test-f069.vmdk" 0
RW 4194304 FLAT "test-f070.vmdk" 0
RW 4194304 FLAT "test-f071.vmdk" 0
RW 4194304 FLAT "test-f072.vmdk" 0
RW 4194304 FLAT "test-f073.vmdk" 0
RW 4194304 FLAT "test-f074.vmdk" 0
RW 4194304 FLAT "test-f075.vmdk" 0
RW 4194304 FLAT "test-f076.vmdk" 0
RW 4194304 FLAT "test-f077.vmdk" 0
RW 4194304 FLAT "test-f078.vmdk" 0
RW 4194304 FLAT "test-f079.vmdk" 0
RW 4194304 FLAT "test-f080.vmdk" 0
RW 4194304 FLAT "test-f081.vmdk" 0
RW 4194304 FLAT "test-f082.vmdk" 0
RW 4194304 FLAT "test-f083.vmdk" 0
RW 4194304 FLAT "test-f084.vmdk" 0
RW 4194304 FLAT "test-f085.vmdk" 0
RW 4194304 FLAT "test-f086.vmdk" 0
RW 4194304 FLAT "test-f087.vmdk" 0
RW 4194304 FLAT "test-f088.vmdk" 0
RW 4194304 FLAT "test-f089.vmdk" 0
RW 4194304 FLAT "test-f090.vmdk" 0
RW 4194304 FLAT "test-f091.vmdk" 0
RW 4194304 FLAT "test-f092.vmdk" 0
RW 4194304 FLAT "test-f093.vmdk" 0
RW 4194304 FLAT "test-f094.vmdk" 0
RW 4194304 FLAT "test-f095.vmdk" 0
RW 4194304 FLAT "test-f096.vmdk" 0
RW 4194304 FLAT "test-f097.vmdk" 0
RW 4194304 FLAT "test-f098.vmdk" 0
RW 4194304 FLAT "test-f099.vmdk" 0
RW 4194304 FLAT "test-f100.vmdk" 0
RW 4194304 FLAT "test-f101.vmdk" 0
RW 4194304 FLAT "test-f102.vmdk" 0
RW 4194304 FLAT "test-f103.vmdk" 0
RW 4194304 FLAT "test-f104.vmdk" 0
RW 4194304 FLAT "test-f105.vmdk" 0
RW 4194304 FLAT "test-f106.vmdk" 0
RW 4194304 FLAT "test-f107.vmdk" 0
RW 4194304 FLAT "test-f108.vmdk" 0
RW 4194304 FLAT "test-f109.vmdk" 0
RW 4194304 FLAT "test-f110.vmdk" 0
RW 4194304 FLAT "test-f111.vmdk" 0
RW 4194304 FLAT "test-f112.vmdk" 0
RW 4194304 FLAT "test-f113.vmdk" 0
RW 4194304 FLAT "test-                        <- From here the description file is truncated

Comment 1 Fam Zheng 2014-04-18 08:04:04 UTC
Fix posted to upstream and merged for v2.0.0.

Moving to POST and waiting for rebase.

Thanks,
Fam

Comment 5 Fam Zheng 2014-07-31 06:30:28 UTC
Backporting upstream fix for both subformat:

commit af057fe74092df2e7a576448ddbdc0daac1370bf
Author: Fam Zheng <famz>
Date:   Tue Dec 3 10:41:05 2013 +0800

    vmdk: Fix creating big description file

    The buffer for description file was 4096 which only covers a few
    hundred of extents. This changes the buffer to dynamic allocated with
    g_strdup_printf in order to support bigger cases.

    Signed-off-by: Fam Zheng <famz>
    Signed-off-by: Stefan Hajnoczi <stefanha>

Comment 6 Jeff Nelson 2014-08-08 16:55:13 UTC
Fix included in qemu-kvm-1.5.3-67.el7

Comment 8 Sibiao Luo 2014-08-28 07:37:24 UTC
Reproduce this issue on qemu-kvm-1.5.3-66.el7.x86_64 with twoGbMaxExtentSparse and twoGbMaxExtentFlat subformat.

host info:
# uname -r && rpm -q qemu-kvm
3.10.0-145.el7.x86_64
qemu-kvm-1.5.3-66.el7.x86_64

# qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off subformat='twoGbMaxExtentFlat' zeroed_grain=off 

[root@dhcp-11-154 home]# cat test.vmdk 
# Disk DescriptorFile
version=1
CID=53fed956
parentCID=ffffffff
createType="twoGbMaxExtentFlat"

# Extent description
RW 4194304 FLAT "test-f001.vmdk" 0
RW 4194304 FLAT "test-f002.vmdk" 0
RW 4194304 FLAT "test-f003.vmdk" 0
......
RW 4194304 FLAT "test-f109.vmdk" 0
RW 4194304 FLAT "test-f110.vmdk" 0
RW 4194304 FLAT "test-f111.vmdk" 0
RW 4194304 FLAT "test-f112.vmdk" 0
RW 4194304 FLAT "test-f113.vmdk" 0
RW 4194304 FLAT "test-[root@dhcp-11-154 home]#

# qemu-img create -f vmdk -o subformat=twoGbMaxExtentSparse test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off subformat='twoGbMaxExtentSparse' zeroed_grain=off 

[root@dhcp-11-154 home]# cat test.vmdk 
# Disk DescriptorFile
version=1
CID=53fed9e0
parentCID=ffffffff
createType="twoGbMaxExtentSparse"

# Extent description
RW 4194304 SPARSE "test-s001.vmdk"
RW 4194304 SPARSE "test-s002.vmdk"
RW 4194304 SPARSE "test-s003.vmdk"
RW 4194304 SPARSE "test-s004.vmdk"
RW 4194304 SPARSE "test-s005.vmdk"
......
RW 4194304 SPARSE "test-s108.vmdk"
RW 4194304 SPARSE "test-s109.vmdk"
RW 4194304 SPARSE "test-s110.vmdk"
RW 4194304 SPARSE "test-s111.vmdk"
RW 4194304 SPARSE "test-s112.vmdk"
RW 4194304 SPARSE "test-s113.vmdk"
RW 4194304 SPARSE "t[root@dhcp-11-154 home]#

###########################################

Verify this issue on qemu-kvm-1.5.3-67.el7.x86_64 with twoGbMaxExtentSparse and twoGbMaxExtentFlat subformat.

host info:
# uname -r && rpm -q qemu-kvm
3.10.0-145.el7.x86_64
qemu-kvm-1.5.3-67.el7.x86_64

# qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off subformat='twoGbMaxExtentFlat' zeroed_grain=off 

# cat test.vmdk 
# Disk DescriptorFile
version=1
CID=53feda9f
parentCID=ffffffff
createType="twoGbMaxExtentFlat"

# Extent description
RW 4194304 FLAT "test-f001.vmdk" 0
RW 4194304 FLAT "test-f002.vmdk" 0
RW 4194304 FLAT "test-f003.vmdk" 0
RW 4194304 FLAT "test-f004.vmdk" 0
RW 4194304 FLAT "test-f005.vmdk" 0
......
RW 4194304 FLAT "test-f508.vmdk" 0
RW 4194304 FLAT "test-f509.vmdk" 0
RW 4194304 FLAT "test-f510.vmdk" 0
RW 4194304 FLAT "test-f511.vmdk" 0
RW 4194304 FLAT "test-f512.vmdk" 0

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.geometry.cylinders = "2130440"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "ide"

# qemu-img create -f vmdk -o subformat=twoGbMaxExtentSparse test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off subformat='twoGbMaxExtentSparse' zeroed_grain=off 

# cat test.vmdk 
# Disk DescriptorFile
version=1
CID=53fedae9
parentCID=ffffffff
createType="twoGbMaxExtentSparse"

# Extent description
RW 4194304 SPARSE "test-s001.vmdk"
RW 4194304 SPARSE "test-s002.vmdk"
RW 4194304 SPARSE "test-s003.vmdk"
RW 4194304 SPARSE "test-s004.vmdk"
RW 4194304 SPARSE "test-s005.vmdk"
RW 4194304 SPARSE "test-s006.vmdk"
......
RW 4194304 SPARSE "test-s508.vmdk"
RW 4194304 SPARSE "test-s509.vmdk"
RW 4194304 SPARSE "test-s510.vmdk"
RW 4194304 SPARSE "test-s511.vmdk"
RW 4194304 SPARSE "test-s512.vmdk"

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.geometry.cylinders = "2130440"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "ide"

Base on above, this issue has been fixed correctly, move to VERIFIED status.

Best Regards,
sluo

Comment 9 Sibiao Luo 2014-09-01 08:16:16 UTC
Comment #8 was tried on Intel host.

Now i tried the qemu-kvm-rhev on AMD host to verify it.
host info:
# uname -r && rpm -q qemu-kvm-rhev
3.10.0-147.el7.x86_64
qemu-kvm-rhev-2.1.0-2.el7.x86_64

[root@hp-dl385g7-01 home]# qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off subformat='twoGbMaxExtentFlat' 
[root@hp-dl385g7-01 home]# cat test.vmdk 
# Disk DescriptorFile
version=1
CID=54042a78
parentCID=ffffffff
createType="twoGbMaxExtentFlat"

# Extent description
RW 4194304 FLAT "test-f001.vmdk" 0
RW 4194304 FLAT "test-f002.vmdk" 0
RW 4194304 FLAT "test-f003.vmdk" 0
RW 4194304 FLAT "test-f004.vmdk" 0
......
RW 4194304 FLAT "test-f508.vmdk" 0
RW 4194304 FLAT "test-f509.vmdk" 0
RW 4194304 FLAT "test-f510.vmdk" 0
RW 4194304 FLAT "test-f511.vmdk" 0
RW 4194304 FLAT "test-f512.vmdk" 0

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.geometry.cylinders = "2130440"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "ide"


[root@hp-dl385g7-01 home]# qemu-img create -f vmdk -o subformat=twoGbMaxExtentSparse test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off subformat='twoGbMaxExtentSparse' 
[root@hp-dl385g7-01 home]# cat test.vmdk 
# Disk DescriptorFile
version=1
CID=54042af5
parentCID=ffffffff
createType="twoGbMaxExtentSparse"

# Extent description
RW 4194304 SPARSE "test-s001.vmdk"
RW 4194304 SPARSE "test-s002.vmdk"
RW 4194304 SPARSE "test-s003.vmdk"
RW 4194304 SPARSE "test-s004.vmdk"
......
RW 4194304 SPARSE "test-s508.vmdk"
RW 4194304 SPARSE "test-s509.vmdk"
RW 4194304 SPARSE "test-s510.vmdk"
RW 4194304 SPARSE "test-s511.vmdk"
RW 4194304 SPARSE "test-s512.vmdk"

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.geometry.cylinders = "2130440"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "ide"

[root@hp-dl385g7-01 home]# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                16
On-line CPU(s) list:   0-15
Thread(s) per core:    1
Core(s) per socket:    8
Socket(s):             2
NUMA node(s):          3
Vendor ID:             AuthenticAMD
CPU family:            16
Model:                 9
Model name:            AMD Opteron(tm) Processor 6128
Stepping:              1
CPU MHz:               2000.081
BogoMIPS:              4000.38
Virtualization:        AMD-V
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              5118K
NUMA node0 CPU(s):     0,2,4,6
NUMA node1 CPU(s):     8-15
NUMA node3 CPU(s):     1,3,5,7

Best Regards,
sluo

Comment 10 Sibiao Luo 2014-09-01 08:16:42 UTC
According to comment #8 and comment #9, this issue has been fixed correctly, move to VERIFIED status.

Best Regards,
sluo

Comment 12 errata-xmlrpc 2015-03-05 08:02:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0349.html