RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 997839 - GOS installation fails on a virtio-blk device(img file) created on a XFS filesystem with 4k sector size.
Summary: GOS installation fails on a virtio-blk device(img file) created on a XFS file...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm
Version: 6.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Kevin Wolf
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-08-16 09:34 UTC by Inbaraj
Modified: 2013-09-18 09:48 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-09-18 09:48:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
VM Anaconda installer logs (24.73 KB, application/x-gzip)
2013-08-19 11:34 UTC, Inbaraj
no flags Details
host messages,qemu logs,libvirt logs,failure screen shot (102.32 KB, application/x-gzip)
2013-08-19 11:48 UTC, Inbaraj
no flags Details
screenshot for guest installration error. (32.54 KB, image/png)
2013-08-23 08:33 UTC, Sibiao Luo
no flags Details

Description Inbaraj 2013-08-16 09:34:26 UTC
Description of problem:

While trying to install an Guest operating system (linux)  on a virtio blk device (disk image), installation failed with I/O error. The disk image is created from an DMMP device formatted with XFS file system.

Version-Release number of selected component:
OS-Hypervisor: RHEL 6.4 KVM
XFS version: xfsprogs-3.1.1-10.el6.x86_64
qemu-kvm-0.12.1.2-2.355.el6.x86_64
kernel: 2.6.32-358.6.1.el6.x86_64              

How reproducible:
Every time.

steps to reproduce the issue :
1) map a lun to  the hypervisor.
2) Discover the device and configure multipath.
2) create a XFS file system on the DM-MP device using mkfs.xfs command.
4) Mount the device  and create an image file for 30g on this mount point.
5) Using virt-manager create a new VM and assign the image file as the virtio blk device for installation. 

Expected result:

Partition creation should pass and guest installation should complete successfully.

Actual result:

Installation fails during partition creation with an i/o error.


Additional info:

Default XFS file system aligns the sector size to the physical block size of the device. So the above issue is seen , when a img file is created out of a XFS formatted device with sector size of 4096 .
Whereas the issue is not seen after tweaking the sector size  to 512 .

Issue seen with the default XFS file system values:
[root@ibmx3550-210-108 ~]# xfs_info /mnt/test2
meta-data=/dev/mapper/test_xfs-dmmp_lvm_img_part isize=256    agcount=8, agsize=163839 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=1310712, imaxpct=25
         =                       sunit=1      swidth=16 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Issue is not seen on XFS file system with sector size 512..
[root@ibmx3550-210-108 ~]# xfs_info /mnt/test1
meta-data=/dev/mapper/360a98000324669384a2443744a78474f isize=256    agcount=16, agsize=491519 blks
         =                       sectsz=512   attr=2, projid32bit=0
data     =                       bsize=4096   blocks=7864304, imaxpct=25
         =                       sunit=1      swidth=16 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=3840, version=2
         =                       sectsz=512   sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Comment 1 Inbaraj 2013-08-19 11:34:49 UTC
Created attachment 788013 [details]
VM Anaconda installer logs

Comment 2 Inbaraj 2013-08-19 11:48:16 UTC
Created attachment 788018 [details]
host messages,qemu logs,libvirt logs,failure screen shot

Comment 4 Sibiao Luo 2013-08-23 08:32:18 UTC
Reproduce this issue on latest qemu-kvm and kernel version with sectsz=4096, btw, sectsz=512 did not met this issue.

Host info:
2.6.32-412.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.397.el6.x86_64
xfsprogs-3.1.1-10.el6_4.1.x86_64
Guest info:
rhel6.4GA x84_64bit

How reproducible:
100%

steps to reproduce the issue:
1) map a lun to  the hypervisor.
# ls -l /dev/disk/by-path/ip-10.66.90.100\:3260-iscsi-iqn.2001-05.com.equallogic\:0-8a0906-0971f7d03-1dff49b26885073d-s2-sluo-172259-lun-0 
lrwxrwxrwx. 1 root root 9 Aug 22 03:27 /dev/disk/by-path/ip-10.66.90.100:3260-iscsi-iqn.2001-05.com.equallogic:0-8a0906-0971f7d03-1dff49b26885073d-s2-sluo-172259-lun-0 -> ../../sdb
2) Discover the device and configure multipath.
# yum install device-mapper.x86_64 device-mapper-multipath.x86_64
# modprobe dm-multipath
# modprobe dm-round-robin
# lsmod | grep multipath
dm_multipath           17711  2 dm_round_robin
dm_mod                 82743  14 dm_multipath,dm_mirror,dm_log
 
# service multipathd start
Starting multipathd daemon:                                [  OK  ]
# chkconfig multipathd on  //ensure sure that the multipath daemon starts on bootup
# /sbin/mpathconf --enable
# ls -l /etc/multipath.conf
-rw-------. 1 root root 2754 Aug 23 01:34 /etc/multipath.conf
# cat /etc/multipath.conf
blacklist {
	wwid 1ATA_ST3500413AS_S2A29GR7
}
defaults {
user_friendly_names no
path_grouping_policy multibus
getuid_callout "/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/%n"
}
multipaths {
    multipath {
    wwid 36090a038d0f771093d078568b249ff1d //wwid of /dev/sdb
    alias multipath-disk
    path_grouping_policy multibus
    path_selector "round-robin 0"
    }
}
# service multipathd reload
Reloading multipathd:                                      [  OK  ]
# fdisk -l
...
Disk /dev/sdb: 107.4 GB, 107379425280 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/multipath-disk: 107.4 GB, 107379425280 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
2) create a XFS file system with sectsz=4096 on the DM-MP device using mkfs.xfs command.
# mkfs.xfs -s size=4096 /dev/mapper/multipath-disk -f
meta-data=/dev/mapper/multipath-disk isize=256    agcount=4, agsize=6553920 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=26215680, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=12800, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
4) Mount the device  and create an image file for 30g on this mount point.
# mount /dev/mapper/multipath-disk /mnt/
# mount
...
/dev/mapper/multipath-disk on /mnt type xfs (rw)
# qemu-img create -f raw /mnt/bug997839testing.raw 30G
Formatting '/mnt/bug997839testing.raw', fmt=raw size=32212254720 
# xfs_info /mnt/bug997839testing.raw
meta-data=/dev/mapper/multipath-disk isize=256    agcount=4, agsize=6553920 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=26215680, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12800, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
5) Using qemu-kvm create a new VM and assign the image file as the virtio blk device for installation. 
# /usr/libexec/qemu-kvm -M pc -cpu SandyBridge -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 316a0cb5-6ea7-4920-902c-afc06184b292 -rtc base=localtime,clock=host,driftfix=slew -drive file=/mnt/bug997839testing.raw,if=none,id=drive-ide-disk,format=raw,cache=none,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x3,drive=drive-ide-disk,id=ide-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,id=e1000-net-pci0,mac=2C:41:38:B6:40:11,bus=pci.0,addr=0x5 -k en-us -boot menu=on -qmp tcp:0:5555,server,nowait -spice disable-ticketing,port=5931 -monitor stdio -drive file=/home/RHEL6.4-20130130.0-Server-x86_64-DVD1.iso,if=none,id=drive-cd-disk,format=raw,cache=none,media=cdrom -device ide-drive,bus=ide.0,unit=1,drive=drive-cd-disk,id=cd-disk,bootindex=0

Results:
after step 5, qemu will prompt "block I/O error in device 'drive-ide-disk': Invalid argument (22)" in HMP monitor.
(qemu) main_channel_link: add main channel client
main_channel_handle_parsed: net test: latency 45.702000 ms, bitrate 6564102564 bps (6260.016026 Mbps)
inputs_connect: inputs channel client create
red_dispatcher_set_cursor_peer: 
block I/O error in device 'drive-ide-disk': Invalid argument (22)
block I/O error in device 'drive-ide-disk': Invalid argument (22)
block I/O error in device 'drive-ide-disk': Invalid argument (22)

And guest also prompt some error and cann't continue install, i will attach the screenshot later.


Base on above, mark qa_ack+ to it, Please correct me if any mistake.

Best Regards,
sluo

Comment 5 Sibiao Luo 2013-08-23 08:33:50 UTC
Created attachment 789500 [details]
screenshot for guest installration error.

Comment 7 Kevin Wolf 2013-09-18 09:48:15 UTC
You need to use the option logical_block_size=4096 for the virtio-blk-pci device,
which will present a 4k sectors disk to the guest. However, this will mean that
you can't boot from the disk but only use it as a data disk.

The other options is to use a cache mode that doesn't involve O_DIRECT, for
example cache=writeback. In this case, qemu can write even blocks smaller than
the XFS sector size.


Note You need to log in before you can comment on or make changes to this bug.