Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 658744

Summary:

Formatting disk on nfs caused guest paused

Product:

Red Hat Enterprise Linux 6

Reporter:

Amos Kong <akong>

Component:

kernel

Assignee:

Jeff Layton <jlayton>

Status:

CLOSED DUPLICATE

QA Contact:

Red Hat Kernel QE team <kernel-qe>

Severity:

medium

Docs Contact:

Priority:

high

Version:

6.1

CC:

ailan, bfields, dhowells, jlayton, kwolf, mkenneth, riek, rwheeler, steved, virt-maint

Target Milestone:

Keywords:

Regression

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2010-12-02 12:40:53 UTC

Type:

---

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

580954

Attachments:

Description	Flags
strace output	none

Description Amos Kong 2010-12-01 06:09:42 UTC

Description of problem:

I tried to format a disk, which is on nfs server, format didn't complete and guest paused.

===> qemu output:
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)
handle_dev_input: stop
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)
handle_dev_input: start
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)
handle_dev_input: stop
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)
block I/O error in device 'drive-virtio-disk2': Cannot allocate memory (12)

Version-Release number of selected component (if applicable):
host kernel: 2.6.32-84.el6.x86_64
guest kernel: 2.6.32-71.7.1.el6.i686
# rpm -qa |grep qemu
qemu-kvm-debuginfo-0.12.1.2-2.120.el6.x86_64
gpxe-roms-qemu-0.9.7-6.3.el6.noarch
qemu-kvm-0.12.1.2-2.120.el6.x86_64
qemu-kvm-tools-0.12.1.2-2.120.el6.x86_64
qemu-img-0.12.1.2-2.120.el6.x86_64


How reproducible:
always

Steps to Reproduce:
1. setup nfs on localhost
# mkdir -p /tmp/nfs_dir && mkdir -p images/nfs && service nfs restart && exportfs 127.0.0.1:/tmp/nfs_dir -o rw,no_root_squash && mount 127.0.0.1:/tmp/nfs_dir images/nfs -o rw,soft,timeo=1,retrans=1,vers=3
2. create a image on nfs dir
# qemu-img create -f qcow2 -opreallocation=metadata -ocluster_size=2M /root/autotest-devel/client/tests/kvm/images/nfs/storage.qcow2 10G
3. boot up guest with two images (one on localhost, one on nfs), werror=on
4. login guest and format second disk
guest) # mkfs vdb


Actual results:
format didn't complete, guest paused, qemu process outputted some error msg.

Expected results:
success to format disk on nfs

Additional info:

1. qemu commandline:
# qemu-kvm -name vm1 -chardev socket,id=human_monitor_fVCa,path=/tmp/monitor-humanmonitor1-20101201-125251-YB6k,server,nowait -mon chardev=human_monitor_fVCa,mode=readline -chardev socket,id=serial_wykW,path=/tmp/serial-20101201-125251-YB6k,server,nowait -device isa-serial,chardev=serial_wykW -drive file=/root/autotest-devel/client/tests/kvm/images/RHEL-Server-6.0-32-virtio.qcow2,index=0,if=none,id=drive-virtio-disk1,media=disk,cache=none,werror=stop,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,id=virtio-disk1 -drive file=/root/autotest-devel/client/tests/kvm/images/nfs/storage.qcow2,if=none,id=drive-virtio-disk2,media=disk,cache=none,werror=stop,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk2,id=virtio-disk2 -device virtio-net-pci,mac=9a:65:71:99:93:3a,netdev=idQpJUNB,id=ndev00idQpJUNB,bus=pci.0,addr=0x3 -netdev tap,id=idQpJUNB,vhost=on,ifname=t0-125251-YB6k,script=/root/autotest-devel/client/tests/kvm/scripts/qemu-ifup-switch,downscript=no,vhost=on -m 2048 -smp 2,cores=1,threads=1,sockets=2 -cpu cpu64-rhel6,+sse2,+x2apic -vnc :0 -spice port=8000,disable-ticketing -vga qxl -rtc base=utc,clock=host,driftfix=none -M rhel6.0.0 -usbdevice tablet -no-kvm-pit-reinjection -enable-kvm

Comment 4 chellwig@redhat.com 2010-12-01 13:42:06 UTC

Please try aio=threads instead of aio=native and attach the strace -f output of the qemu binary, to figure out where the ENOMEM comes from.  I'm pretty sure it's the kernel and not qemu, but the strace output is required to verify that theory.

Comment 5 Amos Kong 2010-12-02 06:04:05 UTC

Created attachment 464169 [details]
strace output

.......
2550  pread(13, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 8355840) = 16384
2550  pread(13, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 8372224) = 16384
2550  pwrite(13, "\0\1\0\1\0\1\0\1\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512, 6291456) = -1 ENOMEM (Cannot allocate memory)
2550  mprotect(0x7f8cec261000, 987136, PROT_READ|PROT_WRITE) = 0
2565  <... epoll_wait resumed> {}, 10, 10) = 0
2565  epoll_wait(22,  <unfinished ...>
2550  pwrite(13, "\0\1\0\1\0\1\0\1\0\1\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512, 6291456) = -1 ENOMEM (Cannot allocate memory)
2550  madvise(0x7f8cec275000, 905216, MADV_DONTNEED) = 0
2550  write(2, "block I/O error in device 'drive"..., 76) = 76
2550  ioctl(10, 0x4008af30, 0x7f8cf34934b0) = 0
2550  ioctl(10, 0x4008af30, 0x7f8cf34934b0) = 0
.......

Comment 6 Jeff Layton 2010-12-02 12:40:53 UTC

This is almost certainly the problem fixed by this patch:

    http://post-office.corp.redhat.com/archives/rhkernel-list/2010-November/msg00712.html

I'm going to close this as a duplicate of that bug. Please reopen if that patchset doesn't fix this.

*** This bug has been marked as a duplicate of bug 653066 ***