RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1412183 - VM fails to access disk after migration
Summary: VM fails to access disk after migration
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Virtualization Maintenance
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-11 13:22 UTC by Andrei Stepanov
Modified: 2017-01-11 13:42 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-11 13:42:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Andrei Stepanov 2017-01-11 13:22:33 UTC 
After local TCP migration VM cannot work with its disk.

Host info:
kernel: 3.10.0-514.el7.x86_64
qemu-kvm-rhev-2.6.0-28.el7_3.3.x86_64


* Wed Jan 04 2017 Miroslav Rezanina <mrezanin> - rhev-2.6.0-28.el7_3.3
- kvm-pc_piix-fix-compat-props-typo-for-RHEL6-machine-type.patch [bz#1408122]
- kvm-net-don-t-poke-at-chardev-internal-QemuOpts.patch [bz#1410200]
- Resolves: bz#1408122
  (Opteron_G4 CPU model broken in QEMU 2.6 with RHEL 6 machine type)
- Resolves: bz#1410200
  (qemu gets SIGSEGV when hot-plug a vhostuser network)



How reproducible: always


Steps to Reproduce:
0. Disable firewall (if you have).
1. Have a qcow2 image with OS. (I have RHEL6.9 32bit)

2. Create first QEMU instance (Source) to migrate from:

export SRCPORT=3001
export GUEST_IMAGE_PATH=/local_storage/rhel6932.img

/usr/libexec/qemu-kvm -m 1024 -spice port=$SRCPORT,disable-ticketing,seamless-migration=on -vga qxl -global qxl-vga.vram_size=67108864 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x8 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 $GUEST_IMAGE_PATH -monitor stdio

3. Create secondary QEMU instance (Destination) to migrate to:

export DSTPORT=3001
export GUEST_IMAGE_PATH=/local_storage/rhel6932.img
export DSTMIGPORT=4444
export DSTIPADDR=0.0.0.0

/usr/libexec/qemu-kvm -m 1024 -spice port=$DSTPORT,disable-ticketing,seamless-migration=on -vga qxl -global qxl-vga.vram_size=67108864 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x8 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 $GUEST_IMAGE_PATH -monitor stdio -incoming tcp:$DSTIPADDR:$DSTMIGPORT

4. Connect with remote-viewer to VM (first QEMU instance):

remote-viewer spice://<IP>:3001

5. Migrate VM to the second VM instance:

(qemu) client_migrate_info spice <IP> 3002
(qemu) main_channel_client_handle_migrate_connected: client 0x7fa82fa71a80 connected: 1 seamless 1
(qemu) migrate -d tcp:127.0.0.1:4444
(qemu) red_client_migrate: migrate client with #channels 4
red_dispatcher_cursor_migrate: channel type 4 id 0
red_dispatcher_display_migrate: channel type 2 id 0
red_channel_client_disconnect: rcc=0x7fa8329fc000 (channel=0x7fa82fb42000 type=3 id=0)
red_channel_client_disconnect: rcc=0x7fa832466000 (channel=0x7fa8305e6600 type=2 id=0)
red_channel_client_disconnect: rcc=0x7fa8305f7000 (channel=0x7fa832006000 type=4 id=0)
red_channel_client_disconnect: rcc=0x7fa8305f2000 (channel=0x7fa82fb3a000 type=1 id=0)
main_channel_client_on_disconnect: rcc=0x7fa8305f2000
red_client_destroy: destroy client 0x7fa82fa71a80 with #channels=4
red_dispatcher_disconnect_cursor_peer: 
red_dispatcher_disconnect_display_peer: 


After this step VM fails to access its disk:

Log from VM:

ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
ata1.00: BMDMA stat 0x5
ata1.00: failed command: WRITE DMA
ata1.00: cmd ca/00:38:b0:1e:c4/00:00:00:00:00/e1 tag 0 dma 28672 out
         res 00/00:00:00:00:00/00:00:00:00:00/00 Emask 0x2 (HSM violation)
ata1: soft resetting link
ata1.00: revalidation failed (errno=-2)
ata1: soft resetting link
ata1.00: revalidation failed (errno=-2)
ata1: soft resetting link
ata1.00: revalidation failed (errno=-2)
ata1.00: disabled
ata1: soft resetting link
ata1: EH complete
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 01 c4 1e b0 00 00 38 00
end_request: I/O error, dev sda, sector 29630128
Aborting journal on device sda1-8.
EXT4-fs (sda1): delayed block allocation failed for inode 278698 at logical offset 0 with max blocks 8 with error -30
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 01 c4 08 00 00 00 08 00
end_request: I/O error, dev sda, sector 29624320
Buffer I/O error on device sda1, logical block 3702784
lost page write due to I/O error on sda1

This should not happen!!  Data will be lost
EXT4-fs error (device sda1) in ext4_reserve_inode_write: Journal has aborted
JBD2: I/O error detected when updating journal superblock for sda1-8.
EXT4-fs error (device sda1): ext4_journal_start_sb: Detected aborted journal
EXT4-fs (sda1): Remounting filesystem read-only
EXT4-fs (sda1): ext4_da_writepages: jbd2_start: 1024 pages, ino 278698; err -30

sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 00 08 00 00 00 08 00
end_request: I/O error, dev sda, sector 2048
Buffer I/O error on device sda1, logical block 0
lost page write due to I/O error on sda1
EXT4-fs warning (device sda1): ext4_delete_inode: couldn't mark inode dirty (err -30)
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 01 51 08 00 00 08 00
end_request: I/O error, dev sda, sector 86280
Buffer I/O error on device sda1, logical block 10529
lost page write due to I/O error on sda1
JBD2: Detected IO errors while flushing file data on sda1-8
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 44 63 d0 00 00 08 00
end_request: I/O error, dev sda, sector 21259216
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 44 63 d0 00 00 08 00
end_request: I/O error, dev sda, sector 21259216
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 1a 4c 40 00 00 10 00
end_request: I/O error, dev sda, sector 52055104
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 1a 4c 40 00 00 08 00
end_request: I/O error, dev sda, sector 52055104
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 1a 4c 40 00 00 08 00
end_request: I/O error, dev sda, sector 52055104
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 44 63 d0 00 00 08 00
end_request: I/O error, dev sda, sector 21259216
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 1a 4c 50 00 00 18 00
end_request: I/O error, dev sda, sector 52055120
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 1a 4c 50 00 00 08 00
end_request: I/O error, dev sda, sector 52055120
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 47 2a 98 00 00 08 00
end_request: I/O error, dev sda, sector 21441176
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 47 2a 98 00 00 08 00
end_request: I/O error, dev sda, sector 21441176
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 07 cc 00 00 00 20 00
end_request: I/O error, dev sda, sector 50842624
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 07 cc 00 00 00 08 00
end_request: I/O error, dev sda, sector 50842624
Core dump to |/usr/libexec/abrt-hook-ccpp 6 0 2339 0 0 1484136191 e pipe failed
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 44 63 d0 00 00 08 00
end_request: I/O error, dev sda, sector 21259216
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 01 47 2a 98 00 00 08 00
end_request: I/O error, dev sda, sector 21441176
sd 0:0:0:0: [sda]  Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
sd 0:0:0:0: [sda] CDB: Read(10): 28 00 03 07 cc 00 00 00 08 00
end_request: I/O error, dev sda, sector 50842624
Core dump to |/usr/libexec/abrt-hook-ccpp 6 0 2575 0 0 1484136780 e pipe failed

VM is broken and doesn't work as expected.
Comment 1 Andrei Stepanov 2017-01-11 13:42:37 UTC 
Okay, it was my mistake. Secondary QEMU instance had bad cmdline.
Note You need to log in before you can comment on or make changes to this bug.