Bug 1316870 - Migration during system_reset failed to bootup file system
Migration during system_reset failed to bootup file system
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kernel-rt (Show other bugs)
7.3
x86_64 Linux
unspecified Severity high
: rc
: ---
Assigned To: Peter Xu
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-11 05:52 EST by xiywang
Modified: 2016-03-22 06:13 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-22 06:13:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description xiywang 2016-03-11 05:52:33 EST
Description of problem:
Do migration during system_reset in hmp, guest failed to bootup file system and get imcomplete call trace from console.


Version-Release number of selected component (if applicable):
host & guest kernel:
3.10.0-355.rt56.234.el7.x86_64
host qemu-kvm:
qemu-kvm-rhev-2.5.0-2.el7.x86_64


How reproducible:
60%

Steps to Reproduce:
1. bootup guest in src host
/usr/libexec/qemu-kvm -name rhel7.2-rt-355 -S -machine pc-i440fx-rhel7.2.0 -cpu IvyBridge \

-m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 \

-drive file=/home/rhel7.2-rt-355.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,snapshot=off -device virtio-blk-pci,drive=drive-virtio-disk0,id=virtio-disk0 \

-netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:a1:d0:5f \

-monitor stdio -device qxl-vga,id=video0 -serial unix:/tmp/console,server,nowait -vnc :1 -spice port=5900,disable-ticketing

2. run commands in src host hmp
  1). c
  2). migrate_set_speed 2G
  3). migrate_set_capability xbzrle on

3. boot guest in dst host

4. run "system_reset" in src host hmp and do "migrate -d tcp:$dst_host_ip:4444" immediatly (do not wait for guest reboot finished)

5. observer console on dst host

Actual results:
file system call trace appeared and cannot bootup file system in guest in dst host

console output is as below:
# nc -U /tmp/console 
[    4.210994] intel_rapl: no valid rapl domains found in package 0
[    4.287174] intel_rapl: no valid rapl domains found in package 0
[    4.303701] intel_rapl: no valid rapl domains found in package 0
[    4.310246] intel_rapl: no valid rapl domains found in package 0
[    4.604606] ffff8800b9819000: 42 4d 41 50 00 00 00 42 ff ff ff ff ff ff ff ff  BMAP...B........
[    4.606120] ffff8800b9819010: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00  ................
[    4.607574] ffff8800b9819020: 00 00 10 22 e8 00 00 01 00 00 00 00 00 00 02 00  ..."............
[    4.609127] ffff8800b9819030: 00 00 10 22 e7 80 00 01 00 00 00 00 00 00 04 00  ..."............
[    4.610656] XFS (dm-0): Internal error xfs_bmap_read_extents(1) at line 1321 of file fs/xfs/libxfs/xfs_bmap.c.  Caller xfs_iread_extents+0x75/0xd0 [xfs]
[    4.613264] XFS (dm-0): Corruption detected. Unmount and run xfs_repair
[    4.614464] ffff8800b9819000: 42 4d 41 50 00 00 00 42 ff ff ff ff ff ff ff ff  BMAP...B........
[    4.615926] ffff8800b9819010: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00  ................
[    4.617365] ffff8800b9819020: 00 00 10 22 e8 00 00 01 00 00 00 00 00 00 02 00  ..."............
[    4.618850] ffff8800b9819030: 00 00 10 22 e7 80 00 01 00 00 00 00 00 00 04 00  ..."............
[    4.620334] XFS (dm-0): Internal error xfs_bmap_read_extents(1) at line 1321 of file fs/xfs/libxfs/xfs_bmap.c.  Caller xfs_iread_extents+0x75/0xd0 [xfs]
[    4.622812] XFS (dm-0): Corruption detected. Unmount and run xfs_repair
[    4.623963] XFS (dm-0): Internal error xfs_trans_cancel at line 990 of file fs/xfs/xfs_trans.c.  Caller xfs_setattr_size+0x394/0x410 [xfs]
[    4.635581] XFS (dm-0): Corruption of in-memory data detected.  Shutting down filesystem
[    4.636969] XFS (dm-0): Please umount the filesystem and rectify the problem(s)
[  OK  ] Started Show Plymouth Boot Screen.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Basic System.
[  OK  ] Found device /dev/mapper/rhel_dhcp--66--106--180-root.
Starting File System Check on /dev/.../rhel_dhcp--66--106--180-root...
[  OK  ] Started File System Check on /dev/mapper/rhel_dhcp--66--106--180-root.
[  OK  ] Started dracut initqueue hook.
Mounting /sysroot...
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
[  OK  ] Mounted /sysroot.
[  OK  ] Reached target Initrd Root File System.
Starting Reload Configuration from the Real Root...
[  OK  ] Started Reload Configuration from the Real Root.
[  OK  ] Reached target Initrd File Systems.
[  OK  ] Reached target Initrd Default Target.
Starting dracut pre-pivot and cleanup hook...
[  OK  ] Started dracut pre-pivot and cleanup hook.
[  OK  ] Stopped udev Kernel Device Manager.
Starting udev Kernel Device Manager...
Starting Cleaning Up and Shutting Down Daemons...
[  OK  ] Stopped target Timers.
Starting Plymouth switch root service...
[  OK  ] Stopped Cleaning Up and Shutting Down Daemons.
[  OK  ] Stopped dracut pre-pivot and cleanup hook.
Stopping dracut pre-pivot and cleanup hook...
[  OK  ] Stopped target Remote File Systems.
[  OK  ] Stopped target Remote File Systems (Pre).
[  OK  ] Stopped dracut initqueue hook.
Stopping dracut initqueue hook...
[  OK  ] Stopped target Initrd Default Target.
[  OK  ] Stopped target Basic System.
[  OK  ] Stopped target Sockets.
[  OK  ] Stopped target Slices.
[  OK  ] Stopped target Paths.
[  OK  ] Stopped target System Initialization.
[  OK  ] Stopped target Local File Systems.
[  OK  ] Stopped Apply Kernel Variables.
Stopping Apply Kernel Variables...
[  OK  ] Stopped target Swap.
[  OK  ] Stopped udev Coldplug all Devices.
Stopping udev Coldplug all Devices...
Stopping udev Kernel Device Manager...
[  OK  ] Stopped udev Kernel Device Manager.
[  OK  ] Started Plymouth switch root service.
[  OK  ] Stopped dracut pre-udev hook.
Stopping dracut pre-udev hook...
[  OK  ] Stopped dracut cmdline hook.
Stopping dracut cmdline hook...
[  OK  ] Stopped Create Static Device Nodes in /dev.
Stopping Create Static Device Nodes in /dev...
[  OK  ] Stopped Create list of required sta...ce nodes for the current kernel.
Stopping Create list of required st... nodes for the current kernel...
[  OK  ] Closed udev Control Socket.
[  OK  ] Closed udev Kernel Socket.
Starting Cleanup udevd DB...
[  OK  ] Started Cleanup udevd DB.
[  OK  ] Reached target Switch Root.
Starting Switch Root...
Welcome to Red Hat Enterprise Linux Server 7.2 (Maipo)!
[  OK  ] Stopped Switch Root.
[  OK  ] Listening on LVM2 metadata daemon socket.
Mounting RPC Pipe File System...
[  OK  ] Listening on Delayed Shutdown Socket.
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Listening on udev Kernel Socket.
[  OK  ] Created slice system-getty.slice.
[  OK  ] Created slice system-serial\x2dgetty.slice.
[  OK  ] Listening on Device-mapper event daemon FIFOs.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Reached target Slices.
[  OK  ] Stopped target Switch Root.
[  OK  ] Stopped File System Check on Root Device.
Stopping File System Check on Root Device...
Starting Apply Kernel Variables...
Starting Monitoring of LVM2 mirrors... dmeventd or progress polling...
Mounting Debug File System...
[  OK  ] Stopped target Initrd Root File System.
[  OK  ] Stopped target Initrd File Systems.
Starting Create list of required st... nodes for the current kernel...
[  OK  ] Stopped Flush Journal to Persistent Storage.
[  OK  ] Stopped Journal Service.
Starting Journal Service...
Starting Remount Root and Kernel File Systems...
Mounting POSIX Message Queue File System...
Mounting Huge Pages File System...
Mounting NFSD configuration filesystem...
[  OK  ] Listening on udev Control Socket.
[  OK  ] Listening on LVM2 poll daemon socket.
[  OK  ] Set up automount Arbitrary Executab...ats File System Automount Point.
[  OK  ] Mounted RPC Pipe File System.
[  OK  ] Mounted Debug File System.
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Started Journal Service.
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Create list of required sta...ce nodes for the current kernel.
[  OK  ] Started Remount Root and Kernel File Systems.
[  OK  ] Mounted NFSD configuration filesystem.
[  OK  ] Started LVM2 metadata daemon.
Starting LVM2 metadata daemon...
Starting udev Coldplug all Devices...
Starting Load/Save Random Seed...
Starting Configure read-only root support...
Starting Create Static Device Nodes in /dev...
Starting Flush Journal to Persistent Storage...
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started Configure read-only root support.
[  OK  ] Started Flush Journal to Persistent Storage.
[  OK  ] Started Create Static Device Nodes in /dev.
[  OK  ] Reached target Local File Systems (Pre).
Starting udev Kernel Device Manager...
[  OK  ] Started udev Coldplug all Devices.
Starting udev Wait for Complete Device Initialization...
[  OK  ] Started udev Kernel Device Manager.
[  OK  ] Started Monitoring of LVM2 mirrors,...ng dmeventd or progress polling.
[  OK  ] Found device /dev/ttyS0.
[  OK  ] Created slice system-lvm2\x2dpvscan.slice.
Starting LVM2 PV scan on device 252:2...
[  OK  ] Found device /dev/disk/by-uuid/709d1146-4ea7-440a-bfed-53f6c4b1a840.
Mounting /boot...
[  OK  ] Started LVM2 PV scan on device 252:2.
[  OK  ] Found device /dev/mapper/rhel_dhcp--66--106--180-swap.
Activating swap /dev/mapper/rhel_dhcp--66--106--180-swap...
[  OK  ] Activated swap /dev/mapper/rhel_dhcp--66--106--180-swap.
[  OK  ] Mounted /boot.
[  OK  ] Reached target Swap.
[  OK  ] Started udev Wait for Complete Device Initialization.
Starting Activation of DM RAID sets...
[  OK  ] Started Activation of DM RAID sets.
[  OK  ] Reached target Local File Systems.
Starting Tell Plymouth To Write Out Runtime Data...
Starting Import network configuration from initramfs...
Starting Preprocess NFS configuration...
[  OK  ] Reached target Encrypted Volumes.
[  OK  ] Started Preprocess NFS configuration.
[  OK  ] Started Tell Plymouth To Write Out Runtime Data.
[  OK  ] Started Import network configuration from initramfs.
Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
Starting Security Auditing Service...
[  OK  ] Started Security Auditing Service.
Starting Update UTMP about System Boot/Shutdown...
[FAILED] Failed to start Update UTMP about System Boot/Shutdown.
See 'systemctl status systemd-update-utmp.service' for details.
[DEPEND] Dependency failed for Update UTMP about System Runlevel Changes.
[  OK  ] Reached target System Initialization.
[FAILED] Failed to listen on Open-iSCSI iscsiuio Socket.
See 'systemctl status iscsiuio.socket' for details.
[FAILED] Failed to listen on CUPS Printing Service Sockets.
See 'systemctl status cups.socket' for details.
[  OK  ] Reached target Timers.
[FAILED] Failed to listen on RPCbind Server Activation Socket.
See 'systemctl status rpcbind.socket' for details.
[  OK  ] Reached target Paths.
[FAILED] Failed to listen on D-Bus System Message Bus Socket.
See 'systemctl status dbus.socket' for details.
[DEPEND] Dependency failed for D-Bus System Message Bus.
[FAILED] Failed to listen on Avahi mDNS/DNS-SD Stack Activation Socket.
See 'systemctl status avahi-daemon.socket' for details.
[DEPEND] Dependency failed for Avahi mDNS/DNS-SD Stack.
[FAILED] Failed to listen on Open-iSCSI iscsid Socket.
See 'systemctl status iscsid.socket' for details.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
Starting Dump dmesg to /var/log/dmesg...
Starting GSSAPI Proxy Daemon...
Starting System Logging Service...
[  OK  ] Started RHEL-RT environment details setup.
Starting RHEL-RT environment details setup...
Starting Modem Manager...
[  OK  ] Started Hardware RNG Entropy Gatherer Daemon.
Starting Hardware RNG Entropy Gatherer Daemon...
Starting OpenSSH Server Key Generation...
Starting Resets System Activity Logs...
[  OK  ] Started libstoragemgmt plug-in server daemon.
Starting libstoragemgmt plug-in server daemon...
Starting NTP client/server...
[  OK  ] Started kernel threads real time priority setup.
Starting kernel threads real time priority setup...
Starting Kernel Samepage Merging...
Starting firewalld - dynamic firewall daemon...
[  OK  ] Started ABRT Automated Bug Reporting Tool.
Starting ABRT Automated Bug Reporting Tool...
[  OK  ] Started ABRT kernel log watcher.
Starting ABRT kernel log watcher...
Starting Install ABRT coredump hook...
Starting Login Service...
[  OK  ] Started Self Monitoring and Reporting Technology (SMART) Daemon.
Starting Self Monitoring and Reporting Technology (SMART) Daemon...
Starting Restore Sound Card State...
[FAILED] Failed to start Dump dmesg to /var/log/dmesg.
See 'systemctl status rhel-dmesg.service' for details.
[FAILED] Failed to start GSSAPI Proxy Daemon.
See 'systemctl status gssproxy.service' for details.
[FAILED] Failed to start System Logging Service.
See 'systemctl status rsyslog.service' for details.
[FAILED] Failed to start Modem Manager.
See 'systemctl status ModemManager.service' for details.
[FAILED] Failed to start OpenSSH Server Key Generation.
See 'systemctl status sshd-keygen.service' for details.
[FAILED] Failed to start Resets System Activity Logs.
See 'systemctl status sysstat.service' for details.
[FAILED] Failed to start NTP client/server.
See 'systemctl status chronyd.service' for details.
[FAILED] Failed to start Kernel Samepage Merging.
See 'systemctl status ksm.service' for details.
[DEPEND] Dependency failed for Kernel Samepage Merging (KSM) Tuning Daemon.
[FAILED] Failed to start firewalld - dynamic firewall daemon.
See 'systemctl status firewalld.service' for details.
[FAILED] Failed to start Install ABRT coredump hook.
See 'systemctl status abrt-ccpp.service' for details.
[FAILED] Failed to start Login Service.
See 'systemctl status systemd-logind.service' for details.
[  OK  ] Started Restore Sound Card State.
[  OK  ] Stopped Login Service.
[FAILED] Failed to listen on D-Bus System Message Bus Socket.
See 'systemctl status dbus.socket' for details.
Starting Login Service...
Starting Network Manager...
[  OK  ] Reached target NFS client services.
[FAILED] Failed to start Login Service.
See 'systemctl status systemd-logind.service' for details.
[FAILED] Failed to start Network Manager.
See 'systemctl status NetworkManager.service' for details.
[DEPEND] Dependency failed for Network Manager Wait Online.
[  OK  ] Stopped Login Service.
[FAILED] Failed to listen on D-Bus System Message Bus Socket.
See 'systemctl status dbus.socket' for details.
Starting Login Service...
Starting LSB: Bring up/down networking...
[FAILED] Failed to start Login Service.
See 'systemctl status systemd-logind.service' for details.
[FAILED] Failed to start LSB: Bring up/down networking.
See 'systemctl status network.service' for details.
[  OK  ] Stopped Login Service.
[FAILED] Failed to listen on D-Bus System Message Bus Socket.
See 'systemctl status dbus.socket' for details.
Starting Login Service...
[  OK  ] Reached target Network.
[FAILED] Failed to start CUPS Printing Service.
See 'systemctl status cups.service' for details.
Starting CUPS Printing Service...
Starting Notify NFS peers of a restart...
[  OK  ] Reached target Network is Online.
Starting Login and scanning of iSCSI devices...
Starting Dynamic System Tuning Daemon...
Starting Postfix Mail Transport Agent...
Starting Logout off all iSCSI sessions on shutdown...
[FAILED] Failed to start OpenSSH server daemon.
See 'systemctl status sshd.service' for details.
Starting OpenSSH server daemon...
Starting Enable periodic update of entitlement certificates....
[FAILED] Failed to start Login Service.
See 'systemctl status systemd-logind.service' for details.
[  OK  ] Started Notify NFS peers of a restart.
[FAILED] Failed to start Dynamic System Tuning Daemon.
See 'systemctl status tuned.service' for details.
[  OK  ] Started Logout off all iSCSI sessions on shutdown.
[FAILED] Failed to start Enable periodic update of entitlement certificates..
See 'systemctl status rhsmcertd.service' for details.
[  OK  ] Started Login and scanning of iSCSI devices.
[FAILED] Failed to start Postfix Mail Transport Agent.
See 'systemctl status postfix.service' for details.
[  OK  ] Stopped Login Service.
[FAILED] Failed to listen on D-Bus System Message Bus Socket.
See 'systemctl status dbus.socket' for details.
Starting Login Service...
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
Starting Crash recovery kernel arming...
Starting Permit User Sessions...
Starting LSB: Starts the Spacewalk Daemon...
Starting Availability of block devices...
[FAILED] Failed to start Login Service.
See 'systemctl status systemd-logind.service' for details.
[FAILED] Failed to start Crash recovery kernel arming.
See 'systemctl status kdump.service' for details.
[FAILED] Failed to start Permit User Sessions.
See 'systemctl status systemd-user-sessions.service' for details.
[FAILED] Failed to start LSB: Starts the Spacewalk Daemon.
See 'systemctl status rhnsd.service' for details.
[FAILED] Failed to start Availability of block devices.
See 'systemctl status blk-availability.service' for details.
[  OK  ] Stopped Login Service.
[FAILED] Failed to listen on D-Bus System Message Bus Socket.
See 'systemctl status dbus.socket' for details.
[FAILED] Failed to start Login Service.
See 'systemctl status systemd-logind.service' for details.
[FAILED] Failed to start Command Scheduler.
See 'systemctl status crond.service' for details.
Starting Command Scheduler...
Starting Terminate Plymouth Boot Screen...
[FAILED] Failed to start Job spooling tools.
See 'systemctl status atd.service' for details.
Starting Job spooling tools...
Starting Wait for Plymouth Boot Screen to Quit...
Starting Harvest vmcores for ABRT...
[  OK  ] Started Terminate Plymouth Boot Screen.
[  OK  ] Started Wait for Plymouth Boot Screen to Quit.
[  OK  ] Started Serial Getty on ttyS0.
Starting Serial Getty on ttyS0...
[  OK  ] Started Getty on tty1.
Starting Getty on tty1...
[  OK  ] Reached target Login Prompts.
[FAILED] Failed to start Harvest vmcores for ABRT.
See 'systemctl status abrt-vmcore.service' for details.
[  OK  ] Stopped System Logging Service.
Starting System Logging Service...
[FAILED] Failed to start System Logging Service.
See 'systemctl status rsyslog.service' for details.
[  OK  ] Reached target Multi-User System.



Expected results:
guest in dst host can boot up normally

Additional info:
1. run the same test with qemu-kvm-rhev-2.3.0-31.el7, guest in dst host can bootup, but also displayed xfs call trace
[   19.333954] XFS (dm-0): Corruption detected. Unmount and run xfs_repair
[   19.336294] XFS (dm-0): corrupt dinode 135951107, (btree extents).
[   19.336298] ffff880134f7a000: 42 4d 41 50 00 00 00 41 ff ff ff ff ff ff ff ff  BMAP...A........
[   19.339582] ffff880134f7a010: ff ff ff ff ff ff ff ff 00 00 00 00 00 00 00 00  ................
[   19.342583] ffff880134f7a020: 00 00 10 22 e8 00 00 01 00 00 00 00 00 00 02 00  ..."............
[   19.345587] ffff880134f7a030: 00 00 10 22 e7 80 00 01 00 00 00 00 00 00 04 00  ..."............
[   19.348049] XFS (dm-0): Internal error xfs_bmap_read_extents(1) at line 1321 of file fs/xfs/libxfs/xfs_bmap.c.  Caller xfs_iread_extents+0x75/0xd0 [xfs]
[   19.353150] CPU: 3 PID: 1427 Comm: login Not tainted 3.10.0-355.rt56.234.el7.x86_64 #
[   19.353154] Hardware name: Red Hat KVM, BIOS seabios-1.7.5-11.el7 04/01/2014
[   19.353158]  ffff880131734000 0000000067410efc ffff8801315bfb30 ffffffff816343a5
[   19.353159]  ffff8801315bfb48 ffffffffa01f784b ffffffffa01e78a5 ffff8801315bfb80
[   19.353160]  ffffffffa01f78a5 0000052900000008 ffff8800ba9c63c0 ffff8800ba9c6400
[   19.353163] Call Trace:
[   19.353175]  [<ffffffff816343a5>] dump_stack+0x19/0x1b
[   19.353228]  [<ffffffffa01f784b>] xfs_error_report+0x3b/0x40 [xfs]
[   19.353243]  [<ffffffffa01e78a5>] ? xfs_iread_extents+0x75/0xd0 [xfs]
[   19.353256]  [<ffffffffa01f78a5>] xfs_corruption_error+0x55/0x80 [xfs]
[   19.353268]  [<ffffffffa01c46f5>] xfs_bmap_read_extents+0x3c5/0x3d0 [xfs]
[   19.353281]  [<ffffffffa01e78a5>] ? xfs_iread_extents+0x75/0xd0 [xfs]
[   19.353294]  [<ffffffffa01e78a5>] xfs_iread_extents+0x75/0xd0 [xfs]
[   19.353305]  [<ffffffffa01c59ad>] xfs_bmapi_read+0x2dd/0x300 [xfs]
[   19.353310]  [<ffffffff810dc94e>] ? rt_down_write_nested+0xe/0x10
[   19.353326]  [<ffffffffa020632c>] ? xfs_ilock+0x11c/0x190 [xfs]
[   19.353339]  [<ffffffffa01ed976>] __xfs_get_blocks+0xe6/0x880 [xfs]
[   19.353343]  [<ffffffff81165d91>] ? __find_lock_page+0x21/0x70
[   19.353356]  [<ffffffffa01ee121>] xfs_get_blocks+0x11/0x20 [xfs]
[   19.353359]  [<ffffffff8120d58d>] block_truncate_page+0x11d/0x2b0
[   19.353371]  [<ffffffffa01ee110>] ? __xfs_get_blocks+0x880/0x880 [xfs]
[   19.353386]  [<ffffffffa020517b>] xfs_setattr_size+0x19b/0x410 [xfs]
[   19.353387]  [<ffffffff810dc94e>] ? rt_down_write_nested+0xe/0x10
[   19.353401]  [<ffffffffa0205455>] xfs_vn_setattr+0x65/0x90 [xfs]
[   19.353405]  [<ffffffff811f3ed9>] notify_change+0x279/0x3d0
[   19.353407]  [<ffffffff811d3db3>] do_truncate+0x73/0xc0
[   19.353410]  [<ffffffff811d86b8>] ? __sb_start_write+0x58/0x110
[   19.353412]  [<ffffffff811d415c>] do_sys_ftruncate.constprop.17+0x11c/0x180
[   19.353413]  [<ffffffff811d41fe>] SyS_ftruncate+0xe/0x10
[   19.353416]  [<ffffffff81642954>] tracesys+0xdd/0xe2
[   19.353419] XFS (dm-0): Corruption detected. Unmount and run xfs_repair


2. if guest run with non-rt kernel, the issue didn't occur
Comment 3 xiywang 2016-03-14 04:29:30 EDT
I have tested this case based on the newest rt kernel 3.10.0-364.rt56.241.el7 and 7.2 release version 3.10.0-327.rt56.204.el7. All the three versions (327, 355 and 364) share the same issue.
Comment 4 Rik van Riel 2016-03-14 11:38:43 EDT
I have a question about this test. One of the steps is this:

"3. boot guest in dst host"

At that point, is the guest running simultaneously on the src and dst hosts, simultaneously accessing the same filesystem from both instances?

If so, that is expected to cause filesystem corruption (like you observed).
Comment 5 xiywang 2016-03-14 21:13:13 EDT
(In reply to Rik van Riel from comment #4)
> I have a question about this test. One of the steps is this:
> 
> "3. boot guest in dst host"
> 
> At that point, is the guest running simultaneously on the src and dst hosts,
> simultaneously accessing the same filesystem from both instances?
> 
> If so, that is expected to cause filesystem corruption (like you observed).

No. The two guests are not running simultaneously.
When the guest running in src host, the command for dst host to boot guest was added "-incoming tcp:0:4444", which means the guest in dst host was before migration and the result of "info status" was "paused".
And the two guests were using different filesystem on each host, which I copied from src host to dst host before testing.
Comment 6 Peter Xu 2016-03-16 02:09:39 EDT
(In reply to xiywang from comment #5)

[...]

> And the two guests were using different filesystem on each host, which I
> copied from src host to dst host before testing.

Could you explain what does "using different filesystem on each host" mean? AFAIK, we should make sure that both src and dst QEMU are using the same block device backend for migration (e.g., using NFS share folders to store the qcow2 file, and mount them on both sides). Am I correct?

Peter
Comment 7 xiywang 2016-03-22 06:13:54 EDT
(In reply to Peter Xu from comment #6)
> (In reply to xiywang from comment #5)
> 
> [...]
> 
> > And the two guests were using different filesystem on each host, which I
> > copied from src host to dst host before testing.
> 
> Could you explain what does "using different filesystem on each host" mean?
> AFAIK, we should make sure that both src and dst QEMU are using the same
> block device backend for migration (e.g., using NFS share folders to store
> the qcow2 file, and mount them on both sides). Am I correct?
> 
> Peter

I forgot to mount nfs the last time I tested this case. And after mount nfs root, this issue gone. So I'm closing this bug as NOTABUG.

Note You need to log in before you can comment on or make changes to this bug.