| Summary: | 2.2.z: after migration backend calls destroy for VM but VM still appears as Down on source host | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 5 | Reporter: | Dafna Ron <dron> | ||||||
| Component: | vdsm22 | Assignee: | Igor Lvovsky <ilvovsky> | ||||||
| Status: | CLOSED DEFERRED | QA Contact: | yeylon <yeylon> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | low | ||||||||
| Version: | 5.6 | CC: | abaron, bazulay, danken, iheim, lpeer, srevivo, ykaul | ||||||
| Target Milestone: | rc | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2011-06-20 09:00:18 UTC | Type: | --- | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Attachments: |
|
||||||||
This seems similar to RHEV 2.3's bug 672493: we fail destroy() since tearDown cannot be accomplished without active storage. The only problem is that RHEV-M does not expect destroy() to fail; we may need to move such VM to "Powering down" instead. I'm not sure that we want such an intrusive change in zsteam, but Igor, would you take a look? *** Bug 684586 has been marked as a duplicate of this bug. *** Dafna, I need to check whether 'qemu' process of this VM still exists on source host this is source host: no longer SPM and non-operational. [root@blond-vdsf ~]# vdsClient -s 0 list table 2c082f3c-bb05-49b4-8d33-e8b97360ea00 5492 PinToHost1 Paused b6b1c807-10fb-4401-8b03-43bdec530e95 5921 MigrateVM1 Down 92e827f4-dd2b-4004-8efc-faa91a6f5090 5629 PinToHost2 Paused f7f57ba3-b077-4be2-9938-b7b9864f68ac 5776 MigrateVM2 Down [root@blond-vdsf ~]# ps -elf |grep qemu 0 S vdsm 5483 1 0 76 -5 - 15963 wait 14:36 ? 00:00:00 /bin/sh -c /usr/bin/sudo /usr/bin/tunctl -d rtl8139_10_1; /usr/bin/sudo /usr/bin/tunctl -b -u vdsm -t rtl8139_10_1;/usr/bin/sudo /sbin/ip link set dev rtl8139_10_1 address fe:1a:4a:23:71:00;/usr/bin/sudo /sbin/ip link set dev rtl8139_10_1 up;/usr/bin/sudo /usr/sbin/brctl addif rhevm rtl8139_10_1;TZ=UTC /bin/nice -n 5 /usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T12:36:08 -name PinToHost1 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:00,model=rtl8139 -net tap,vlan=1,ifname=rtl8139_10_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/eda275b7-b740-4ccb-91d7-8d3a7f5c57b0/23129365-2c8e-45ed-b56a-10d876ea74b5,media=disk,if=ide,cache=off,index=0,serial=cb-91d7-8d3a7f5c57b0,boot=off,format=qcow2,werror=stop -pidfile /var/vdsm/2c082f3c-bb05-49b4-8d33-e8b97360ea00.pid -vnc 0:10,password -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer="Red Hat",product="RHEL",version=5Server-5.6.0.3,serial="44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca",uuid="2c082f3c-bb05-49b4-8d33-e8b97360ea00" -vmchannel di:0200,unix:/var/vdsm/2c082f3c-bb05-49b4-8d33-e8b97360ea00.guest.socket,server -monitor unix:/var/vdsm/2c082f3c-bb05-49b4-8d33-e8b97360ea00.monitor.socket,server 1>/var/vdsm/2c082f3c-bb05-49b4-8d33-e8b97360ea00.stdio.dump 2>&1; /usr/bin/sudo /usr/bin/tunctl -d rtl8139_10_1; 0 S vdsm 5492 5483 4 75 0 - 312427 - 14:36 ? 00:00:19 /usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T12:36:08 -name PinToHost1 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:00,model=rtl8139 -net tap,vlan=1,ifname=rtl8139_10_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/eda275b7-b740-4ccb-91d7-8d3a7f5c57b0/23129365-2c8e-45ed-b56a-10d876ea74b5,media=disk,if=ide,cache=off,index=0,serial=cb-91d7-8d3a7f5c57b0,boot=off,format=qcow2,werror=stop -pidfile /var/vdsm/2c082f3c-bb05-49b4-8d33-e8b97360ea00.pid -vnc 0:10,password -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer=Red Hat,product=RHEL,version=5Server-5.6.0.3,serial=44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca,uuid=2c082f3c-bb05-49b4-8d33-e8b97360ea00 -vmchannel di:0200,unix:/var/vdsm/2c082f3c-bb05-49b4-8d33-e8b97360ea00.guest.socket,server -monitor unix:/var/vdsm/2c082f3c-bb05-49b4-8d33-e8b97360ea00.monitor.socket,server 0 S vdsm 5620 1 0 73 -5 - 15963 wait 14:36 ? 00:00:00 /bin/sh -c /usr/bin/sudo /usr/bin/tunctl -d rtl8139_11_1; /usr/bin/sudo /usr/bin/tunctl -b -u vdsm -t rtl8139_11_1;/usr/bin/sudo /sbin/ip link set dev rtl8139_11_1 address fe:1a:4a:23:71:01;/usr/bin/sudo /sbin/ip link set dev rtl8139_11_1 up;/usr/bin/sudo /usr/sbin/brctl addif rhevm rtl8139_11_1;TZ=UTC /bin/nice -n 5 /usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T13:36:09 -name PinToHost2 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:01,model=rtl8139 -net tap,vlan=1,ifname=rtl8139_11_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/a8c1554f-c397-4efc-b834-a3c62e8081b6/e6d19e3f-d9d1-4764-adaf-7af631655064,media=disk,if=ide,cache=off,index=0,serial=fc-b834-a3c62e8081b6,boot=off,format=qcow2,werror=stop -pidfile /var/vdsm/92e827f4-dd2b-4004-8efc-faa91a6f5090.pid -vnc 0:11,password -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer="Red Hat",product="RHEL",version=5Server-5.6.0.3,serial="44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca",uuid="92e827f4-dd2b-4004-8efc-faa91a6f5090" -vmchannel di:0200,unix:/var/vdsm/92e827f4-dd2b-4004-8efc-faa91a6f5090.guest.socket,server -monitor unix:/var/vdsm/92e827f4-dd2b-4004-8efc-faa91a6f5090.monitor.socket,server 1>/var/vdsm/92e827f4-dd2b-4004-8efc-faa91a6f5090.stdio.dump 2>&1; /usr/bin/sudo /usr/bin/tunctl -d rtl8139_11_1; 0 S vdsm 5629 5620 22 75 0 - 312427 - 14:36 ? 00:01:40 /usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T13:36:09 -name PinToHost2 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:01,model=rtl8139 -net tap,vlan=1,ifname=rtl8139_11_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/a8c1554f-c397-4efc-b834-a3c62e8081b6/e6d19e3f-d9d1-4764-adaf-7af631655064,media=disk,if=ide,cache=off,index=0,serial=fc-b834-a3c62e8081b6,boot=off,format=qcow2,werror=stop -pidfile /var/vdsm/92e827f4-dd2b-4004-8efc-faa91a6f5090.pid -vnc 0:11,password -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer=Red Hat,product=RHEL,version=5Server-5.6.0.3,serial=44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca,uuid=92e827f4-dd2b-4004-8efc-faa91a6f5090 -vmchannel di:0200,unix:/var/vdsm/92e827f4-dd2b-4004-8efc-faa91a6f5090.guest.socket,server -monitor unix:/var/vdsm/92e827f4-dd2b-4004-8efc-faa91a6f5090.monitor.socket,server 0 S vdsm 5767 1 0 76 -5 - 15963 wait 14:36 ? 00:00:00 /bin/sh -c /usr/bin/sudo /usr/bin/tunctl -d virtio_12_1; /usr/bin/sudo /usr/bin/tunctl -b -u vdsm -t virtio_12_1;/usr/bin/sudo /sbin/ip link set dev virtio_12_1 address fe:1a:4a:23:71:03;/usr/bin/sudo /sbin/ip link set dev virtio_12_1 up;/usr/bin/sudo /usr/sbin/brctl addif rhevm virtio_12_1;TZ=UTC /bin/nice -n 5 /usr/libexec/qemu-kvm -no-hpet -no-kvm-pit-reinjection -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T12:36:12 -name MigrateVM2 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:03,model=virtio -net tap,vlan=1,ifname=virtio_12_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/4ef9001c-23d8-48b9-b586-3f0a6c41adbd/5df5122d-c0b8-4c68-b3dc-6b6d655135de,media=disk,if=virtio,cache=off,serial=b9-b586-3f0a6c41adbd,boot=on,format=qcow2,werror=stop -pidfile /var/vdsm/f7f57ba3-b077-4be2-9938-b7b9864f68ac.pid -vnc 0:12,password -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer="Red Hat",product="RHEL",version=5Server-5.6.0.3,serial="44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca",uuid="f7f57ba3-b077-4be2-9938-b7b9864f68ac" -vmchannel di:0200,unix:/var/vdsm/f7f57ba3-b077-4be2-9938-b7b9864f68ac.guest.socket,server -monitor unix:/var/vdsm/f7f57ba3-b077-4be2-9938-b7b9864f68ac.monitor.socket,server 1>/var/vdsm/f7f57ba3-b077-4be2-9938-b7b9864f68ac.stdio.dump 2>&1; /usr/bin/sudo /usr/bin/tunctl -d virtio_12_1; 0 S vdsm 5776 5767 6 75 0 - 318620 - 14:36 ? 00:00:30 /usr/libexec/qemu-kvm -no-hpet -no-kvm-pit-reinjection -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T12:36:12 -name MigrateVM2 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:03,model=virtio -net tap,vlan=1,ifname=virtio_12_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/4ef9001c-23d8-48b9-b586-3f0a6c41adbd/5df5122d-c0b8-4c68-b3dc-6b6d655135de,media=disk,if=virtio,cache=off,serial=b9-b586-3f0a6c41adbd,boot=on,format=qcow2,werror=stop -pidfile /var/vdsm/f7f57ba3-b077-4be2-9938-b7b9864f68ac.pid -vnc 0:12,password -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer=Red Hat,product=RHEL,version=5Server-5.6.0.3,serial=44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca,uuid=f7f57ba3-b077-4be2-9938-b7b9864f68ac -vmchannel di:0200,unix:/var/vdsm/f7f57ba3-b077-4be2-9938-b7b9864f68ac.guest.socket,server -monitor unix:/var/vdsm/f7f57ba3-b077-4be2-9938-b7b9864f68ac.monitor.socket,server 0 S vdsm 5912 1 0 79 -5 - 15963 wait 14:36 ? 00:00:00 /bin/sh -c /usr/bin/sudo /usr/bin/tunctl -d virtio_13_1; /usr/bin/sudo /usr/bin/tunctl -b -u vdsm -t virtio_13_1;/usr/bin/sudo /sbin/ip link set dev virtio_13_1 address fe:1a:4a:23:71:02;/usr/bin/sudo /sbin/ip link set dev virtio_13_1 up;/usr/bin/sudo /usr/sbin/brctl addif rhevm virtio_13_1;TZ=UTC /bin/nice -n 5 /usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T12:36:14 -name MigrateVM1 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:02,model=virtio -net tap,vlan=1,ifname=virtio_13_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/665c742e-dffa-4ee6-b0f2-dfc4964295d9/7c2c6b53-923d-405f-b96b-a6e336afa813,media=disk,if=virtio,cache=off,serial=e6-b0f2-dfc4964295d9,boot=on,format=qcow2,werror=stop -pidfile /var/vdsm/b6b1c807-10fb-4401-8b03-43bdec530e95.pid -vnc 0:13,password -incoming tcp::49153 -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer="Red Hat",product="RHEL",version=5Server-5.6.0.3,serial="44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca",uuid="b6b1c807-10fb-4401-8b03-43bdec530e95" -vmchannel di:0200,unix:/var/vdsm/b6b1c807-10fb-4401-8b03-43bdec530e95.guest.socket,server -monitor unix:/var/vdsm/b6b1c807-10fb-4401-8b03-43bdec530e95.monitor.socket,server 1>/var/vdsm/b6b1c807-10fb-4401-8b03-43bdec530e95.stdio.dump 2>&1; /usr/bin/sudo /usr/bin/tunctl -d virtio_13_1; 0 S vdsm 5921 5912 5 75 0 - 319291 - 14:36 ? 00:00:26 /usr/libexec/qemu-kvm -no-hpet -usbdevice tablet -rtc-td-hack -startdate 2011-03-23T12:36:14 -name MigrateVM1 -smp 1,cores=1 -k en-us -m 1024 -boot c -net nic,vlan=1,macaddr=00:1a:4a:23:71:02,model=virtio -net tap,vlan=1,ifname=virtio_13_1,script=no -drive file=/rhev/data-center/26667093-cfe4-46ea-8ec4-43b4150c796a/a7536651-1aa6-4ff4-a71e-a1d570475fb7/images/665c742e-dffa-4ee6-b0f2-dfc4964295d9/7c2c6b53-923d-405f-b96b-a6e336afa813,media=disk,if=virtio,cache=off,serial=e6-b0f2-dfc4964295d9,boot=on,format=qcow2,werror=stop -pidfile /var/vdsm/b6b1c807-10fb-4401-8b03-43bdec530e95.pid -vnc 0:13,password -incoming tcp::49153 -cpu qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm -M rhel5.5.0 -notify all -balloon none -smbios type=1,manufacturer=Red Hat,product=RHEL,version=5Server-5.6.0.3,serial=44454C4C-3800-1046-8035-B1C04F34344A_00:22:19:14:7e:ca,uuid=b6b1c807-10fb-4401-8b03-43bdec530e95 -vmchannel di:0200,unix:/var/vdsm/b6b1c807-10fb-4401-8b03-43bdec530e95.guest.socket,server -monitor unix:/var/vdsm/b6b1c807-10fb-4401-8b03-43bdec530e95.monitor.socket,server 0 S root 10502 4234 0 77 0 - 15298 pipe_w 14:43 pts/0 00:00:00 grep qemu [root@blond-vdsf ~]# Created attachment 487249 [details]
new vdsm log
|
Created attachment 478925 [details] logs Description of problem: when running migrateble VM's and non-migratble VM's on the same host -> creating storage disconnection on host, the VM's which can be migrated are migrated but remain in source host in down state even though backend sends destroy. Version-Release number of selected component (if applicable): ic92 vdsm22-debuginfo-4.5-63.17.el5_6 vdsm22-cli-4.5-63.17.el5_6 vdsm22-faqemu-4.5-63.17.el5_6 vdsm22-4.5-63.17.el5_6 kvm-qemu-img-83-224.el5 vdsm22-faqemu-4.5-63.17.el5_6 How reproducible: 100% Steps to Reproduce: 1. run both pin-to-host and non-pinned VM's on source host 2. block storage using iptables 3. VM will migrate to destination host Actual results: VM that migrated are shown on source host in down state and in destination host in up state Expected results: VM's should only appear as up on destination after backend send destroy. Additional info: complete logs attached [root@blond-vdsf ~]# less /var/log/vdsm/vdsm.log |grep '13daec20-4dd0-43ce-95f0-6f1d3ddc22ac.*destroy' Thread-167::DEBUG::2011-02-15 18:22:14,653::vm::1309::vds.vmlog.13daec20-4dd0-43ce-95f0-6f1d3ddc22ac::destroy Called Thread-167::DEBUG::2011-02-15 18:22:15,816::vm::1361::vds.vmlog.13daec20-4dd0-43ce-95f0-6f1d3ddc22ac::Total desktops after destroy of 13daec20-4dd0-43ce-95f0-6f1d3ddc22ac is 6 Thread-79::DEBUG::2011-02-15 18:41:08,529::vm::1309::vds.vmlog.13daec20-4dd0-43ce-95f0-6f1d3ddc22ac::destroy Called Thread-79::DEBUG::2011-02-15 18:41:09,693::vm::1361::vds.vmlog.13daec20-4dd0-43ce-95f0-6f1d3ddc22ac::Total desktops after destroy of 13daec20-4dd0-43ce-95f0-6f1d3ddc22ac is 2 [root@blond-vdsf tmp]# vdsClient -s 0 list table |grep -i down 13daec20-4dd0-43ce-95f0-6f1d3ddc22ac 23216 RHEL6 Down 493980b6-730a-4ba9-babf-521482977559 23542 SesyPrep2003tmp Down d0199102-4a4b-42b4-b51f-d6fc5c1e714a 23330 2008R2 Down 8572858d-6874-44c4-9de0-f352bc18ba5d 23439 2008 Down [root@blond-vdsg tmp]# vdsClient -s 0 list table 493980b6-730a-4ba9-babf-521482977559 4572 SesyPrep2003tmp Up 8572858d-6874-44c4-9de0-f352bc18ba5d 4458 2008 Up 13daec20-4dd0-43ce-95f0-6f1d3ddc22ac 4237 RHEL6 Up d0199102-4a4b-42b4-b51f-d6fc5c1e714a 4347 2008R2 Up