Bug 882660

Summary: [TEXT] [Storage] [Hot Plug] Log error message on each plug or unplug disk on NFS storage although the plug and unplug successful.
Product: Red Hat Enterprise Virtualization Manager Reporter: Leonid Natapov <lnatapov>
Component: ovirt-engineAssignee: Arik <ahadas>
Status: CLOSED DUPLICATE QA Contact: Leonid Natapov <lnatapov>
Severity: low Docs Contact:
Priority: low    
Version: unspecifiedCC: abaron, amureini, dyasny, hateya, iheim, lpeer, ofrenkel, Rhev-m-bugs, sgrinber, vvyazmin, yeylon, ykaul
Target Milestone: ---   
Target Release: 3.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: virt
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 885984 (view as bug list) Environment:
Last Closed: 2013-02-07 15:20:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 885984    
Attachments:
Description Flags
engine log none

Description Leonid Natapov 2012-12-02 14:34:12 UTC
Created attachment 656026 [details]
engine log

[Upgrade] [Storage] [Hot Plug] Error message on each plug or unplug disk on NFS storage although the plug and unplug successful. 

How to reproduce:
1.Create Vm with several disks.
2.Start VM.
3.Plug and unplug disk.

Reproduces 100% on several environments on NFS storage.
Doesn't happen on FC Storage. Probably won't happen on ISCSI too.

2012-12-02 15:30:54,160 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-69) VM d96ab89d-30f3-4b97-bd09-e6cb97068e0a managed non pluggable device was removed unexpectedly from libvirt: VmDevice {vmId=d96ab89d-30f3-4b97-bd09-e6cb97068e0a, deviceId=66324a57-8dbf-4e80-a26e-edfd7037dd3a, device=disk, type=disk, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x0a, function=0x0}, managed=true, plugged=false, readOnly=false, alias=virtio-disk2}

Also the error message format seems to be incorrect, For example "alias=virtio-disk2" is incorrect. There is no alias virtio-disk2.


Full log attached.

Comment 1 Vered Volansky 2012-12-11 09:19:25 UTC
bug 885984 was cloned from this one for Error message issue.

This bug will deal with the VdsUpdateRunTimeInfo ERROR alone.
It did reproduce for me, thought not every time.
Plug and unplug end with success - ALWAYS.
There is some race between plug/unplug and VdsUpdateRunTimeInfo, depends on when exactly VdsUpdateRunTimeInfo actually runs.
plug/unplug probably changes data that VdsUpdateRunTimeInfo is trying to retrieve.

The error occurs in VdsUpdateRunTimeInfo and has no effect on the success/failure of plug/unplug.
It's only detectable in the log.

lesik - Why did you mark upgrade in the bug's title?

Comment 2 Tal Nisan 2013-01-14 17:12:11 UTC
What basically happens is that on detach of any device (Not necessarily a disk) there can be a race and after the unplug command is sent to VDSM but before the update in the DB, the VdsUpdateRunTimeInfo returns the list of active devices without the unplugged device (Since VDSM already unplugged it) and then VdsUpdateRunTimeInfo handles it as a device which was removed unexpectedly.
Given the facts, this is a virt issue so moving to virt team

Comment 3 vvyazmin@redhat.com 2013-01-16 15:48:30 UTC
Same error found on RHEVM 3.2 - SF03 environment ( Hot-Plug and Hot-UnPlug Direct LUN in FC DC environment, with preallocated disk only)

RHEVM: rhevm-3.2.0-4.el6ev.noarch
VDSM: vdsm-4.10.2-3.0.el6ev.x86_64
LIBVIRT: libvirt-0.10.2-13.el6.x86_64
QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.348.el6.x86_64
SANLOCK: sanlock-2.6-2.el6.x86_64

Comment 4 Vered Volansky 2013-01-31 19:22:51 UTC
Cleared needinfo flag on me since no question was ever asked.

Comment 5 Arik 2013-02-07 15:20:36 UTC
not exactly the same because this one talks about disks and 872165 talks about nics, but it's a general problem for pluggable devices. it seems that the fix for 872165 should solve this case too.

*** This bug has been marked as a duplicate of bug 872165 ***