Bug 1315100 - Command 'org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand' failed: Failed managing transaction
Command 'org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand' failed:...
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: General (Show other bugs)
3.6.3.3
x86_64 Linux
unspecified Severity medium (vote)
: ovirt-4.0.2
: 4.0.2.1
Assigned To: Martin Betak
Nisim Simsolo
:
: 1336193 1358737 1395798 (view as bug list)
Depends On: 1306333
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-06 12:03 EST by Christoph
Modified: 2016-11-23 02:35 EST (History)
12 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-08-22 08:32:30 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.0.z+
rule-engine: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
engine.log and vdsm.log (1.56 MB, application/x-gzip)
2016-03-06 12:03 EST, Christoph
no flags Details
server.log (5.86 MB, text/plain)
2016-03-07 11:58 EST, Christoph
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 60661 master MERGED backend: Filter orphaned Host Devieces on refresh 2016-07-14 09:54 EDT
oVirt gerrit 60759 ovirt-engine-4.0 MERGED backend: Filter orphaned Host Devieces on refresh 2016-07-17 04:19 EDT

  None (edit)
Description Christoph 2016-03-06 12:03:52 EST
Created attachment 1133534 [details]
engine.log and vdsm.log

Hi,

I have a oVirt Setup which includes two host. The first host is also
running the oVirt engine.
Today I installed a couple VMs on the second host from USB without any
issue. Now I tried to perform this also on the first host but it looks
like the USB is not accessible.

I'm able to add the device to the host but I can see the following
message in the engine.log which does not look correct.

I'm running oVirt 3.6.3.4-1.el7.centos

Let me know if you need more information.

Best regards
Christoph 

oVirt User List Thread:
http://lists.ovirt.org/pipermail/users/2016-March/038276.html
Comment 1 Martin Perina 2016-03-07 00:51:18 EST
Martin, could you please take a look?
Comment 2 Martin Betak 2016-03-07 04:48:49 EST
Hi Christoph,

please provide server.log so we can see the underlying cause of the transaction failure.
Comment 3 Martin Betak 2016-03-07 04:51:09 EST
If I had to guess I'd say this is another manifestation of BZ#1306333 but let's wait for the server.log
Comment 4 Christoph 2016-03-07 11:58 EST
Created attachment 1133874 [details]
server.log

Hi,

please find the server.log in the attachment.

Best regards
Christoph
Comment 5 Martin Betak 2016-03-08 05:25:54 EST
Yes I can confirm this is a consequence of libvirt BZ#1306333 because it sends internaly inconsistent host devices to VDSM - more specifically the device named scsi_host8 is specified as parent of one device but not included in the listing.

Caused by: org.postgresql.util.PSQLException: ERROR: insert or update on table "host_device" violates foreign key constraint "fk_host_device_parent_name"
  Detail: Key (host_id, parent_device_name)=(548efa0a-d053-4189-b45d-a0134270573f, scsi_host8) is not present in table "host_device".

I believe the best current workaround is the restart of libvirt daemon, then the reported devices should become internally consistent again and the RefreshHostDevicesCommand should succeed.
Comment 6 Christoph 2016-03-08 08:16:47 EST
(In reply to Martin Betak from comment #5)
> Yes I can confirm this is a consequence of libvirt BZ#1306333 because it
> sends internaly inconsistent host devices to VDSM - more specifically the
> device named scsi_host8 is specified as parent of one device but not
> included in the listing.
> 
> Caused by: org.postgresql.util.PSQLException: ERROR: insert or update on
> table "host_device" violates foreign key constraint
> "fk_host_device_parent_name"
>   Detail: Key (host_id,
> parent_device_name)=(548efa0a-d053-4189-b45d-a0134270573f, scsi_host8) is
> not present in table "host_device".
> 
> I believe the best current workaround is the restart of libvirt daemon, then
> the reported devices should become internally consistent again and the
> RefreshHostDevicesCommand should succeed.

Hi Martin,

it is possible to do that without stopping the VMs or should I stop all VMs first.

Best regards
Christoph
Comment 7 Martin Betak 2016-03-08 10:30:55 EST
Hi Christoph,

I believe the proper *supported* way is to put the host to maintenance mode and only then you should restart some services on that host.


However.

----- BEGIN UNSUPPORTED ADVICE -----

In this instance my best judgement tells me this *should* be ok (restart just the libvirtd) and the system should survive this. (restarting vdsmd usually causes problems outside maint. mode), but note that in this case you are on your own.

----- END UNSUPPORTED ADVICE -----

Hope this helps :-)

Best regards,
Martin
Comment 8 Tomas Jelinek 2016-03-11 02:39:37 EST
any news?
Comment 9 Christoph 2016-03-11 02:48:58 EST
(In reply to Tomas Jelinek from comment #8)
> any news?

Hi Tomas,

I have to plan the shutdown for the VMs as they are in production.
Please let me know if it is important to get the feedback regarding the restart of the libvirt daemon.

If yes I could try to perform it on the weekend.

Best regards
Christoph
Comment 10 Tomas Jelinek 2016-03-11 03:10:27 EST
Hi Christoph,

no, not that important.
Comment 11 Martin Perina 2016-05-16 04:15:30 EDT
*** Bug 1336193 has been marked as a duplicate of this bug. ***
Comment 12 Martin Polednik 2016-07-13 03:07:09 EDT
After further consideration, the bug should be fixed at engine level. VDSM does not work with assumption that the devices are single tree, and it's not even feasible due to possible filtering by device capability. Trying to "fix" this inside VDSM would create special (incorrect) case for the API verb.
Comment 13 Michal Skrivanek 2016-07-22 02:28:29 EDT
*** Bug 1358737 has been marked as a duplicate of this bug. ***
Comment 14 Nisim Simsolo 2016-08-22 07:10:41 EDT
Verified: 
ovirt-engine-4.0.2.7-0.1.el7ev
qemu-kvm-rhev-2.3.0-31.el7_2.21.x86_64
vdsm-4.18.11-1.el7ev.x86_64
libvirt-client-1.2.17-13.el7_2.5.x86_64
sanlock-3.2.4-3.el7_2.x86_64

Verification scenario:
1. Insert USB DOK to first host, refresh host capabilities, verify USB DOK is listed in host devices, attach it to VM, run VM and verify USB DOK is accessible from the VM.
2. Disconnect USB DOK from first host, connect it to second host, refresh host capabilities, verify USB DOK is listed in host devices, attach it to VM, run VM and verify USB DOK is accessible from the VM.
Comment 15 Tomas Jelinek 2016-11-18 08:59:19 EST
*** Bug 1395798 has been marked as a duplicate of this bug. ***
Comment 16 Tomas Jelinek 2016-11-23 02:35:57 EST
*** Bug 1395798 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.