Bug 871196

Summary:	qemu: could not load kernel ... Permission denied
Product:	[Community] Virtualization Tools	Reporter:	Richard W.M. Jones <rjones>
Component:	libvirt	Assignee:	Libvirt Maintainers <libvirt-maint>
Status:	CLOSED DUPLICATE	QA Contact:	Michal Privoznik <mprivozn>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	berrange, clalancette, crobinso, itamar, jforbes, laine, libvirt-maint, mprivozn, rbalakri, rjones, veillard, virt-maint
Target Milestone:	---	Keywords:	Reopened
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	921135 (view as bug list)		Environment:
Last Closed:	2016-01-15 20:22:00 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	910269, 910270, 922891

Description Richard W.M. Jones 2012-10-29 21:31:44 UTC

Description of problem:

This manifests itself as the libguestfs test suite
occasionally failing like this:

*stdin*:2: libguestfs: error: could not create appliance through libvirt: internal error process exited while connecting to monitor: 2012-10-29 21:27:55.614+0000: 20366: debug : virFileClose:72 : Closed fd 22
2012-10-29 21:27:55.614+0000: 20366: debug : virFileClose:72 : Closed fd 35
2012-10-29 21:27:55.630+0000: 20366: debug : virFileClose:72 : Closed fd 3
2012-10-29 21:27:55.653+0000: 20367: debug : virCommandHook:2143 : Hook is done 0
qemu: could not load kernel '/home/rjones/d/libguestfs/tmp/.guestfs-1000/kernel.20244': Permission denied
 [code=1 domain=10]

There are multiple parallel libguestfs instances running.
The best explanation of this I can come up with is that
libvirtd is unlabelling the <kernel/> element at the same
time that another qemu is trying to open it.

Version-Release number of selected component (if applicable):

libvirt-0.10.2-3.fc18.x86_64

How reproducible:

Very rare.

Comment 1 Richard W.M. Jones 2013-03-13 11:12:22 UTC

Still happening with libguestfs upstream and recent libvirt.

libvirt-daemon-1.0.2-3.fc19.x86_64

A test program which demonstrates this and nearly always
fails for me is here:

https://github.com/libguestfs/libguestfs/blob/master/tests/parallel/test-parallel.c

Comment 2 Michal Privoznik 2013-03-13 14:46:43 UTC

I believe this can be solved with my "Keep original file label" patchset:

https://www.redhat.com/archives/libvir-list/2013-March/msg00497.html

Although I am solving this issue for DAC only for now. I am introducing reference counter on XATTR level to trace how many times did libvirt label a file, so it the label is restored only at the last restore request. So if your explanation is right, the kernel image will be:

1. labeled due to domain A startup, refCount = 1,
2. labeled due to domain B startup, refCount = 2,
3. domain A shutdown will just decrease refCount to 1,
4. shutting down domain B will decrease refCount and since it's 0 now, the original file label is restored.

Comment 3 Richard W.M. Jones 2013-10-17 12:16:45 UTC

Just seen this again with libvirt-daemon-1.1.3-2.fc21.x86_64
when running the highly parallel virt-df test in the
libguestfs test suite.

Comment 4 Cole Robinson 2015-04-25 23:07:47 UTC

This bug is pretty old... Rich do you still see it on occasion?

Comment 5 Richard W.M. Jones 2015-04-26 08:15:19 UTC

Not recently.

Comment 6 Richard W.M. Jones 2016-01-13 10:02:44 UTC

Reopening as this still happens with libvirt-1.3.0-1.fc24

Comment 7 Richard W.M. Jones 2016-01-15 20:22:00 UTC


*** This bug has been marked as a duplicate of bug 1269975 ***