RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 923946 - RHEL-6.4: migration is failing: libvirtError: XML error: missing security model when using multiple labels
Summary: RHEL-6.4: migration is failing: libvirtError: XML error: missing security mod...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.4
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 896690 928879
TreeView+ depends on / blocked
 
Reported: 2013-03-20 18:44 UTC by Haim
Modified: 2014-01-13 00:56 UTC (History)
19 users (show)

Fixed In Version: libvirt-0.10.2-19.el6
Doc Type: Bug Fix
Doc Text:
When SELinux was disabled on a host, or the qemu driver was configured not to use it, and the domain XML configuration contained an explicit seclabel option, the code parsed the seclabel option, but ignored it later on when it was generating labels on domain start, and created a new and empty seclabel entry [seclabeltype='none'/]. Consequently, a migration between two hosts running Red Hat Enterprise Linux 6.4 failed with the following error message: libvirtError: XML error: missing security model when using multiple labels With this update, if a seclabel entry already exists, a new one is no longer created, and the migration works as expected in the described scenario
Clone Of:
Environment:
Last Closed: 2013-11-21 08:52:32 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
libvirt and others. (17.82 MB, application/x-tar)
2013-03-20 18:49 UTC, Haim 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1581 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2013-11-21 01:11:35 UTC

Description Haim 2013-03-20 18:44:33 UTC 
Description of problem:

migration between host running libvirt libvirt-0.10.2-18 to host running libvirt- 0.10.2-18.el6_4.2 is failing with the following error:

Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 270, in run
    self._startUnderlyingMigration()
  File "/usr/share/vdsm/libvirtvm.py", line 493, in _startUnderlyingMigration
    None, maxBandwidth)
  File "/usr/share/vdsm/libvirtvm.py", line 529, in f
    ret = attr(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 104, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1178, in migrateToURI2
    if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self)
libvirtError: XML error: missing security model when using multiple labels

repro steps:

1) have a VM running on source running latest release of libvirt
2) migrate VM from source to dest running libvirt's RHEL6.4.z version
Comment 1 Haim 2013-03-20 18:49:20 UTC 
Created attachment 713416 [details]
libvirt and others.
Comment 3 Jiri Denemark 2013-03-21 10:22:56 UTC 
So the root cause of this problem is in libvirt's security driver. In case SELinux is disabled on the host or qemu driver is configured not to use it (security_driver = "none" in /etc/libvirt/qemu.conf) and domain XML contains explicit seclabel (<seclabel type='dynamic' model='selinux' relabel='yes'/>). The code parses the seclabel but ignores it later on when it is generating labels on domain start and create new <seclabel type='none'/>, which is only valid if no other seclabel elements are present. As a result, even restarting libvirtd is enough to observe the bug, since the domain (while still running) will not be listed as running by libvirt.

Luckily, this bug was not introduced in the 6.4.z build. However, it is a regression since 6.3.

Moreover, I think that a domain with explicitly configured seclabel should fail to start if libvirt is not able to provide that seclabel (because the corresponding security driver is disabled). However, this behavior is consistent with 6.3.
Comment 5 Jiri Denemark 2013-03-21 10:35:39 UTC 
Package version: libvirt-0.10.2-18

Steps:
1. set security_driver = "none" in /etc/libvirt/qemu.conf (or alternatively disable SELinux on the host)
2. restart libvirtd service (or the host in case the alternative route in step 1 was chosen)
3. start a domain which contains <seclabel type='dynamic' model='selinux' relabel='yes'/> in its XML configuration
4. restart libvirtd service
5. the domain disappears from virsh list and "XML error: security label is missing" error can be found in the logs when libvirtd is trying to parse the domain's XML config from /var/run/libvirt/qemu

Alternatively, instead of the steps 4 and 5 above, one can do:
4. managedsave DOM
5. start DOM

in this case step 5 fails with "XML error: missing security model when using multiple labels".
Comment 7 Michal Privoznik 2013-03-21 10:47:10 UTC 
Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2013-March/msg01217.html
Comment 9 Michal Privoznik 2013-03-28 09:14:28 UTC 
Patch is not pushed upstream:

commit e4a28a328188e715de8d587b390cb44e2d6c3fb4
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Mar 21 16:32:07 2013 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Mar 28 10:01:06 2013 +0100

    security: Don't add seclabel of type none if there's already a seclabel
    
    https://bugzilla.redhat.com/show_bug.cgi?id=923946
    
    The <seclabel type='none'/> should be added iff there is no other
    seclabel defined within a domain. This bug can be easily reproduced:
    1) configure selinux seclabel for a domain
    2) disable system's selinux and restart libvirtd
    3) observe <seclabel type='none'/> being appended to a domain on its
       startup

commit 6c4de1161425a610797495549349d194b90fb023
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Mar 21 16:12:55 2013 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Mar 28 10:01:06 2013 +0100

    security_manager: Don't manipulate domain XML in virDomainDefGetSecurityLabelDef
    
    The virDomainDefGetSecurityLabelDef was modifying the domain XML.
    It tried to find a seclabel corresponding to given sec driver. If the
    label wasn't found, the function created one which is wrong. In fact
    it's security manager which should modify this part of domain XML.

v1.0.4-rc2-5-ge4a28a3
Comment 11 Eric Blake 2013-04-01 16:04:06 UTC 
(In reply to comment #3)
> So the root cause of this problem is in libvirt's security driver. In case
> SELinux is disabled on the host or qemu driver is configured not to use it
> (security_driver = "none" in /etc/libvirt/qemu.conf) and domain XML contains
> explicit seclabel (<seclabel type='dynamic' model='selinux'
> relabel='yes'/>). The code parses the seclabel but ignores it later on when
> it is generating labels on domain start and create new <seclabel
> type='none'/>, which is only valid if no other seclabel elements are
> present. As a result, even restarting libvirtd is enough to observe the bug,
> since the domain (while still running) will not be listed as running by
> libvirt.
> 
> Luckily, this bug was not introduced in the 6.4.z build. However, it is a
> regression since 6.3.

There is an upstream report of someone hitting something that sounds very much like this problem, but while while using libvirt 0.10.2 for migration (which means 6.3 is probably also impacted):
https://www.redhat.com/archives/libvirt-users/2013-April/msg00003.html
Comment 12 Jiri Denemark 2013-04-02 07:06:55 UTC 
(In reply to comment #11)
> There is an upstream report of someone hitting something that sounds very
> much like this problem, but while while using libvirt 0.10.2 for migration
> (which means 6.3 is probably also impacted):

0.10.2 is actually 6.4 (6.3 had 0.9.10). We did not have any support for multiple security labels in 6.3.
Comment 13 Michal Privoznik 2013-04-02 10:11:06 UTC 
As we've found out in bug 928879 the summary does not correspond to the behaviour described in report. Both are bugs, but a different ones. I am changing the summary back to he original one (slightly modified) and for the other bug I've reported bug 947387.
Comment 15 yanbing du 2013-07-09 11:23:57 UTC 
I'm testing this bug, and following steps in comment5, the results are:

1. set security_driver = "none" in /etc/libvirt/qemu.conf (or alternatively disable SELinux on the host)
2. restart libvirtd service (or the host in case the alternative route in step 1 was chosen)
3. start a domain which contains <seclabel type='dynamic' model='selinux' relabel='yes'/> in its XML configuration
4. restart libvirtd service

then the running domian will shutoff, and error in libvirtd.log:
...
013-07-09 11:13:41.161+0000: 23141: info : libvirt version: 0.10.2, package: 19.el6 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2013-07-08-04:51:26, x86-023.build.eng.bos.redhat.com)
2013-07-09 11:13:41.161+0000: 23141: warning : virSecurityManagerNew:148 : Configured security driver "none" disables default policy to create confined guests
 2013-07-09 11:13:41.375+0000: 23141: error : virSecurityLabelDefParseXML:3226 : XML error: security label is missing

OR
4. managedsave DOM
# virsh managedsave tsu-1

Domain tsu-1 state saved by libvirt

5. start DOM
# virsh start tsu-1
Domain tsu-1 started

domain can start, but there are many cgroup error in libvirtd.log:
13-07-09 11:14:39.817+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/cpu/libvirt/qemu/tsu-1//emulator (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/cpu/libvirt/qemu/tsu-1/ (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/cpuacct/libvirt/qemu/tsu-1//emulator (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/cpuacct/libvirt/qemu/tsu-1/ (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/cpuset/libvirt/qemu/tsu-1//emulator (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/cpuset/libvirt/qemu/tsu-1/ (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/memory/libvirt/qemu/tsu-1/ (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/devices/libvirt/qemu/tsu-1/ (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/freezer/libvirt/qemu/tsu-1/ (16)
2013-07-09 11:14:39.840+0000: 23134: error : virCgroupRemoveRecursively:730 : Unable to remove /cgroup/blkio/libvirt/qemu/tsu-1/ (16)


I'm not sure if the result can verify this bug, so need your confirm, thanks!
Comment 16 Michal Privoznik 2013-07-10 12:28:32 UTC 
(In reply to yanbing du from comment #15)
> I'm testing this bug, and following steps in comment5, the results are:
> 
> 1. set security_driver = "none" in /etc/libvirt/qemu.conf (or alternatively
> disable SELinux on the host)
> 2. restart libvirtd service (or the host in case the alternative route in
> step 1 was chosen)
> 3. start a domain which contains <seclabel type='dynamic' model='selinux'
> relabel='yes'/> in its XML configuration

Can you provide virsh dumpxml  after this step?

> 4. restart libvirtd service
> 
> then the running domian will shutoff, and error in libvirtd.log:
> ...
> 013-07-09 11:13:41.161+0000: 23141: info : libvirt version: 0.10.2, package:
> 19.el6 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>,
> 2013-07-08-04:51:26, x86-023.build.eng.bos.redhat.com)
> 2013-07-09 11:13:41.161+0000: 23141: warning : virSecurityManagerNew:148 :
> Configured security driver "none" disables default policy to create confined
> guests
>  2013-07-09 11:13:41.375+0000: 23141: error :
> virSecurityLabelDefParseXML:3226 : XML error: security label is missing
> 
> OR
> 4. managedsave DOM
> # virsh managedsave tsu-1
> 
> Domain tsu-1 state saved by libvirt
> 
> 5. start DOM
> # virsh start tsu-1
> Domain tsu-1 started
Comment 17 yanbing du 2013-07-11 03:02:13 UTC 
(In reply to Michal Privoznik from comment #16)
> (In reply to yanbing du from comment #15)
> > I'm testing this bug, and following steps in comment5, the results are:
> > 
> > 1. set security_driver = "none" in /etc/libvirt/qemu.conf (or alternatively
> > disable SELinux on the host)
> > 2. restart libvirtd service (or the host in case the alternative route in
> > step 1 was chosen)
> > 3. start a domain which contains <seclabel type='dynamic' model='selinux'
> > relabel='yes'/> in its XML configuration
> 
> Can you provide virsh dumpxml  after this step?

# virsh start test
Domain test started

# virsh dumpxml test|grep seclabel

  <seclabel type='dynamic' model='selinux' relabel='yes'/>

If delete the <seclabel/> element and start the guest:
# virsh dumpxml test|grep seclabel
  <seclabel type='none'/>

then restart libvirtd, the guest will still running and an warning message in libvirtd.log: 
2013-07-10 18:50:24.670+0000: 48362: warning : virSecurityManagerNew:148 : Configured security driver "none" disables default policy to create confined guests

> 
> > 4. restart libvirtd service
> > 
> > then the running domian will shutoff, and error in libvirtd.log:
> > ...
> > 013-07-09 11:13:41.161+0000: 23141: info : libvirt version: 0.10.2, package:
> > 19.el6 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>,
> > 2013-07-08-04:51:26, x86-023.build.eng.bos.redhat.com)
> > 2013-07-09 11:13:41.161+0000: 23141: warning : virSecurityManagerNew:148 :
> > Configured security driver "none" disables default policy to create confined
> > guests
> >  2013-07-09 11:13:41.375+0000: 23141: error :
> > virSecurityLabelDefParseXML:3226 : XML error: security label is missing
> > 
> > OR
> > 4. managedsave DOM
> > # virsh managedsave tsu-1
> > 
> > Domain tsu-1 state saved by libvirt
> > 
> > 5. start DOM
> > # virsh start tsu-1
> > Domain tsu-1 started
Comment 18 Michal Privoznik 2013-07-12 07:43:53 UTC 
(In reply to yanbing du from comment #17)
> (In reply to Michal Privoznik from comment #16)
> > (In reply to yanbing du from comment #15)
> > > I'm testing this bug, and following steps in comment5, the results are:
> > > 
> > > 1. set security_driver = "none" in /etc/libvirt/qemu.conf (or alternatively
> > > disable SELinux on the host)
> > > 2. restart libvirtd service (or the host in case the alternative route in
> > > step 1 was chosen)
> > > 3. start a domain which contains <seclabel type='dynamic' model='selinux'
> > > relabel='yes'/> in its XML configuration
> > 
> > Can you provide virsh dumpxml  after this step?
> 
> # virsh start test
> Domain test started
> 
> # virsh dumpxml test|grep seclabel
> 
>   <seclabel type='dynamic' model='selinux' relabel='yes'/>
> 
> If delete the <seclabel/> element and start the guest:
> # virsh dumpxml test|grep seclabel
>   <seclabel type='none'/>
> 
> then restart libvirtd, the guest will still running and an warning message
> in libvirtd.log: 
> 2013-07-10 18:50:24.670+0000: 48362: warning : virSecurityManagerNew:148 :
> Configured security driver "none" disables default policy to create confined
> guests
> 

Right, thank you for confirmation. I think this is a separate issue - see bug 947387. So I think this bug is fixed.
Comment 19 yanbing du 2013-07-12 13:56:59 UTC 
Thanks Michal, then this bug can move to VERIFIED.
Comment 21 errata-xmlrpc 2013-11-21 08:52:32 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1581.html
Note You need to log in before you can comment on or make changes to this bug.