Bug 1396597
| Summary: | Libvirt should update device CGroups on hotplug | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Michal Privoznik <mprivozn> |
| Component: | libvirt | Assignee: | Michal Privoznik <mprivozn> |
| Status: | CLOSED ERRATA | QA Contact: | yisun |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.4 | CC: | dyuan, jdenemar, lmen, mprivozn, rbalakri, xuzhang, yanqzhan |
| Target Milestone: | rc | Keywords: | Upstream |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-2.5.0-1.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-08-01 17:19:14 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Patches proposed upstream: https://www.redhat.com/archives/libvir-list/2016-November/msg00854.html Moving to POST:
commit 5d9c2c708111835ee8e625a9c8a98353687c3fa5
Author: Michal Privoznik <mprivozn>
AuthorDate: Fri Nov 18 11:45:44 2016 +0100
Commit: Michal Privoznik <mprivozn>
CommitDate: Wed Nov 23 16:38:02 2016 +0100
qemu: Update cgroup on chardev hotplug
Just like in the previous commit, we are not updating CGroups on
chardev hot(un-)plug and thus leaving qemu unable to access any
non-default device users are trying to hotplug.
Signed-off-by: Michal Privoznik <mprivozn>
commit 085692c8bb9154f6f5e66b2e3fa150daef03fa19
Author: Michal Privoznik <mprivozn>
AuthorDate: Fri Nov 18 11:17:51 2016 +0100
Commit: Michal Privoznik <mprivozn>
CommitDate: Wed Nov 23 16:37:57 2016 +0100
qemu: Update cgroup on RNG hotplug
If users try to hotplug RNG device with a backend different to
/dev/random or /dev/urandom the whole operation fails as qemu is
unable to access the device. The problem is we don't update
device CGroups during the operation.
Signed-off-by: Michal Privoznik <mprivozn>
v2.4.0-196-g5d9c2c7
(In reply to Michal Privoznik from comment #2) > Moving to POST: > > commit 5d9c2c708111835ee8e625a9c8a98353687c3fa5 > Author: Michal Privoznik <mprivozn> > AuthorDate: Fri Nov 18 11:45:44 2016 +0100 > Commit: Michal Privoznik <mprivozn> > CommitDate: Wed Nov 23 16:38:02 2016 +0100 > > qemu: Update cgroup on chardev hotplug > > Just like in the previous commit, we are not updating CGroups on > chardev hot(un-)plug and thus leaving qemu unable to access any > non-default device users are trying to hotplug. > > Signed-off-by: Michal Privoznik <mprivozn> > > commit 085692c8bb9154f6f5e66b2e3fa150daef03fa19 > Author: Michal Privoznik <mprivozn> > AuthorDate: Fri Nov 18 11:17:51 2016 +0100 > Commit: Michal Privoznik <mprivozn> > CommitDate: Wed Nov 23 16:37:57 2016 +0100 > > qemu: Update cgroup on RNG hotplug > > If users try to hotplug RNG device with a backend different to > /dev/random or /dev/urandom the whole operation fails as qemu is > unable to access the device. The problem is we don't update > device CGroups during the operation. > > Signed-off-by: Michal Privoznik <mprivozn> > > > v2.4.0-196-g5d9c2c7 Hi Michal, I saw you update chardev related code, but during my try. I just cannot hotplug some typical chardev to a running vm. ## cat g <graphics type='spice' autoport='yes'> <listen type='address'/> <image compression='off'/> </graphics> ## virsh attach-device r g error: Failed to attach device from g error: Operation not supported: live attach of device 'graphics' is not supported ## cat c <console type='pty'> <target type='serial' port='0'/> </console> ## virsh attach-device r c error: Failed to attach device from c error: Operation not supported: attaching serial console is not supported So what kind of chardev you'll suggest to be tested? and do you think we'll support such kind of hotplug in future(maybe a rfe could be open)? thx (In reply to yisun from comment #4) > (In reply to Michal Privoznik from comment #2) > > Moving to POST: > > > > commit 5d9c2c708111835ee8e625a9c8a98353687c3fa5 > > Author: Michal Privoznik <mprivozn> > > AuthorDate: Fri Nov 18 11:45:44 2016 +0100 > > Commit: Michal Privoznik <mprivozn> > > CommitDate: Wed Nov 23 16:38:02 2016 +0100 > > > > qemu: Update cgroup on chardev hotplug > > > > Just like in the previous commit, we are not updating CGroups on > > chardev hot(un-)plug and thus leaving qemu unable to access any > > non-default device users are trying to hotplug. > > > > Signed-off-by: Michal Privoznik <mprivozn> > > > > commit 085692c8bb9154f6f5e66b2e3fa150daef03fa19 > > Author: Michal Privoznik <mprivozn> > > AuthorDate: Fri Nov 18 11:17:51 2016 +0100 > > Commit: Michal Privoznik <mprivozn> > > CommitDate: Wed Nov 23 16:37:57 2016 +0100 > > > > qemu: Update cgroup on RNG hotplug > > > > If users try to hotplug RNG device with a backend different to > > /dev/random or /dev/urandom the whole operation fails as qemu is > > unable to access the device. The problem is we don't update > > device CGroups during the operation. > > > > Signed-off-by: Michal Privoznik <mprivozn> > > > > > > v2.4.0-196-g5d9c2c7 > Hi Michal, > I saw you update chardev related code, but during my try. I just cannot > hotplug some typical chardev to a running vm. > > ## cat g > <graphics type='spice' autoport='yes'> > <listen type='address'/> > <image compression='off'/> > </graphics> This is not a chardev nor RNG. Live hotplug of graphics is not implemented as qemu doesn't support it yet. > > ## virsh attach-device r g > error: Failed to attach device from g > error: Operation not supported: live attach of device 'graphics' is not > supported > > ## cat c > <console type='pty'> > <target type='serial' port='0'/> > </console> Hotplug for chardevs is supported only for virtio. (In reply to Michal Privoznik from comment #5) > (In reply to yisun from comment #4) > > (In reply to Michal Privoznik from comment #2) > > > Moving to POST: > > > > > > commit 5d9c2c708111835ee8e625a9c8a98353687c3fa5 > > > Author: Michal Privoznik <mprivozn> > > > AuthorDate: Fri Nov 18 11:45:44 2016 +0100 > > > Commit: Michal Privoznik <mprivozn> > > > CommitDate: Wed Nov 23 16:38:02 2016 +0100 > > > > > > qemu: Update cgroup on chardev hotplug > > > > > > Just like in the previous commit, we are not updating CGroups on > > > chardev hot(un-)plug and thus leaving qemu unable to access any > > > non-default device users are trying to hotplug. > > > > > > Signed-off-by: Michal Privoznik <mprivozn> > > > > > > commit 085692c8bb9154f6f5e66b2e3fa150daef03fa19 > > > Author: Michal Privoznik <mprivozn> > > > AuthorDate: Fri Nov 18 11:17:51 2016 +0100 > > > Commit: Michal Privoznik <mprivozn> > > > CommitDate: Wed Nov 23 16:37:57 2016 +0100 > > > > > > qemu: Update cgroup on RNG hotplug > > > > > > If users try to hotplug RNG device with a backend different to > > > /dev/random or /dev/urandom the whole operation fails as qemu is > > > unable to access the device. The problem is we don't update > > > device CGroups during the operation. > > > > > > Signed-off-by: Michal Privoznik <mprivozn> > > > > > > > > > v2.4.0-196-g5d9c2c7 > > Hi Michal, > > I saw you update chardev related code, but during my try. I just cannot > > hotplug some typical chardev to a running vm. > > > > ## cat g > > <graphics type='spice' autoport='yes'> > > <listen type='address'/> > > <image compression='off'/> > > </graphics> > > This is not a chardev nor RNG. Live hotplug of graphics is not implemented > as qemu doesn't support it yet. > > > > > ## virsh attach-device r g > > error: Failed to attach device from g > > error: Operation not supported: live attach of device 'graphics' is not > > supported > > > > ## cat c > > <console type='pty'> > > <target type='serial' port='0'/> > > </console> > > Hotplug for chardevs is supported only for virtio. thx for the info. I tried with a virtio console, and seems it cannot be connected if hotplugged, but can be connected with cold plug, pls have a check. 1. make sure no console setup for vm ## virsh console vm1 Connected to domain vm1 Escape character is ^] error: internal error: cannot find character device <null> 2. prepare a console xml ## cat vcon <console type='pty'> <target type='virtio' port='0'/> </console> 3. do the hot plug ## virsh attach-device vm1 vcon Device attached successfully 4. try to connect to the console ## virsh console vm1 Connected to domain vm1 Escape character is ^] <=== just hanging there ~~ well, if I cold plug the console with "virsh edit" 1. edit vm's xml before starting it ## virsh dumpxml vm1 --inactive | grep console -a2 ... <console type='pty'> <target type='virtio' port='0'/> </console> ## virsh start vm1 Domain vm1 started 2. try to connect to the console ## virsh console vm1 Connected to domain vm1 Escape character is ^] Red Hat Enterprise Linux Server 7.3 Beta (Maipo) Kernel 3.10.0-461.el7.x86_64 on an x86_64 localhost login: <=== can connect to guest console (In reply to yisun from comment #6) > > thx for the info. > I tried with a virtio console, and seems it cannot be connected if > hotplugged, but can be connected with cold plug, pls have a check. > > 1. make sure no console setup for vm > ## virsh console vm1 > Connected to domain vm1 > Escape character is ^] > error: internal error: cannot find character device <null> > > 2. prepare a console xml > ## cat vcon > <console type='pty'> > <target type='virtio' port='0'/> > </console> > > 3. do the hot plug > ## virsh attach-device vm1 vcon > Device attached successfully > > > 4. try to connect to the console > ## virsh console vm1 > Connected to domain vm1 > Escape character is ^] > <=== just hanging there ~~ This is because guest OS hasn't spawned any getty process on the console. You can either try running it by hand (from say SSH session), or just verify there's new /dev/vportXpY after you've hotplugged the console. seems needs to needinfo you again~~
I tested rng hotplug on:
libvirt-3.2.0-7.el7.x86_64
qemu-kvm-rhev-2.9.0-7.el7.x86_64
kernel-3.10.0-660.el7.x86_64
I tried to attach a rng device with backing device = /dev/sdd and it's failed.
Steps:
## ll /dev/sdd
brw-rw----. 1 root disk 8, 48 Jun 1 15:28 /dev/sdd
## cat rng
<rng model='virtio'>
<rate bytes='1234' period='2000'/>
<backend model='random'>/dev/sdd</backend>
</rng>
## virsh attach-device vm1 rng
error: Failed to attach device from rng
error: internal error: unable to execute QEMU command 'object-add': Could not open '/dev/sdd': Permission denied
<==== here still a permission denied error
well, if I manually chmod the /dev/sdd to a+rw, it can be attached but seems not what you wanted to implement.
## chmod a+rw /dev/sdd
## ll /dev/sdd
brw-rw-rw-. 1 root disk 8, 48 Jun 1 15:28 /dev/sdd
## virsh attach-device vm1 rng
Device attached successfully
(In reply to yisun from comment #8) > seems needs to needinfo you again~~ > I tested rng hotplug on: > libvirt-3.2.0-7.el7.x86_64 > qemu-kvm-rhev-2.9.0-7.el7.x86_64 > kernel-3.10.0-660.el7.x86_64 > > I tried to attach a rng device with backing device = /dev/sdd and it's > failed. > Steps: > ## ll /dev/sdd > brw-rw----. 1 root disk 8, 48 Jun 1 15:28 /dev/sdd > > ## cat rng > <rng model='virtio'> > <rate bytes='1234' period='2000'/> > <backend model='random'>/dev/sdd</backend> > </rng> > > ## virsh attach-device vm1 rng > error: Failed to attach device from rng > error: internal error: unable to execute QEMU command 'object-add': Could > not open '/dev/sdd': Permission denied > <==== here still a permission denied error > > well, if I manually chmod the /dev/sdd to a+rw, it can be attached but seems > not what you wanted to implement. In fact, this is what we really want to have. RNGs are special type of devices. Usually, you don't have /dev/sdd as a RNG source but /dev/random or /dev/hwrng. Both these are meant to be shared with other applications on the system. Therefore if libvirt would label them, other apps might lose access. So we don't. verified on:
libvirt-3.2.0-7.el7.x86_64
1. chardev
in vm:
# ll /dev/vport0p*
crw-------. 1 root root 246, 0 Jun 2 19:58 /dev/vport0p0
crw-------. 1 root root 246, 1 Jun 2 19:58 /dev/vport0p1
crw-------. 1 root root 246, 5 Jun 2 19:58 /dev/vport0p5
in host:
## cat vport
<console type='pty'>
<target type='virtio' port='0'/>
</console>
## virsh attach-device vm1 vport
Device attached successfully
in vm:
# ll /dev/vport0p*
crw-------. 1 root root 246, 0 Jun 2 19:58 /dev/vport0p0
crw-------. 1 root root 246, 1 Jun 2 19:58 /dev/vport0p1
crw-------. 1 root root 246, 2 Jun 2 19:59 /dev/vport0p2 ** newly added
crw-------. 1 root root 246, 5 Jun 2 19:58 /dev/vport0p5
2. rng dev
in vm:
# ll /dev | grep -i rng
<== nothing
in host:
## ll /dev/sdd
brw-rw-rw-. 1 root disk 8, 48 Jun 1 15:28 /dev/sdd
## cat rng
<rng model='virtio'>
<rate bytes='1234' period='2000'/>
<backend model='random'>/dev/sdd</backend>
</rng>
## virsh attach-device vm1 rng
Device attached successfully
in guest:
# ll /dev | grep -i rng
crw-------. 1 root root 10, 183 Jun 2 20:02 hwrng ** newly added
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 |
Description of problem: While working on a different issue, I've came across device hotplug code in QEMU driver. Trouble is, that for some devices we don't whitelist them in device CGroup corresponding to the domain. I've identified RNG and chardev. Version-Release number of selected component (if applicable): Any How reproducible: 100% Steps to Reproduce: 1. Consider the following RNG: <rng model='virtio'> <rate bytes='1234' period='2000'/> <backend model='random'>/dev/sdb</backend> </rng> Yes, /dev/sdb is really crappy random source, but it will serve its purpose here. 2. Try to hotplug it to a domain. 3. Observe qemu denied access. Actual results: Device can't be hotplugged. Expected results: Device is hotplugged. Additional info: