Hide Forgot
Description of problem: virsh command domiftune bound parameter checking error (found by virt-test). Version-Release number of selected component (if applicable): libvirt-1.1.1-14.el7.x86_64 How reproducible: 100% Steps to Reproduce: With a vm test-vm, 1. Get vm's interface MAC address (52:54:00:dc:dd:de in this case). # virsh domiflist test-vm Interface Type Source Model MAC ------------------------------------------------------- vnet0 bridge virbr0 virtio 52:54:00:dc:dd:de 2. Change vm's interface bound with different values (XXXXXX). # virsh domiftune test-vm 52:54:00:dc:dd:de --inbound XXXXXX 3. Check change result. # virsh domiftune test-vm 52:54:00:dc:dd:de inbound.average: XXXXXX inbound.peak : 0 inbound.burst : 0 outbound.average: 0 outbound.peak : 0 outbound.burst : 0 Actual results: if XXXXXX < 4294968: Successfully changed. elif 4294968 <= XXXXXXX < 4294967296: Return: error: Unable to set interface parameters error: internal error: cannot set bandwidth limits on 52:54:00:dc:dd:de elif (the first 10 digits of XXXXXX > 4294967296) (Yes, this is a strange condition): Successfully changed to a number which is first 10 digit of XXXXXX minus 4294967296, then concatenate its remaining part. elif XXXXXX < 0: XXXXXX = complement of XXXXXX goto start Expected results: No funny behavior and if XXXXXX < 0 set to unlimited might be a reasonable behavior.
Upstream patch: https://www.redhat.com/archives/libvir-list/2014-July/msg01396.html
Now pushed upstream: commit 6dac5d06f5c155f7db679d3105ad7dbcc8d7d78f Author: Ján Tomko <jtomko> CommitDate: 2014-08-04 16:59:28 +0200 Don't overwrite errors from virNetDevBandwidthSet Otherwise this beautiful error would be overwritten when the function is called with a really high rate number: 2014-07-28 12:51:47.920+0000: 2304: error : virCommandWait:2399 : internal error: Child process (/sbin/tc class add dev vnet0 parent 1: classid 1:1 htb rate 4294968kbps) unexpected exit status 1: Illegal "rate" Usage: ... qdisc add ... htb [default N] [r2q N] default minor id of class to which unclassified packets are sent {0} r2q DRR quantums are computed as rate in Bps/r2q {10} debug string of 16 numbers each 0-3 {0} ... class add ... htb rate R1 [burst B1] [mpu B] [overhead O] [prio P] [slot S] [pslot PS] [ceil R2] [cburst B2] [mtu MTU] [quantum Q] rate rate allocated to this class (class can still borrow) burst max bytes burst which can be accumulated during idle period {computed} mpu minimum packet size used in rate computations overhead per-packet size overhead used in rate computations linklay adapting to a linklayer e.g. atm ceil definite upper class rate (no borrows) {rate} cburst burst but for ceil {computed} mtu max packet size we create rate map for {1600} prio priority of leaf; lowe https://bugzilla.redhat.com/show_bug.cgi?id=1043735 commit ee668206cd87afffa26e0fcd49d16b8889ffd869 Author: Ján Tomko <jtomko> CommitDate: 2014-08-04 16:59:27 +0200 virsh: check if domiftune parameters fit into UINT We parse the bandwidth rates as unsinged long long, then try to fit them in VIR_TYPED_PARAM_UINT. Report an error if they exceed UINT_MAX instead of quietly using wrong values. https://bugzilla.redhat.com/show_bug.cgi?id=1043735 git describe: v1.2.7-7-g6dac5d0
Verify this issue with build libvirt-1.2.8-1.el7.x86_64 # virsh domiflist r6 Interface Type Source Model MAC ------------------------------------------------------- vnet0 network default e1000 52:54:00:cc:a3:82 # virsh domiftune r6 52:54:00:cc:a3:82 --inbound XXXXXX error: inbound format is incorrect # virsh domiftune r6 52:54:00:cc:a3:82 --inbound 4294968 error: Unable to set interface parameters error: internal error: Child process (/sbin/tc class add dev vnet0 parent 1: classid 1:1 htb rate 4294968kbps) unexpected exit status 1: Illegal "rate" Usage: ... qdisc add ... htb [default N] [r2q N] default minor id of class to which unclassified packets are sent {0} r2q DRR quantums are computed as rate in Bps/r2q {10} debug string of 16 numbers each 0-3 {0} ... class add ... htb rate R1 [burst B1] [mpu B] [overhead O] [prio P] [slot S] [pslot PS] [ceil R2] [cburst B2] [mtu MTU] [quantum Q] rate rate allocated to this class (class can still borrow) burst max bytes burst which can be accumulated during idle period {computed} mpu minimum packet size used in rate computations overhead per-packet size overhead used in rate computations linklay adapting to a linklayer e.g. atm ceil definite upper class rate (no borrows) {rate} cburst burst but for ceil {computed} mtu max packet size we create rate map for {1600} prio priority of leaf; lowe # virsh domiftune r6 52:54:00:cc:a3:82 --inbound 4294969 error: Unable to set interface parameters error: internal error: Child process (/sbin/tc class add dev vnet0 parent 1: classid 1:1 htb rate 4294969kbps) unexpected exit status 1: Illegal "rate" Usage: ... qdisc add ... htb [default N] [r2q N] default minor id of class to which unclassified packets are sent {0} r2q DRR quantums are computed as rate in Bps/r2q {10} debug string of 16 numbers each 0-3 {0} ... class add ... htb rate R1 [burst B1] [mpu B] [overhead O] [prio P] [slot S] [pslot PS] [ceil R2] [cburst B2] [mtu MTU] [quantum Q] rate rate allocated to this class (class can still borrow) burst max bytes burst which can be accumulated during idle period {computed} mpu minimum packet size used in rate computations overhead per-packet size overhead used in rate computations linklay adapting to a linklayer e.g. atm ceil definite upper class rate (no borrows) {rate} cburst burst but for ceil {computed} mtu max packet size we create rate map for {1600} prio priority of leaf; lowe # virsh domiftune r6 52:54:00:cc:a3:82 --inbound 4294967 # virsh domiftune r6 52:54:00:cc:a3:82 inbound.average: 4294967 inbound.peak : 0 inbound.burst : 0 outbound.average: 0 outbound.peak : 0 outbound.burst : 0 # virsh domiftune r6 52:54:00:cc:a3:82 --inbound 1 # virsh domiftune r6 52:54:00:cc:a3:82 inbound.average: 1 inbound.peak : 0 inbound.burst : 0 outbound.average: 0 outbound.peak : 0 outbound.burst : 0 # virsh domiftune r6 52:54:00:cc:a3:82 --inbound 0 # virsh domiftune r6 52:54:00:cc:a3:82 inbound.average: 0 inbound.peak : 0 inbound.burst : 0 outbound.average: 0 outbound.peak : 0 outbound.burst : 0 # virsh domiftune r6 52:54:00:cc:a3:82 --inbound -1 error: inbound rate larger than maximum 4294967295 # virsh domiftune r6 52:54:00:cc:a3:82 --inbound 4294967296 error: inbound rate larger than maximum 4294967295 # virsh domiftune r6 52:54:00:cc:a3:82 --inbound 4294967297 error: inbound rate larger than maximum 4294967295
*** Bug 1142816 has been marked as a duplicate of this bug. ***
Hi, Jan I try to verify this bug on build libvirt-1.2.8-8.el7.x86_64 But I found that when setting inbound or outbound value between 4294968 and 4294967296, libvirt check that they are illegal and report with error. But the reported error seems not accurate, could you please help us check that? verify steps: 1. prepare a guest in running and list interface # virsh domiflist rhel7new Interface Type Source Model MAC ------------------------------------------------------- vnet1 network default rtl8139 52:54:00:c2:c6:e4 # virsh list --all Id Name State ---------------------------------------------------- 30 rhel7new running 2. set inbound with domiftune command # virsh domiftune rhel7new 52:54:00:c2:c6:e4 --inbound 0 [root@server ~]# virsh domiftune rhel7new 52:54:00:c2:c6:e4 inbound.average: 0 inbound.peak : 0 inbound.burst : 0 outbound.average: 4294967 outbound.peak : 0 outbound.burst : 0 # virsh domiftune rhel7new 52:54:00:c2:c6:e4 --inbound 1 # virsh domiftune rhel7new 52:54:00:c2:c6:e4 inbound.average: 1 inbound.peak : 0 inbound.burst : 0 outbound.average: 4294967 outbound.peak : 0 outbound.burst : 0 # virsh domiftune rhel7new 52:54:00:c2:c6:e4 --inbound 4294967 # virsh domiftune rhel7new 52:54:00:c2:c6:e4 inbound.average: 4294967 inbound.peak : 0 inbound.burst : 0 outbound.average: 4294967 outbound.peak : 0 outbound.burst : 0 [root@server ~]# virsh domiftune rhel7new 52:54:00:c2:c6:e4 --inbound 4294968 error: Unable to set interface parameters error: internal error: Child process (/sbin/tc class add dev vnet1 parent 1: classid 1:1 htb rate 4294968kbps) unexpected exit status 1: 2014-11-26 02:26:20.653+0000: 19782: debug : virFileClose:99 : Closed fd 24 2014-11-26 02:26:20.653+0000: 19782: debug : virFileClose:99 : Closed fd 27 2014-11-26 02:26:20.653+0000: 19782: debug : virFileClose:99 : Closed fd 22 Illegal "rate" Usage: ... qdisc add ... htb [default N] [r2q N] default minor id of class to which unclassified packets are sent {0} r2q DRR quantums are computed as rate in Bps/r2q {10} debug string of 16 numbers each 0-3 {0} ... class add ... htb rate R1 [burst B1] [mpu B] [overhead O] [prio P] [slot S] [pslot PS] [ceil R2] [cburst B2] [mtu MTU] [quantum Q] rate rate allocated to this class (class can still borrow) burst max bytes burst which can be accumulated during idle period {computed} mpu minimum packet size used in rate computations overhead per-packet size overhead used in rate computations ------->this error reported out may be wrong [root@server ~]# virsh domiftune rhel7new 52:54:00:c2:c6:e4 --inbound 4294967295 error: Unable to set interface parameters error: internal error: Child process (/sbin/tc class add dev vnet1 parent 1: classid 1:1 htb rate 4294967295kbps) unexpected exit status 1: 2014-11-26 02:26:33.478+0000: 19789: debug : virFileClose:99 : Closed fd 24 2014-11-26 02:26:33.478+0000: 19789: debug : virFileClose:99 : Closed fd 27 2014-11-26 02:26:33.478+0000: 19789: debug : virFileClose:99 : Closed fd 22 Illegal "rate" Usage: ... qdisc add ... htb [default N] [r2q N] default minor id of class to which unclassified packets are sent {0} r2q DRR quantums are computed as rate in Bps/r2q {10} debug string of 16 numbers each 0-3 {0} ... class add ... htb rate R1 [burst B1] [mpu B] [overhead O] [prio P] [slot S] [pslot PS] [ceil R2] [cburst B2] [mtu MTU] [quantum Q] rate rate allocated to this class (class can still borrow) burst max bytes burst which can be accumulated during idle period {computed} mpu minimum packet size used in rate computations overhead per-packet size overhead used in rate computati [root@server ~]# virsh domiftune rhel7new 52:54:00:c2:c6:e4 --inbound 4294967296 error: inbound rate larger than maximum 4294967295 [root@server ~]# virsh domiftune rhel7new 52:54:00:c2:c6:e4 --inbound -1 error: inbound rate larger than maximum 4294967295 3. when setting outbound, result are the same with inbound setting
The error comes directly from tc and the important part is there: Illegal "rate" The RHEL version of tc does not accept 64-bit rates.
(In reply to Jan Tomko from comment #10) > The error comes directly from tc and the important part is there: > Illegal "rate" > > The RHEL version of tc does not accept 64-bit rates. thanks, Jan I check with TC command to see TC stores sizes internally as 32-bit unsigned integer in byte, so we can specify a max size of 4294967295 bytes. but for rhel, libvirt now check the network traffic range support 64-bit rates, so the check range does not match now. If this result is acceptable now, we should wait for rhel upgrade to higher version which would accept 64-bit rates, do you know when will be the deliver plan for that?
thanks, Jan I check with TC command to see TC stores sizes internally as 32-bit unsigned integer in byte, so we can specify a max size of 4294967295 bytes. but for rhel, libvirt now check the network traffic range support 64-bit rates, so the check range does not match now. If this result is acceptable now, we should wait for rhel upgrade to higher version which would accept 64-bit rates, do you know when will be the deliver plan for that?
Libvirt only checks that the rate fits into its own structures. We don't know the range supported by tc in advance, but since it correctly reports an error if it's too large and libvirt passes it back to the user, everything's OK from libvirt's point of view.
update log_level from 1 to 3 debug log from libvirtd dismissed when TC report error, since everything works OK from libvirt view, so change to verified # virsh domiftune rhel7 52:54:00:c6:3b:95 --inbound 4294967295 error: Unable to set interface parameters error: internal error: Child process (/sbin/tc class add dev vnet1 parent 1: classid 1:1 htb rate 4294967295kbps) unexpected exit status 1: Illegal "rate" Usage: ... qdisc add ... htb [default N] [r2q N] default minor id of class to which unclassified packets are sent {0} r2q DRR quantums are computed as rate in Bps/r2q {10} debug string of 16 numbers each 0-3 {0} ... class add ... htb rate R1 [burst B1] [mpu B] [overhead O] [prio P] [slot S] [pslot PS] [ceil R2] [cburst B2] [mtu MTU] [quantum Q] rate rate allocated to this class (class can still borrow) burst max bytes burst which can be accumulated during idle period {computed} mpu minimum packet size used in rate computations overhead per-packet size overhead used in rate computations linklay adapting to a linklayer e.g. atm ceil definite upper class rate (no borrows) {rate} cburst burst but for ceil {computed} mtu max packet size we create rate map for {1600} prio priority of leaf; l # virsh domiftune rhel7 52:54:00:c6:3b:95 --inbound 4294968 error: Unable to set interface parameters error: internal error: Child process (/sbin/tc class add dev vnet1 parent 1: classid 1:1 htb rate 4294968kbps) unexpected exit status 1: Illegal "rate" Usage: ... qdisc add ... htb [default N] [r2q N] default minor id of class to which unclassified packets are sent {0} r2q DRR quantums are computed as rate in Bps/r2q {10} debug string of 16 numbers each 0-3 {0} ... class add ... htb rate R1 [burst B1] [mpu B] [overhead O] [prio P] [slot S] [pslot PS] [ceil R2] [cburst B2] [mtu MTU] [quantum Q] rate rate allocated to this class (class can still borrow) burst max bytes burst which can be accumulated during idle period {computed} mpu minimum packet size used in rate computations overhead per-packet size overhead used in rate computations linklay adapting to a linklayer e.g. atm ceil definite upper class rate (no borrows) {rate} cburst burst but for ceil {computed} mtu max packet size we create rate map for {1600} prio priority of leaf; lowe
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0323.html