Bug 1595184
| Summary: | Libvirtd crashed when domifstat for interface type: user, udp | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Fangge Jin <fjin> | ||||
| Component: | libvirt | Assignee: | Michal Privoznik <mprivozn> | ||||
| Status: | CLOSED ERRATA | QA Contact: | yalzhang <yalzhang> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 7.6 | CC: | dyuan, lmen, mprivozn, xuzhang, yafu, yalzhang | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | libvirt-4.5.0-3.el7 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1596512 (view as bug list) | Environment: | |||||
| Last Closed: | 2018-10-30 09:56:58 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1596512 | ||||||
| Attachments: |
|
||||||
No crash when domifstat other interface by either interface name or mac addres When restart libvirtd with such interface, there is error in libvirtd.log: 2018-07-04 10:04:48.870+0000: 16277: debug : virNWFilterBindingLookupByPortDev:584 : conn=0x7f5d00023de0, name=<null> 2018-07-04 10:04:48.870+0000: 16277: error : virNWFilterBindingLookupByPortDev:589 : portdev in virNWFilterBindingLookupByPortDev must not be NULL 2018-07-04 10:04:48.870+0000: 16277: debug : virNWFilterBindingLookupByPortDev:584 : conn=0x7f5d00023de0, name=<null> 2018-07-04 10:04:48.870+0000: 16277: error : virNWFilterBindingLookupByPortDev:589 : portdev in virNWFilterBindingLookupByPortDev must not be NULL The root cause for this may be same, please double confirm when verifying this bug When restart libvirtd with such interface, there is error in libvirtd.log: 2018-07-04 10:04:48.870+0000: 16277: debug : virNWFilterBindingLookupByPortDev:584 : conn=0x7f5d00023de0, name=<null> 2018-07-04 10:04:48.870+0000: 16277: error : virNWFilterBindingLookupByPortDev:589 : portdev in virNWFilterBindingLookupByPortDev must not be NULL 2018-07-04 10:04:48.870+0000: 16277: debug : virNWFilterBindingLookupByPortDev:584 : conn=0x7f5d00023de0, name=<null> 2018-07-04 10:04:48.870+0000: 16277: error : virNWFilterBindingLookupByPortDev:589 : portdev in virNWFilterBindingLookupByPortDev must not be NULL The root cause for this may be same, please double confirm when verifying this bug Patch proposed upstream: https://www.redhat.com/archives/libvir-list/2018-July/msg00827.html Test on libvirt-4.5.0-3.el7.x86_64, libvirtd do not crash, the issue is fixed. 1. Start the vm with multiple interfaces as below: # virsh domiflist rhel Interface Type Source Model MAC ------------------------------------------------------- vnet0 network default virtio 52:54:00:49:92:db - user - rtl8139 00:11:22:33:44:55 - udp - rtl8139 52:54:00:54:02:e5 - mcast - rtl8139 52:54:00:29:ae:b7 - server - rtl8139 52:54:00:22:c9:42 - client - rtl8139 52:54:00:8b:c9:51 # pidof libvirtd 76350 2. both will report error as below except for the network type: # virsh domifstat rhel 00:11:22:33:44:55 error: Failed to get interface stats rhel 00:11:22:33:44:55 error: internal error: Interface name not provided # virsh domifstat rhel 52:54:00:54:02:e5 error: Failed to get interface stats rhel 52:54:00:54:02:e5 error: internal error: Interface name not provided ... 3. destroy the guest and configure the target name for them, then start vm, the error message will change, but libvirt not crash: # virsh domiflist rhel Interface Type Source Model MAC ------------------------------------------------------- vnet0 network default virtio 52:54:00:49:92:db user_target user - rtl8139 00:11:22:33:44:55 udp_target udp - rtl8139 52:54:00:54:02:e5 mcast_target mcast - rtl8139 52:54:00:29:ae:b7 server_target server - rtl8139 52:54:00:22:c9:42 client_target client - rtl8139 52:54:00:8b:c9:51 # virsh domifstat rhel mcast_target error: Failed to get interface stats rhel mcast_target error: internal error: /proc/net/dev: Interface not found # virsh domifstat rhel 52:54:00:22:c9:42 error: Failed to get interface stats rhel 52:54:00:22:c9:42 error: internal error: /proc/net/dev: Interface not found # pidof libvirtd 76350 Hi Michal, 1) I think the result on comment 8 is acceptable. Please help to confirm, thank you! 2) And for the error message in comment 4, I think it is not related with this bug. Furthermore, I can not reproduce when restart libvirtd with interfaces in comment 8 or comment 1, so if you agree, I will move this bug to be verified. 3) I found the same error message here: https://www.redhat.com/archives/libvir-list/2018-June/msg01348.html I'm not sure if that patch is to fix this kind of issue, and I can not tell if that patch is merged downstream in libvirt-4.5.0-3.el7.x86_64. Could you please help to check? because I can still get the error with below scenario: Start with non-exits nwfilter(100% reproducible) 1. prepare a guest with interface as below with not exists nwfitler: <interface type='network'> <source network='default'/> <model type='virtio'/> <filterref filter='no-ip'/> </interface> # virsh start rhel error: Failed to start domain rhel error: internal error: referenced filter 'no-ip' is missing # grep error /var/log/libvirt/libvirtd.log 2018-07-19 07:27:23.139+0000: 79106: error : virNWFilterObjListFindInstantiateFilter:201 : internal error: referenced filter 'no-ip' is missing 2018-07-19 07:27:23.162+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device 2018-07-19 07:27:23.167+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device 2018-07-19 07:27:23.169+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device 2018-07-19 07:27:23.171+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device 2018-07-19 07:27:23.172+0000: 79106: error : virNWFilterBindingLookupByPortDev:589 : portdev in virNWFilterBindingLookupByPortDev must not be NULL 2018-07-19 07:27:23.173+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device 2018-07-19 07:27:23.173+0000: 79106: error : virNWFilterBindingLookupByPortDev:589 : portdev in virNWFilterBindingLookupByPortDev must not be NULL 2018-07-19 07:27:23.175+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device 2018-07-19 07:27:23.177+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device 2018-07-19 07:27:23.179+0000: 79156: error : virNetDevSendEthtoolIoctl:3072 : ethtool ioctl error: No such device As this is a negative scenario, I'm not sure if it is a issue with such error message. Could you please help to confirm? Thank you very much! (In reply to yalzhang from comment #9) > Hi Michal, 1) I think the result on comment 8 is acceptable. Please help to > confirm, thank you! Yes it is. > > 2) And for the error message in comment 4, I think it is not related with > this bug. Furthermore, I can not reproduce when restart libvirtd with > interfaces in comment 8 or comment 1, so if you agree, I will move this bug > to be verified. Yes, it is unrelated. I'm trying to fix it upstream but it is different issue. > > 3) I found the same error message here: > https://www.redhat.com/archives/libvir-list/2018-June/msg01348.html > > I'm not sure if that patch is to fix this kind of issue, and I can not tell > if that patch is merged downstream in libvirt-4.5.0-3.el7.x86_64. Could you > please help to check? because I can still get the error with below scenario: > This is a different issue (the same as in 2) and should be tracked separately. file Bug 1607831 to track the issue in comment 10 and move this one into verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:3113 |
Created attachment 1454615 [details] backtrace of libvirtd Description of problem: Libvirtd crashed when domifstat for interface type: user, udp Version-Release number of selected component (if applicable): libvirt-4.4.0-2.virtcov.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Prepare a guest with interface of type user/udp # virsh dumpxml rhel7.5 --inactive <interface type='udp'> <mac address='52:54:00:22:c9:42'/> <source address='10.66.4.140' port='11116'> <local address='10.66.5.190' port='11115'/> </source> <model type='rtl8139'/> ... <interface type='user'> <mac address='00:11:22:33:44:55'/> <ip address='172.17.2.0' family='ipv4' prefix='24'/> <ip address='2001:db8:ac10:fd01::' family='ipv6' prefix='64'/> <model type='rtl8139'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> </interface> 2.Start guest 3.Get network interface stats by "virsh domifstat": # virsh domiflist rhel7.5 Interface Type Source Model MAC ------------------------------------------------------- vnet0 bridge br0 virtio 00:1a:4a:16:01:55 vnet1 bridge host-bridge rtl8139 52:54:00:0e:24:fe - udp - rtl8139 52:54:00:22:c9:42 vnet2 network default virtio 52:54:00:c6:3b:95 - user - rtl8139 00:11:22:33:44:55 # virsh domifstat rhel7.5 52:54:00:22:c9:42 error: Disconnected from qemu:///system due to end of file error: Failed to get interface stats rhel7.5 52:54:00:22:c9:42 error: End of file while reading data: Input/output error # virsh domifstat rhel7.5 00:11:22:33:44:55 error: Disconnected from qemu:///system due to end of file error: Failed to get interface stats rhel7.5 00:11:22:33:44:55 error: End of file while reading data: Input/output error Actual results: Libvirtd crashed when domifstat for interface type: user, udp Expected results: No crash Additional info: