Bug 1958175
| Summary: | [virtual network] virtio-net device does not function correctly if created with vnet_hdr=False | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Lei Yang <leiyang> |
| Component: | qemu-kvm | Assignee: | ybendito |
| qemu-kvm sub component: | Networking | QA Contact: | Lei Yang <leiyang> |
| Status: | CLOSED MIGRATED | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | aadam, chayang, hshuai, jinzhao, juzhang, leidwang, lvivier, virt-maint, ybendito, yvugenfi |
| Version: | 9.0 | Keywords: | MigratedToJIRA, Reopened, Triaged |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-08-16 12:49:50 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Attachments: | |||
Created attachment 1780703 [details]
nic_hotplug.vhost_nic.nic_virtio error log 'Hotplug nic still can't get ip after reboot vm'
Created attachment 1780705 [details]
nic_hotplug.vhost_nic.nic_virtio pass log
attached for comparison purposes
I think there is a problem with your configuration because even before the reboot the NIC doesn't have an IP address: 2021-05-07 07:03:25: [ 38.027419] pcieport 0000:00:03.0: Slot(0-3): Attention button pressed 2021-05-07 07:03:25: [ 38.029452] pcieport 0000:00:03.0: Slot(0-3) Powering on due to button press 2021-05-07 07:03:25: [ 38.031748] pcieport 0000:00:03.0: Slot(0-3): Card present 2021-05-07 07:03:25: [ 38.034250] pcieport 0000:00:03.0: Slot(0-3): Link Up 2021-05-07 07:03:25: [ 38.168220] pci 0000:05:00.0: [1af4:1041] type 00 class 0x020000 2021-05-07 07:03:25: [ 38.170310] pci 0000:05:00.0: reg 0x14: [mem 0x00000000-0x00000fff] 2021-05-07 07:03:25: [ 38.172443] pci 0000:05:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref] 2021-05-07 07:03:25: [ 38.174690] pci 0000:05:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref] 2021-05-07 07:03:25: [ 38.178610] pci 0000:05:00.0: BAR 6: assigned [mem 0xfe200000-0xfe23ffff pref] 2021-05-07 07:03:25: [ 38.180682] pci 0000:05:00.0: BAR 4: assigned [mem 0xfd000000-0xfd003fff 64bit pref] 2021-05-07 07:03:25: [ 38.183120] pci 0000:05:00.0: BAR 1: assigned [mem 0xfe240000-0xfe240fff] 2021-05-07 07:03:25: [ 38.185182] pcieport 0000:00:03.0: PCI bridge to [bus 05] 2021-05-07 07:03:25: [ 38.186848] pcieport 0000:00:03.0: bridge window [io 0x4000-0x4fff] 2021-05-07 07:03:25: [ 38.198107] pcieport 0000:00:03.0: bridge window [mem 0xfe200000-0xfe3fffff] 2021-05-07 07:03:25: [ 38.201534] pcieport 0000:00:03.0: bridge window [mem 0xfd000000-0xfd1fffff 64bit pref] 2021-05-07 07:03:25: [ 38.208765] virtio-pci 0000:05:00.0: enabling device (0000 -> 0002) 2021-05-07 07:03:25: [ 38.287210] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready 2021-05-07 07:03:25: [ 38.289496] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready 2021-05-07 07:03:25: [ 38.293195] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready 2021-05-07 07:03:25: [ 38.328296] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready 2021-05-07 07:03:26: [ 39.263892] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready ... 2021-05-07 07:10:38: ifconfig -a 2021-05-07 07:10:38: eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 2021-05-07 07:10:38: inet6 2620:52:0:49e0:9806:a8ff:fe9c:f364 prefixlen 64 scopeid 0x0<global> 2021-05-07 07:10:38: ether 9a:06:a8:9c:f3:64 txqueuelen 1000 (Ethernet) 2021-05-07 07:10:38: RX packets 12105 bytes 755387 (737.6 KiB) 2021-05-07 07:10:38: RX errors 0 dropped 0 overruns 0 frame 0 2021-05-07 07:10:38: TX packets 1 bytes 324 (324.0 B) 2021-05-07 07:10:38: TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ... 2021-05-07 07:11:42: shutdown -r now Moreover, the ip-sniffer.log doesn't show the macaddress of the virtio-net device. Is it correctly connected to the host network? (In reply to Laurent Vivier from comment #3) > I think there is a problem with your configuration because even before the > reboot the NIC doesn't have an IP address: Hi, Laurent The logic of code in the configuration file that if hotplug cannot obtain the ip address, it will try to restart to obtain the ip address of the nic. > Moreover, the ip-sniffer.log doesn't show the macaddress of the virtio-net > device. Is it correctly connected to the host network? 1.From my point of view, the problem still occurs on hot plug. ip-sniffer.log is used to record the info captured by the tshark command. If there is no record of the hot pluged nic mac address info, it means that there is a problem in the hot plug process. ==>The tshark command used to automation # tshark -npi any -T fields -E separator=/s -E occurrence=f -E header=y -e ip.src -e ip.dst -e bootp.type -e bootp.id -e bootp.hw.mac_addr -e bootp.ip.your -e bootp.option.dhcp -e ipv6.src -e ipv6.dst 'port 68 or port 546' 2.When this problem reproduced, a sub-interface has been created on the host bridge, but the guest cannot get an ip address. The tshark command did not capture the mac address info of this nic at this time # systemctl status NetworkManager | grep bridge May 17 23:00:55 dell-per730-29.lab.eng.pek2.redhat.com NetworkManager[1645]: <info> [1621306855.0775] device (switch): bridge port t0-I94k7H was attached 3.Comparison info can be found in the case of passing ==>hot plug nic at 07:23:30 07:23:30 DEBUG| Send command: {'execute': 'netdev_add', 'arguments': {'type': 'tap', 'id': 'idj4Nc15', 'fd': '47', 'vhost': True}, 'id': 'ly2Mptca'} 07:23:30 DEBUG| (monitor avocado-vt-vm1.qmpmonitor1) Sending command 'device_add' 07:23:30 DEBUG| Send command: {'execute': 'device_add', 'arguments': OrderedDict([('id', 'idl4baSz'), ('driver', 'virtio-net-pci'), ('netdev', 'idj4Nc15'), ('mac', '9a:06:a8:9c:f3:64'), ('bus', 'pcie_extra_root_port_0'), ('addr', '0x0')]), 'id': 'XDRLKYwo'} 07:23:30 INFO | Check if new interface gets ip address 07:23:34 DEBUG| Updated HWADDR (9a:06:a8:9c:f3:64)<->(10.73.225.127) IP pair into address cache 07:23:35 DEBUG| Found/Verified IP 10.73.225.127 for VM avocado-vt-vm1 NIC 0 07:23:35 INFO | Got the ip address of new nic: 10.73.225.127 ==>Capture the mac address of the nic at 07:23:31: 2021-05-07 07:23:31: 0.0.0.0 255.255.255.255 1 0x207c56bf 9a:06:a8:9c:f3:64 0.0.0.0 3 2021-05-07 07:23:31: 0.0.0.0 255.255.255.255 1 0x207c56bf 9a:06:a8:9c:f3:64 0.0.0.0 3 2021-05-07 07:23:31: 0.0.0.0 255.255.255.255 1 0x207c56bf 9a:06:a8:9c:f3:64 0.0.0.0 3 Best Regards Lei (In reply to Lei Yang from comment #4) > (In reply to Laurent Vivier from comment #3) > > I think there is a problem with your configuration because even before the > > reboot the NIC doesn't have an IP address: > > Hi, Laurent > > The logic of code in the configuration file that if hotplug cannot obtain > the ip address, it will try to restart to obtain the ip address of the nic. > Ok, thank you for the details. If you not hotplug the card but add it directly in the VM before starting the VM, do you get an IP address? Could you run tshark without filters to catch all the traffic on the bridge (normally virbr0) Could you provide the host network configuration? Could you provide the host kernel logs to see if an error comes from the vhost driver when you hotplug the card? (In reply to Laurent Vivier from comment #5) > (In reply to Lei Yang from comment #4) > > (In reply to Laurent Vivier from comment #3) > > > I think there is a problem with your configuration because even before the > > > reboot the NIC doesn't have an IP address: > > > > Hi, Laurent > > > > The logic of code in the configuration file that if hotplug cannot obtain > > the ip address, it will try to restart to obtain the ip address of the nic. > > > > Ok, thank you for the details. > Hi, Laurent > If you not hotplug the card but add it directly in the VM before starting > the VM, do you get an IP address? Add it directly to the VM before starting the VM,then the guest can obtain an IP address. > > Could you run tshark without filters to catch all the traffic on the bridge > (normally virbr0) > Could you provide the host network configuration? The bridge name used on my host is switch. # cat /root/bridge.sh ndev=$(ip link show|grep -i 'state UP'|awk -F: '{if($2 ~ /e.*/) {sub(/^[[:blank:]]*/, "", $2); printf("%s\n\r",$2)}}'|head -n1) mac=$(cat /sys/class/net/$ndev/address) nmcli con add type bridge ifname switch con-name switch stp no nmcli con modify "$ndev" master switch nmcli con down "$ndev" nmcli con up "$ndev" nmcli con modify "System $ndev" master switch nmcli con down "System $ndev" nmcli con up "System $ndev" cat > /etc/qemu-ifup<<EOF #!/bin/sh switch=switch ip link set \$1 up ip link set dev \$1 master \${switch} ip link set \${switch} type bridge forward_delay 0 ip link set \${switch} type bridge stp_state 0 EOF cat > /etc/qemu-ifdown <<EOF #!/bin/sh switch=switch ip link set \$1 down ip link set dev \$1 nomaster EOF chmod a+rx /etc/qemu-if* > Could you provide the host kernel logs to see if an error comes from the > vhost driver when you hotplug the card? Hot plug command line: 10:45:45 DEBUG| (monitor avocado-vt-vm1.qmpmonitor1) Sending command 'netdev_add' 10:45:45 DEBUG| Send command: {'execute': 'netdev_add', 'arguments': {'type': 'tap', 'id': 'idZpzuI7', 'fd': '47', 'vhost': True}, 'id': '9OdZJNC9'} 10:45:45 DEBUG| (monitor avocado-vt-vm1.qmpmonitor1) Sending command 'device_add' 10:45:45 DEBUG| Send command: {'execute': 'device_add', 'arguments': OrderedDict([('id', 'ideaK3h1'), ('driver', 'virtio-net-pci'), ('netdev', 'idZpzuI7'), ('mac', '9a:2a:55:8c:56:79'), ('bus', 'pcie_extra_root_port_0'), ('addr', '0x0')]), 'id': 'IYoGeSov'} # cat /var/log/messages May 18 10:44:47 dell-per730-29 kvm[28375]: 1 guest now active May 18 10:44:47 dell-per730-29 kvm[28378]: 0 guests now active May 18 10:44:47 dell-per730-29 kvm[28406]: 1 guest now active May 18 10:44:47 dell-per730-29 kvm[28410]: 0 guests now active May 18 10:44:47 dell-per730-29 kvm[28425]: 1 guest now active May 18 10:44:48 dell-per730-29 kvm[28431]: 0 guests now active May 18 10:44:49 dell-per730-29 kvm[28457]: 1 guest now active May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.2730] manager: (t0-yRKFFh): new Tun device (/org/freedesktop/NetworkManager/Devices/20) May 18 10:45:45 dell-per730-29 systemd-udevd[28653]: Using default interface naming scheme 'rhel-8.0'. May 18 10:45:45 dell-per730-29 systemd-udevd[28653]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. May 18 10:45:45 dell-per730-29 systemd-udevd[28653]: Could not generate persistent MAC address for t0-yRKFFh: No such file or directory May 18 10:45:45 dell-per730-29 kernel: switch: port 2(t0-yRKFFh) entered blocking state May 18 10:45:45 dell-per730-29 kernel: switch: port 2(t0-yRKFFh) entered disabled state May 18 10:45:45 dell-per730-29 kernel: device t0-yRKFFh entered promiscuous mode May 18 10:45:45 dell-per730-29 kernel: switch: port 2(t0-yRKFFh) entered blocking state May 18 10:45:45 dell-per730-29 kernel: switch: port 2(t0-yRKFFh) entered forwarding state May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3366] device (t0-yRKFFh): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3396] device (t0-yRKFFh): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3406] device (t0-yRKFFh): Activation: starting connection 't0-yRKFFh' (1e197c7e-b2dc-4634-ac37-0b3da1fe69bb) May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3408] device (t0-yRKFFh): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3414] device (t0-yRKFFh): state change: prepare -> config (reason 'none', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3418] device (t0-yRKFFh): state change: config -> ip-config (reason 'none', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3420] device (switch): bridge port t0-yRKFFh was attached May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3420] device (t0-yRKFFh): Activation: connection 't0-yRKFFh' enslaved, continuing activation May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3423] device (t0-yRKFFh): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 dbus-daemon[1637]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.10' (uid=0 pid=1698 comm="/usr/sbin/NetworkManager --no-daemon " label="system_u:system_r:NetworkManager_t:s0") May 18 10:45:45 dell-per730-29 systemd[1]: Starting Network Manager Script Dispatcher Service... May 18 10:45:45 dell-per730-29 dbus-daemon[1637]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' May 18 10:45:45 dell-per730-29 systemd[1]: Started Network Manager Script Dispatcher Service. May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3619] device (t0-yRKFFh): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3622] device (t0-yRKFFh): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external') May 18 10:45:45 dell-per730-29 NetworkManager[1698]: <info> [1621349145.3635] device (t0-yRKFFh): Activation: successful, device activated. May 18 10:45:45 dell-per730-29 systemd[1]: iscsi.service: Unit cannot be reloaded because it is inactive. May 18 10:45:55 dell-per730-29 systemd[1]: NetworkManager-dispatcher.service: Succeeded. May 18 10:48:17 dell-per730-29 kernel: perf: interrupt took too long (3139 > 3127), lowering kernel.perf_event_max_sample_rate to 63000 May 18 10:50:37 dell-per730-29 systemd[1]: Starting system activity accounting tool... May 18 10:50:37 dell-per730-29 systemd[1]: sysstat-collect.service: Succeeded. May 18 10:50:37 dell-per730-29 systemd[1]: Started system activity accounting tool. May 18 10:56:18 dell-per730-29 kernel: device switch left promiscuous mode Created attachment 1784472 [details]
run tshark without filters to catch all the traffic on the bridge
run tshark without filters to catch all the traffic on the bridge
# tshark -i switch > bridge_info
Hit same issue Test Version: kernel-4.18.0-316.el8.x86_64 qemu-kvm-6.0.0-20.module+el8.5.0+11499+199527ef.x86_64 Hi Laurent Is this bug still planned to be fixed in rhel8.5? If it can't be fixed in rhel8.5, could you help to reset the ITR? Thanks in advance. Best Regards Lei Move RHEL-AV bugs to RHEL9. If necessary to resolve in RHEL8, then clone to the current RHEL8 release. Removed the ITR from all bugs as part of the change. Hit same issue Test Version: kernel-4.18.0-350.el8.x86_64 qemu-kvm-6.2.0-1.rc1.scrmod+el8.6.0+13325+d4e3491c.wrb21117.x86_64 Hit same issue Test Version: qemu-kvm-6.2.0-1.module+el8.6.0+13725+61ae1949.x86_64 kernel-4.18.0-358.el8.x86_64 openvswitch2.16-2.16.0-35.el8fdp.x86_64 Hit same issue Test Version: kernel-5.14.0-81.el9.x86_64 qemu-kvm-7.0.0-1.el9.x86_64 edk2-ovmf-20220221gitb24306f15d-1.el9.noarch Hit same issue Test Version: kernel-5.14.0-95.el9.x86_64 qemu-kvm-7.0.0-3.el9.x86_64 edk2-ovmf-20220221gitb24306f15d-1.el9.noarch Guest: RHEL-7.9-20200917.0-Server-x86_64-dvd1.iso Hit same issue Test Version: qemu-kvm-7.1.0-1.el9.x86_64 kernel-5.14.0-165.el9.x86_64 edk2-ovmf-20220526git16779ede2d36-4.el9.noarch Guest: RHEL-9.2.0-20220921.1-x86_64-dvd1.iso After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened. *** Bug 2144427 has been marked as a duplicate of this bug. *** Hit same issue: Test Version: kernel-4.18.0-437.el8.aarch64 qemu-kvm-6.2.0-25.module+el8.8.0+17289+bd3b0bcc.aarch64 Hit same issue on rhel 9.2 Test Version: kernel-5.14.0-226.el9.x86_64 qemu-kvm-7.2.0-2.el9.x86_64 edk2-ovmf-20221207gitfff6d81270b5-1.el9.noarch *** Bug 2002686 has been marked as a duplicate of this bug. *** *** Bug 2084003 has been marked as a duplicate of this bug. *** Hello Laurent Just sync latest info: According to Yuri's debug result which can refer to Bug 2002686#c16，the current problem is caused by “vnet_hdr=False“. Although this problem can be avoided by "vnet_hdr=True", it's still a product problem, feel free to correct me if I'm wrong. And I also confirmed with libvirt QE and developer,libvirt used "vnet_hdr=True" to create device. Test Result: vnet_hdr=Flase: Linux(42/50): http://virtqetools.lab.eng.pek2.redhat.com/kvm_autotest_for_auto_job_detail/?jobid=7581146 Windows(8/10):http://virtqetools.lab.eng.pek2.redhat.com/kvm_autotest_for_auto_job_detail/?jobid=7574345 vnet_hdr=True: Linux(50/50):http://virtqetools.lab.eng.pek2.redhat.com/kvm_autotest_job_log/?jobid=7581154 Windows(70/70): http://virtqetools.lab.eng.pek2.redhat.com/kvm_autotest_for_auto_job_detail/?jobid=7577670 Thanks Lei After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened. Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug. |
Description of problem: An intermittent error occurs on rhel8.5 when executing the test case nic_hotplug.vhost_nic.nic_virtio. Error info: -->FAIL: Hotplug nic still can't get ip after reboot vm Version-Release number of selected component (if applicable): qemu-kvm-6.0.0-16.module+el8.5.0+10848+2dccc46d.x86_64 kernel-4.18.0-305.2.el8.x86_64 Avocado: avocado-framework 87.0 How reproducible: 2/5 Steps to Reproduce: 1.Install latest RHEL8.5 env 2.Please execute the following command to test： python3 ConfigTest.py --testcase=nic_hotplug.vhost_nic.nic_virtio --guestname=RHEL.8.5.0 --machines=q35 --nicmodel=virtio_net --platform=x86_64 Actual results: 07:03:25 DEBUG| Send command: {'execute': 'netdev_add', 'arguments': {'type': 'tap', 'id': 'idFAOSbc', 'fd': '47', 'vhost': True}, 'id': 'TcukncTt'} 07:03:25 DEBUG| (monitor avocado-vt-vm1.qmpmonitor1) Sending command 'device_add' 07:03:25 DEBUG| Send command: {'execute': 'device_add', 'arguments': OrderedDict([('id', 'idYqr9u4'), ('driver', 'virtio-net-pci'), ('netdev', 'idFAOSbc'), ('mac', '9a:06:a8:9c:f3:64'), ('bus', 'pcie_extra_root_port_0'), ('addr', '0x0')]), 'id': 'hbqkSCuH'} 07:03:25 INFO | Check if new interface gets ip address 07:04:55 DEBUG| Attempting to log into 'avocado-vt-vm1' via serial console (timeout 90s) 07:05:06 DEBUG| Sending command: echo %OS% 07:05:06 DEBUG| Sending command (safe): ifconfig -a 07:05:06 DEBUG| Sending command (safe): ip link | grep -B1 '' -i 07:05:06 DEBUG| Sending command: test -d /sys/class/net/eth0 07:05:06 DEBUG| Sending command: echo $? 07:05:07 DEBUG| Sending command: cat /sys/class/net/eth0/address 07:05:07 DEBUG| Sending command (safe): ifconfig eth0 || ip address show eth0 07:05:07 WARNI| No VM's NIC got IP address 07:05:07 DEBUG| Sending command (safe): ifconfig -a 07:05:07 DEBUG| Sending command (safe): ip link | grep -B1 '9a:06:a8:9c:f3:64' -i 07:05:07 DEBUG| Sending command (safe): test -f /etc/sysconfig/network-scripts/ifcfg-eth0 || echo 'DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=yes' > /etc/sysconfig/network-scripts/ifcfg-eth0 07:05:07 DEBUG| Sending command (safe): ifconfig eth0 up 07:05:07 DEBUG| Sending command (safe): dhclient -r 07:05:08 DEBUG| Sending command (safe): dhclient eth0 07:06:09 DEBUG| Sending command (safe): arp -n|awk '/^[1-9]/{print "arp -d " $1}'|sh .... 07:11:42 INFO | Reboot vm after hotplug nic 07:11:42 INFO | Context: Start test iteration 1 --> rebooting 'avocado-vt-vm1' 07:11:42 DEBUG| Send command: shutdown -r now 07:11:42 DEBUG| Sending command: shutdown -r now 07:12:42 INFO | Context: Start test iteration 1 --> rebooting 'avocado-vt-vm1' --> waiting for guest to go down 07:12:42 INFO | Context: Start test iteration 1 --> rebooting 'avocado-vt-vm1' --> logging in after reboot 07:12:42 DEBUG| Attempting to log into 'avocado-vt-vm1' via serial console (timeout 180s) 07:15:03 DEBUG| Attempting to log into 'avocado-vt-vm1' via serial console (timeout 90s) 07:15:14 DEBUG| Sending command: echo %OS% 07:15:14 DEBUG| Sending command (safe): ifconfig -a 07:15:15 DEBUG| Sending command (safe): ip link | grep -B1 '' -i 07:15:15 DEBUG| Sending command: test -d /sys/class/net/eth0 07:15:15 DEBUG| Sending command: echo $? 07:15:15 DEBUG| Sending command: cat /sys/class/net/eth0/address 07:15:15 DEBUG| Sending command (safe): ifconfig eth0 || ip address show eth0 07:15:15 WARNI| No VM's NIC got IP address 07:15:15 DEBUG| Sending command (safe): ifconfig -a 07:15:15 DEBUG| Sending command (safe): ip link | grep -B1 '9a:06:a8:9c:f3:64' -i 07:15:15 DEBUG| Sending command (safe): test -f /etc/sysconfig/network-scripts/ifcfg-eth0 || echo 'DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=yes' > /etc/sysconfig/network-scripts/ifcfg-eth0 07:15:16 DEBUG| Sending command (safe): ifconfig eth0 up 07:15:16 DEBUG| Sending command (safe): dhclient -r 07:15:16 DEBUG| Sending command (safe): dhclient eth0 07:16:17 DEBUG| Sending command (safe): arp -n|awk '/^[1-9]/{print "arp -d " $1}'|sh 07:21:50 INFO | Running 'arp -aen' 07:21:50 DEBUG| [stdout] Address HWtype HWaddress Flags Mask Iface 07:21:50 INFO | Command 'arp -aen' finished with 0 after 0.0018116980063496158s 07:21:50 DEBUG| [stdout] 10.73.227.254 ether 44:aa:50:cc:2b:a1 C switch 07:21:50 DEBUG| Can't get IP address: 07:21:50 DEBUG| Cached IP: None 07:21:50 DEBUG| ARP table: Address HWtype HWaddress Flags Mask Iface 10.73.227.254 ether 44:aa:50:cc:2b:a1 C switch 07:21:50 ERROR| 07:21:50 ERROR| Reproduced traceback from: /usr/local/lib/python3.6/site-packages/avocado_framework_plugin_vt-87.0-py3.6.egg/avocado_vt/test.py:380 07:21:50 ERROR| Traceback (most recent call last): 07:21:50 ERROR| File "/usr/local/lib/python3.6/site-packages/avocado_framework_plugin_vt-87.0-py3.6.egg/virttest/error_context.py", line 135, in new_fn 07:21:50 ERROR| return fn(*args, **kwargs) 07:21:50 ERROR| File "/var/lib/avocado/data/avocado-vt/virttest/test-providers.d/downloads/io-github-autotest-qemu/qemu/tests/nic_hotplug.py", line 220, in run 07:21:50 ERROR| test.fail("Hotplug nic still can't get ip after reboot vm") 07:21:50 ERROR| File "/usr/local/lib/python3.6/site-packages/avocado_framework-87.0-py3.6.egg/avocado/core/test.py", line 955, in fail 07:21:50 ERROR| raise exceptions.TestFail(message) 07:21:50 ERROR| avocado.core.exceptions.TestFail: Hotplug nic still can't get ip after reboot vm 07:21:50 ERROR| Expected results: The vm should get a valid IP address and ping between the guest and the host. Additional info: 1.So far, it only reproduced with the automation. I can't reproduce this problem manually. 2.As mentioned in Bug 1762906#Comment 16. If you delete "vhost = on" from the nic_hotplug.cfg file, there is no the issue any more. So I think this issue is related to "vhost=on",but this can only be reproduced through automation. 3.Let me know if you would like me to set up a machine to reproduce the problem.