Bug 1355661

Summary: [Q35] Nic didn't be found through ifconfig after hot-plug the physical network
Product: Red Hat Enterprise Linux 7 Reporter: jingzhao <jinzhao>
Component: qemu-kvm-rhevAssignee: Marcel Apfelbaum <marcel>
Status: CLOSED DUPLICATE QA Contact: jingzhao <jinzhao>
Severity: high Docs Contact:
Priority: high    
Version: 7.3CC: chayang, juzhang, knoel, virt-maint, yfu, yiwei
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-08 14:18:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
The log of dmesg none

Description jingzhao 2016-07-12 07:51:01 UTC
Description of problem:
Network didn't be found through ifconfig after hot-plug the physical network

Version-Release number of selected component (if applicable):
host kernel:3.10.0-464.el7.x86_64
qemu-kvm-rhev-2.6.0-12.el7.x86_64
seabios-bin-1.9.1-4.el7.noarch

guest kernel:3.10.0-456.el7.x86_64

How reproducible:
3/3

Steps to Reproduce:

ll /sys/bus/pci/devices/0000\:03\:00.0/driver
lrwxrwxrwx. 1 root root 0 Jul 12 01:44 /sys/bus/pci/devices/0000:03:00.0/driver -> ../../../../bus/pci/drivers/vfio-pci


1.Boot guest with following cli:
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35-seabios.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \
-drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \
-device ioh3420,id=root.2,slot=3 \
-device virtio-net-pci,bus=root.2,netdev=tap10,mac=9a:6a:6b:6c:6d:6e -netdev tap,id=tap10 \
-monitor stdio \

2. Hot-plug the physical network through hmp
{"execute":"device_add","arguments":{"driver":"vfio-pci","host":"03:00.0","id":"pf","bus":"downstream3"}}

3.check the network info 
[root@unused ~]# lspci 
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic card (rev 04)
00:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:03.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
02:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Upstream) (rev 02)
03:00.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:01.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
03:02.0 PCI bridge: Texas Instruments XIO3130 PCI Express Switch (Downstream) (rev 01)
04:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
05:00.0 SCSI storage controller: Red Hat, Inc Virtio block device (rev 01)
06:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
07:00.0 Ethernet controller: Red Hat, Inc Virtio network device
[root@unused ~]# lspci -vvv -t
-[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
           +-01.0  Red Hat, Inc. QXL paravirtual graphic card
           +-02.0-[01]----00.0  Red Hat, Inc Virtio block device
           +-03.0-[02-06]----00.0-[03-06]--+-00.0-[04]----00.0  Red Hat, Inc Virtio block device
           |                               +-01.0-[05]----00.0  Red Hat, Inc Virtio block device
           |                               \-02.0-[06]----00.0  Intel Corporation 82576 Gigabit Network Connection
           +-04.0-[07]----00.0  Red Hat, Inc Virtio network device
           +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
           +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
           \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller

[root@unused home]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.66.144.137  netmask 255.255.252.0  broadcast 10.66.147.255
        inet6 2620:52:0:4292:986a:6bff:fe6c:6d6e  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::986a:6bff:fe6c:6d6e  prefixlen 64  scopeid 0x20<link>
        ether 9a:6a:6b:6c:6d:6e  txqueuelen 1000  (Ethernet)
        RX packets 636  bytes 73066 (71.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 309  bytes 139427 (136.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 128  bytes 11780 (11.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 128  bytes 11780 (11.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:00:00:00:00:00  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0-nic: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 52:54:00:41:03:50  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


Actual results:
The physical network didn't be found through "ifconfig" 

The detail error message through "dmesg"

[   95.302198] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
[   95.302201] igb: Copyright (c) 2007-2014 Intel Corporation.
[   95.311126] igb: probe of 0000:06:00.0 failed with error -5


Expected results:
The physical network can be found through "ifconfig" 

Additional info:
1.Didn't hit the issue when hot-plug to the root port
2.Didn't hit the issue with pc machine type

Following is the info of physical net

[root@hp-z800-01 home]# lspci -vvv -s 03:00.0
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
	Physical Slot: 1
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 29
	Region 0: Memory at e4800000 (32-bit, non-prefetchable) [disabled] [size=128K]
	Region 1: Memory at e4000000 (32-bit, non-prefetchable) [disabled] [size=4M]
	Region 2: I/O ports at c000 [disabled] [size=32]
	Region 3: Memory at e4840000 (32-bit, non-prefetchable) [disabled] [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable- Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO+ CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 0e, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-42-33-84
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 1
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 00
		VF offset: 128, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000e4848000 (64-bit, non-prefetchable)
		Region 3: Memory at 00000000e4868000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: vfio-pci
	Kernel modules: igb

Comment 1 jingzhao 2016-07-12 07:51:40 UTC
Created attachment 1178798 [details]
The log of dmesg

Comment 2 jingzhao 2016-07-12 07:52:57 UTC
Also, the nic can be found after the guest reboot

Comment 4 Marcel Apfelbaum 2016-08-08 14:18:30 UTC
Hi,

This is yet another symptom of BZ 1352808. See comment #3 fore more details.
It is not related to devices assignmen, a virtio-net-pci device in virtio-1 mode
will have the same issue. (Please let me know if I am wrong)

The same outcome should be also for a i440fx machine when assigning the device
into a pci-bridge.

It is possible that a newer Linux guest with 4.5 or newer kernel will
succeed to re-balance the PCI resources.


Thanks,
Marcel

*** This bug has been marked as a duplicate of bug 1352808 ***