| Summary: | Kernel Panic with 'intel_iommu=on' Parameter in HP ProLiant SL390s G7 Server | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Yaniv Liberman <yliberma> | ||||
| Component: | kernel | Assignee: | Prarit Bhargava <prarit> | ||||
| kernel sub component: | NIC Drivers | QA Contact: | Jianlin Shi <jishi> | ||||
| Status: | CLOSED WORKSFORME | Docs Contact: | |||||
| Severity: | unspecified | ||||||
| Priority: | unspecified | CC: | danken, network-qe, yliberma, yzaspits | ||||
| Version: | 7.2 | ||||||
| Target Milestone: | rc | ||||||
| Target Release: | 7.3 | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2016-01-04 16:46:29 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Attachments: |
|
||||||
Yaniv, iirc, the same host booted fine with a rhel-7.1.z kernel. If so, can you provide its version? Hi, Yes, and here you go: RHEL 7.1, 3.10.0-229.el7.x86_64 More information:
Before appending "intel_iommu=on" in RHEL 7.1 (3.10.0-229.el7.x86_64),
--
[root@buri08 ~]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64 root=/dev/mapper/vg0-lv_root ro rd.lvm.lv=vg0/lv_root crashkernel=auto rd.lvm.lv=vg0/lv_swap rhgb quiet vconsole.font=latarcyrheb-sun16 vconsole.keymap=us console=tty0 console=ttyS1,115200
[root@buri08 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:24:7c:d8 brd ff:ff:ff:ff:ff:ff
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:24:7c:d9 brd ff:ff:ff:ff:ff:ff
4: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:24:7c:dc brd ff:ff:ff:ff:ff:ff
5: enp5s0d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 78:e7:d1:24:7c:dd brd ff:ff:ff:ff:ff:ff
inet 10.35.19.194/22 brd 10.35.19.255 scope global dynamic enp5s0d1
valid_lft 43055sec preferred_lft 43055sec
inet6 fe80::7ae7:d1ff:fe24:7cdd/64 scope link
valid_lft forever preferred_lft forever
--
And after appending "intel_iommu=on",
--
[root@buri08 ~]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64 root=/dev/mapper/vg0-lv_root ro rd.lvm.lv=vg0/lv_root crashkernel=auto rd.lvm.lv=vg0/lv_swap rhgb quiet vconsole.font=latarcyrheb-sun16 vconsole.keymap=us console=tty0 console=ttyS1,115200 intel_iommu=on
[root@buri08 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:24:7c:d8 brd ff:ff:ff:ff:ff:ff
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:24:7c:d9 brd ff:ff:ff:ff:ff:ff
--
So, as you can see, the 'enp5s0' and 'enp5s0d1' network interfaces, which are the Mellanox NIC [1], are not recognized by the OS anymore after appending the "intel_iommu=on" parameter in RHEL 7.1.
This can be solved by simply removing the "intel_iommu=on" parameter and rebooting the server.
[1],
--
[root@buri08 ~]# lspci -v |grep "Mellanox"
05:00.0 InfiniBand: Mellanox Technologies MT26438 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE Virtualization+] (rev b0)
--
Hmm ... I installed 7.1, and updated the kernel. Everything seems to work AFAICT?
[root@hp-sl390s-01 ~]# hostname
hp-sl390s-01.rhts.eng.bos.redhat.com
[root@hp-sl390s-01 ~]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto console=ttyS1,115200 rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap LANG=en_US.UTF-8 intel_iommu=on
[root@hp-sl390s-01 ~]# dmesg | grep -i iommu
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto console=ttyS1,115200 rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap LANG=en_US.UTF-8 intel_iommu=on
[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto console=ttyS1,115200 rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap LANG=en_US.UTF-8 intel_iommu=on
[ 0.000000] Intel-IOMMU: enabled
[ 0.982394] dmar: IOMMU 0: reg_base_addr efffe000 ver 1:0 cap c90780106f0462 ecap f0207e
[ 8.438663] IOMMU 0 0xefffe000: using Queued invalidation
[ 8.441243] IOMMU: Setting RMRR:
[ 8.442773] IOMMU: Setting identity map for device 0000:02:00.0 [0xe763e000 - 0xe763ffff]
[ 8.446585] IOMMU: Setting identity map for device 0000:02:00.2 [0xe763e000 - 0xe763ffff]
[ 8.450462] IOMMU: Setting identity map for device 0000:00:1d.0 [0xe77f5000 - 0xe77fafff]
[ 8.454248] IOMMU: Setting identity map for device 0000:00:1d.1 [0xe77f5000 - 0xe77fafff]
[ 8.458045] IOMMU: Setting identity map for device 0000:00:1d.2 [0xe77f5000 - 0xe77fafff]
[ 8.561789] IOMMU: Setting identity map for device 0000:00:1d.3 [0xe77f5000 - 0xe77fafff]
[ 8.565576] IOMMU: Setting identity map for device 0000:02:00.0 [0xe77f5000 - 0xe77fafff]
[ 8.569512] IOMMU: Setting identity map for device 0000:02:00.2 [0xe77f5000 - 0xe77fafff]
[ 8.573493] IOMMU: Setting identity map for device 0000:02:00.4 [0xe77f5000 - 0xe77fafff]
[ 8.577575] IOMMU: Setting identity map for device 0000:00:1d.7 [0xe77fc000 - 0xe77fdfff]
[ 8.581405] IOMMU: Prepare 0-16MiB unity mapping for LPC
[ 8.583877] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[root@hp-sl390s-01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 78:e7:d1:21:6e:78 brd ff:ff:ff:ff:ff:ff
inet 10.16.67.67/21 brd 10.16.71.255 scope global dynamic enp4s0f0
valid_lft 86267sec preferred_lft 86267sec
inet6 2620:52:0:1040:7ae7:d1ff:fe21:6e78/64 scope global dynamic
valid_lft 2591937sec preferred_lft 604737sec
inet6 fe80::7ae7:d1ff:fe21:6e78/64 scope link
valid_lft forever preferred_lft forever
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:21:6e:79 brd ff:ff:ff:ff:ff:ff
4: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:21:6e:7d brd ff:ff:ff:ff:ff:ff
5: enp5s0d1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:21:6e:7e brd ff:ff:ff:ff:ff:ff
The BUG() FWIW, appears to be an issue where the device did not reset.
P.
With RHEL7.2 ...
[root@hp-sl390s-01 ~]# hostname
hp-sl390s-01.rhts.eng.bos.redhat.com
[root@hp-sl390s-01 ~]# dmesg | grep -i iommu
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap console=ttyS1,115200 LANG=en_US.UTF-8 intel_iommu=on
[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap console=ttyS1,115200 LANG=en_US.UTF-8 intel_iommu=on
[ 0.000000] Intel-IOMMU: enabled
[ 0.489877] dmar: IOMMU 0: reg_base_addr efffe000 ver 1:0 cap c90780106f0462 ecap f0207e
[ 7.487620] IOMMU 0 0xefffe000: using Queued invalidation
[ 7.490170] IOMMU: Setting RMRR:
[ 7.491825] IOMMU: Setting identity map for device 0000:02:00.0 [0xe763e000 - 0xe763ffff]
[ 7.495646] IOMMU: Setting identity map for device 0000:02:00.2 [0xe763e000 - 0xe763ffff]
[ 7.499412] IOMMU: Setting identity map for device 0000:00:1d.0 [0xe77f5000 - 0xe77fafff]
[ 7.603191] IOMMU: Setting identity map for device 0000:00:1d.1 [0xe77f5000 - 0xe77fafff]
[ 7.607014] IOMMU: Setting identity map for device 0000:00:1d.2 [0xe77f5000 - 0xe77fafff]
[ 7.610831] IOMMU: Setting identity map for device 0000:00:1d.3 [0xe77f5000 - 0xe77fafff]
[ 7.614665] IOMMU: Setting identity map for device 0000:02:00.0 [0xe77f5000 - 0xe77fafff]
[ 7.618427] IOMMU: Setting identity map for device 0000:02:00.2 [0xe77f5000 - 0xe77fafff]
[ 7.622354] IOMMU: Setting identity map for device 0000:02:00.4 [0xe77f5000 - 0xe77fafff]
[ 7.626204] IOMMU: Setting identity map for device 0000:00:1d.7 [0xe77fc000 - 0xe77fdfff]
[ 7.629926] IOMMU: Prepare 0-16MiB unity mapping for LPC
[ 7.632502] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[root@hp-sl390s-01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 78:e7:d1:21:6e:78 brd ff:ff:ff:ff:ff:ff
inet 10.16.67.67/21 brd 10.16.71.255 scope global dynamic enp4s0f0
valid_lft 86306sec preferred_lft 86306sec
inet6 2620:52:0:1040:7ae7:d1ff:fe21:6e78/64 scope global noprefixroute dynamic
valid_lft 2591984sec preferred_lft 604784sec
inet6 fe80::7ae7:d1ff:fe21:6e78/64 scope link
valid_lft forever preferred_lft forever
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:21:6e:79 brd ff:ff:ff:ff:ff:ff
4: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:21:6e:7d brd ff:ff:ff:ff:ff:ff
5: enp5s0d1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 78:e7:d1:21:6e:7e brd ff:ff:ff:ff:ff:ff
P.
AFAICT this works? Closing as WORKSFORME. P. |
Created attachment 1110295 [details] Includes Kernel panic screenshots and the output of 'lspci -vvv'. Description of problem: Kernel panic with 'intel_iommu=on' parameter in RHEL 7.2 - HP ProLiant SL390s G7 server. Version-Release number of selected component (if applicable): 3.10.0-327.el7.x86_64 How reproducible: This can be reproduced by appending the parameter 'intel_iommu=on' to the Kernel in RHEL 7.2, in a HP ProLiant SL390s G7 server. Steps to Reproduce: 1. The OS needs to be RHEL 7.2 2. vi /etc/default/grub 3. Append 'intel_iommu=on' to 'CMDLINE', and then :wq 4. grub2-mkconfig -o /boot/grub2/grub.cfg 5. Reboot the server 6. Kernel panic Actual results: Kernel panic. Expected results: The OS to boot normally (this doesn't happen in RHEL 7.1 and below). Please see the attached file.