Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1294688

Summary: Kernel Panic with 'intel_iommu=on' Parameter in HP ProLiant SL390s G7 Server
Product: Red Hat Enterprise Linux 7 Reporter: Yaniv Liberman <yliberma>
Component: kernelAssignee: Prarit Bhargava <prarit>
kernel sub component: NIC Drivers QA Contact: Jianlin Shi <jishi>
Status: CLOSED WORKSFORME Docs Contact:
Severity: unspecified    
Priority: unspecified CC: danken, network-qe, yliberma, yzaspits
Version: 7.2   
Target Milestone: rc   
Target Release: 7.3   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-01-04 16:46:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Includes Kernel panic screenshots and the output of 'lspci -vvv'. none

Description Yaniv Liberman 2015-12-29 17:09:46 UTC 
Created attachment 1110295 [details]
Includes Kernel panic screenshots and the output of 'lspci -vvv'.

Description of problem:
Kernel panic with 'intel_iommu=on' parameter in RHEL 7.2 - HP ProLiant SL390s G7 server.

Version-Release number of selected component (if applicable):
3.10.0-327.el7.x86_64

How reproducible:
This can be reproduced by appending the parameter 'intel_iommu=on' to the Kernel in RHEL 7.2, in a HP ProLiant SL390s G7 server.

Steps to Reproduce:
1. The OS needs to be RHEL 7.2
2. vi /etc/default/grub
3. Append 'intel_iommu=on' to 'CMDLINE', and then :wq
4. grub2-mkconfig -o /boot/grub2/grub.cfg
5. Reboot the server
6. Kernel panic

Actual results:
Kernel panic.

Expected results:
The OS to boot normally (this doesn't happen in RHEL 7.1 and below).

Please see the attached file.
Comment 2 Dan Kenigsberg 2015-12-30 09:53:41 UTC 
Yaniv, iirc, the same host booted fine with a rhel-7.1.z kernel. If so, can you provide its version?
Comment 3 Yaniv Liberman 2015-12-30 12:20:45 UTC 
Hi,

Yes, and here you go:

RHEL 7.1, 3.10.0-229.el7.x86_64
Comment 4 Yaniv Liberman 2015-12-30 13:05:58 UTC 
More information:

Before appending "intel_iommu=on" in RHEL 7.1 (3.10.0-229.el7.x86_64),
--
[root@buri08 ~]# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64 root=/dev/mapper/vg0-lv_root ro rd.lvm.lv=vg0/lv_root crashkernel=auto rd.lvm.lv=vg0/lv_swap rhgb quiet vconsole.font=latarcyrheb-sun16 vconsole.keymap=us console=tty0 console=ttyS1,115200
[root@buri08 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp4s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:24:7c:d8 brd ff:ff:ff:ff:ff:ff
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:24:7c:d9 brd ff:ff:ff:ff:ff:ff
4: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:24:7c:dc brd ff:ff:ff:ff:ff:ff
5: enp5s0d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 78:e7:d1:24:7c:dd brd ff:ff:ff:ff:ff:ff
    inet 10.35.19.194/22 brd 10.35.19.255 scope global dynamic enp5s0d1
       valid_lft 43055sec preferred_lft 43055sec
    inet6 fe80::7ae7:d1ff:fe24:7cdd/64 scope link 
       valid_lft forever preferred_lft forever
--

And after appending "intel_iommu=on",
--
[root@buri08 ~]# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64 root=/dev/mapper/vg0-lv_root ro rd.lvm.lv=vg0/lv_root crashkernel=auto rd.lvm.lv=vg0/lv_swap rhgb quiet vconsole.font=latarcyrheb-sun16 vconsole.keymap=us console=tty0 console=ttyS1,115200 intel_iommu=on
[root@buri08 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp4s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:24:7c:d8 brd ff:ff:ff:ff:ff:ff
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:24:7c:d9 brd ff:ff:ff:ff:ff:ff
--

So, as you can see, the 'enp5s0' and 'enp5s0d1' network interfaces, which are the Mellanox NIC [1], are not recognized by the OS anymore after appending the "intel_iommu=on" parameter in RHEL 7.1.

This can be solved by simply removing the "intel_iommu=on" parameter and rebooting the server.

[1],
--
[root@buri08 ~]# lspci -v |grep "Mellanox"
05:00.0 InfiniBand: Mellanox Technologies MT26438 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE Virtualization+] (rev b0)
--
Comment 5 Prarit Bhargava 2016-01-04 16:22:52 UTC 
Hmm ... I installed 7.1, and updated the kernel.  Everything seems to work AFAICT?

[root@hp-sl390s-01 ~]# hostname
hp-sl390s-01.rhts.eng.bos.redhat.com
[root@hp-sl390s-01 ~]# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto console=ttyS1,115200 rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap LANG=en_US.UTF-8 intel_iommu=on
[root@hp-sl390s-01 ~]# dmesg | grep -i iommu
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto console=ttyS1,115200 rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap LANG=en_US.UTF-8 intel_iommu=on
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto console=ttyS1,115200 rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap LANG=en_US.UTF-8 intel_iommu=on
[    0.000000] Intel-IOMMU: enabled
[    0.982394] dmar: IOMMU 0: reg_base_addr efffe000 ver 1:0 cap c90780106f0462 ecap f0207e
[    8.438663] IOMMU 0 0xefffe000: using Queued invalidation
[    8.441243] IOMMU: Setting RMRR:
[    8.442773] IOMMU: Setting identity map for device 0000:02:00.0 [0xe763e000 - 0xe763ffff]
[    8.446585] IOMMU: Setting identity map for device 0000:02:00.2 [0xe763e000 - 0xe763ffff]
[    8.450462] IOMMU: Setting identity map for device 0000:00:1d.0 [0xe77f5000 - 0xe77fafff]
[    8.454248] IOMMU: Setting identity map for device 0000:00:1d.1 [0xe77f5000 - 0xe77fafff]
[    8.458045] IOMMU: Setting identity map for device 0000:00:1d.2 [0xe77f5000 - 0xe77fafff]
[    8.561789] IOMMU: Setting identity map for device 0000:00:1d.3 [0xe77f5000 - 0xe77fafff]
[    8.565576] IOMMU: Setting identity map for device 0000:02:00.0 [0xe77f5000 - 0xe77fafff]
[    8.569512] IOMMU: Setting identity map for device 0000:02:00.2 [0xe77f5000 - 0xe77fafff]
[    8.573493] IOMMU: Setting identity map for device 0000:02:00.4 [0xe77f5000 - 0xe77fafff]
[    8.577575] IOMMU: Setting identity map for device 0000:00:1d.7 [0xe77fc000 - 0xe77fdfff]
[    8.581405] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    8.583877] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[root@hp-sl390s-01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp4s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 78:e7:d1:21:6e:78 brd ff:ff:ff:ff:ff:ff
    inet 10.16.67.67/21 brd 10.16.71.255 scope global dynamic enp4s0f0
       valid_lft 86267sec preferred_lft 86267sec
    inet6 2620:52:0:1040:7ae7:d1ff:fe21:6e78/64 scope global dynamic 
       valid_lft 2591937sec preferred_lft 604737sec
    inet6 fe80::7ae7:d1ff:fe21:6e78/64 scope link 
       valid_lft forever preferred_lft forever
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:21:6e:79 brd ff:ff:ff:ff:ff:ff
4: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:21:6e:7d brd ff:ff:ff:ff:ff:ff
5: enp5s0d1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:21:6e:7e brd ff:ff:ff:ff:ff:ff

The BUG() FWIW, appears to be an issue where the device did not reset.

P.
Comment 6 Prarit Bhargava 2016-01-04 16:45:21 UTC 
With RHEL7.2 ...

[root@hp-sl390s-01 ~]# hostname
hp-sl390s-01.rhts.eng.bos.redhat.com
[root@hp-sl390s-01 ~]# dmesg | grep -i iommu
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap console=ttyS1,115200 LANG=en_US.UTF-8 intel_iommu=on
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel_hp--sl390s--01-root ro crashkernel=auto rd.lvm.lv=rhel_hp-sl390s-01/root rd.lvm.lv=rhel_hp-sl390s-01/swap console=ttyS1,115200 LANG=en_US.UTF-8 intel_iommu=on
[    0.000000] Intel-IOMMU: enabled
[    0.489877] dmar: IOMMU 0: reg_base_addr efffe000 ver 1:0 cap c90780106f0462 ecap f0207e
[    7.487620] IOMMU 0 0xefffe000: using Queued invalidation
[    7.490170] IOMMU: Setting RMRR:
[    7.491825] IOMMU: Setting identity map for device 0000:02:00.0 [0xe763e000 - 0xe763ffff]
[    7.495646] IOMMU: Setting identity map for device 0000:02:00.2 [0xe763e000 - 0xe763ffff]
[    7.499412] IOMMU: Setting identity map for device 0000:00:1d.0 [0xe77f5000 - 0xe77fafff]
[    7.603191] IOMMU: Setting identity map for device 0000:00:1d.1 [0xe77f5000 - 0xe77fafff]
[    7.607014] IOMMU: Setting identity map for device 0000:00:1d.2 [0xe77f5000 - 0xe77fafff]
[    7.610831] IOMMU: Setting identity map for device 0000:00:1d.3 [0xe77f5000 - 0xe77fafff]
[    7.614665] IOMMU: Setting identity map for device 0000:02:00.0 [0xe77f5000 - 0xe77fafff]
[    7.618427] IOMMU: Setting identity map for device 0000:02:00.2 [0xe77f5000 - 0xe77fafff]
[    7.622354] IOMMU: Setting identity map for device 0000:02:00.4 [0xe77f5000 - 0xe77fafff]
[    7.626204] IOMMU: Setting identity map for device 0000:00:1d.7 [0xe77fc000 - 0xe77fdfff]
[    7.629926] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    7.632502] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[root@hp-sl390s-01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp4s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 78:e7:d1:21:6e:78 brd ff:ff:ff:ff:ff:ff
    inet 10.16.67.67/21 brd 10.16.71.255 scope global dynamic enp4s0f0
       valid_lft 86306sec preferred_lft 86306sec
    inet6 2620:52:0:1040:7ae7:d1ff:fe21:6e78/64 scope global noprefixroute dynamic 
       valid_lft 2591984sec preferred_lft 604784sec
    inet6 fe80::7ae7:d1ff:fe21:6e78/64 scope link 
       valid_lft forever preferred_lft forever
3: enp4s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:21:6e:79 brd ff:ff:ff:ff:ff:ff
4: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:21:6e:7d brd ff:ff:ff:ff:ff:ff
5: enp5s0d1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 78:e7:d1:21:6e:7e brd ff:ff:ff:ff:ff:ff

P.
Comment 7 Prarit Bhargava 2016-01-04 16:46:29 UTC 
AFAICT this works?  Closing as WORKSFORME.  

P.