Bug 60754 - Installing on S/Merlot with 2 PERC3 cards fails to reboot with SMP kernel
Installing on S/Merlot with 2 PERC3 cards fails to reboot with SMP kernel
Status: CLOSED CURRENTRELEASE
Product: Red Hat Linux
Classification: Retired
Component: kernel (Show other bugs)
7.3
i386 Linux
high Severity high
: ---
: ---
Assigned To: Arjan van de Ven
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2002-03-05 19:16 EST by Tesfamariam Michael
Modified: 2005-10-31 17:00 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2003-02-13 13:14:10 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Displayed message during SMP booting (3.54 KB, text/plain)
2002-03-05 19:17 EST, Tesfamariam Michael
no flags Details
mptable.txt (20.99 KB, patch)
2002-03-07 10:15 EST, Matt Domsch
no flags Details | Diff

  None (edit)
Description Tesfamariam Michael 2002-03-05 19:16:04 EST
Description of Problem:
Installing on Merlot/SlimMerlot with PERC3/QC RAID1 as boot device and other 
PERC3/DC in the system causes the system to crush in reboot. Installation 
completes without any problem but when rebooting with the SMP kernel, it 
crushes. 
Version-Release number of selected component (if applicable):


How Reproducible:
Always.

Steps to Reproduce:
1. Install Pensacola Beta2 or RH7.2 on Merlot/SlimMerlot with PERC3/QC as boot 
device, with another PERC3/DC card on the system.
2. Complete installation.
3. When system reboots to the SMP kernel, it crushes with the attached messages 
(probing for the CPU).

Actual Results:
System crushes when booting with SMP kernel in probing for the CPU.

Expected Results:
For SMP kernel to boot independent of the number of PERC3 cards present in the 
system during installation.

Additional Information:
Installing with a single PERC3 card and then adding the other PERC3 cards 
doesn't cause this problem.
Comment 1 Tesfamariam Michael 2002-03-05 19:17:21 EST
Created attachment 47520 [details]
Displayed message during SMP booting
Comment 2 Arjan van de Ven 2002-03-06 05:15:54 EST
This is REALLY not supposed to happen.
Is this a HT system ?
Comment 3 Dale Kaisner 2002-03-06 12:41:00 EST
Yes, it is an HT system.
Comment 4 Arjan van de Ven 2002-03-06 12:44:31 EST
Ok this looks like an mptable/bios setup bug of some sorts.
Does it work if you pass acpismp=force on the commandline of the kernel ?
(that will not be needed in future builds, but the beta kernel still needed it)
Comment 5 Tesfamariam Michael 2002-03-06 14:24:31 EST
Passing 'apcismp=force' to the kernel doesn't change anything. I also disabled 
the 'Logical Processor' from BIOS, and the problem still persists.
Comment 6 Tesfamariam Michael 2002-03-06 14:33:11 EST
Ooppss, typo. I meant acpismp=force
Comment 7 Matt Domsch 2002-03-07 10:10:18 EST
I think this patch may help.  These systems have 34 busses in the MP table
(NR_MP_BUSSES=32), possibly more when you add in bridge cards.

Hi,

Here's a patch against 2.4.19-pre2 that fixes a problem for machines
that have more busses or irq sources than MAX_MP_BUSSES or
MAX_IRQ_SOURCES has been set to.  This happens on some Intel Foster
machines (or whatever they are calling the processors now) when a PCI
bus expansion unit is plugged in at boot time.

Without this patch, those machines can not boot Linux.

If the machine needs more busses or interrupts, they will be dynamically
allocated at boot time.  If not, the existing MAX_MP_BUSSES and
MAX_IRW_SOURCES value will be used.  Once nice side effect of this patch
is when running a SMP kernel on a UP machine without a MP table, less
kernel memory is used than without the patch.

This patch was originally written by James Cleverdon.

thanks,

greg k-h


diff -Nru a/arch/i386/kernel/mpparse.c b/arch/i386/kernel/mpparse.c
--- a/arch/i386/kernel/mpparse.c	Wed Mar  6 22:11:02 2002
+++ b/arch/i386/kernel/mpparse.c	Wed Mar  6 22:11:02 2002
@@ -35,18 +35,20 @@
  * MP-table.
  */
 int apic_version [MAX_APICS];
-int mp_bus_id_to_type [MAX_MP_BUSSES];
-int mp_bus_id_to_node [MAX_MP_BUSSES];
-int mp_bus_id_to_local [MAX_MP_BUSSES];
 int quad_local_to_mp_bus_id [NR_CPUS/4][4];
-int mp_bus_id_to_pci_bus [MAX_MP_BUSSES] = { [0 ... MAX_MP_BUSSES-1] = -1 };
 int mp_current_pci_id;
+int *mp_bus_id_to_type;
+int *mp_bus_id_to_node;
+int *mp_bus_id_to_local;
+int *mp_bus_id_to_pci_bus;
+int max_mp_busses;
+int max_irq_sources;
 
 /* I/O APIC entries */
 struct mpc_config_ioapic mp_ioapics[MAX_IO_APICS];
 
 /* # of MP IRQ source entries */
-struct mpc_config_intsrc mp_irqs[MAX_IRQ_SOURCES];
+struct mpc_config_intsrc *mp_irqs;
 
 /* MP IRQ source entries */
 int mp_irq_entries;
@@ -302,7 +304,7 @@
 			m->mpc_irqtype, m->mpc_irqflag & 3,
 			(m->mpc_irqflag >> 2) & 3, m->mpc_srcbus,
 			m->mpc_srcbusirq, m->mpc_dstapic, m->mpc_dstirq);
-	if (++mp_irq_entries == MAX_IRQ_SOURCES)
+	if (++mp_irq_entries == max_irq_sources)
 		panic("Max # of irq sources exceeded!!\n");
 }
 
@@ -395,6 +397,9 @@
 	char str[16];
 	int count=sizeof(*mpc);
 	unsigned char *mpt=((unsigned char *)mpc)+count;
+	int num_bus = 0;
+	int num_irq = 0;
+	unsigned char *bus_data;
 
 	if (memcmp(mpc->mpc_signature,MPC_SIGNATURE,4)) {
 		panic("SMP mptable: bad signature [%c%c%c%c]!\n",
@@ -440,9 +445,70 @@
 		mpc_record = 0;
 	}
 
+	/* Pre-scan to determine the number of bus and 
+	 * interrupts records we have
+	 */
+	while (count < mpc->mpc_length) {
+		switch (*mpt) {
+			case MP_PROCESSOR:
+				mpt += sizeof(struct mpc_config_processor);
+				count += sizeof(struct mpc_config_processor);
+				break;
+			case MP_BUS:
+				++num_bus;
+				mpt += sizeof(struct mpc_config_bus);
+				count += sizeof(struct mpc_config_bus);
+				break;
+			case MP_INTSRC:
+				++num_irq;
+				mpt += sizeof(struct mpc_config_intsrc);
+				count += sizeof(struct mpc_config_intsrc);
+				break;
+			case MP_IOAPIC:
+				mpt += sizeof(struct mpc_config_ioapic);
+				count += sizeof(struct mpc_config_ioapic);
+				break;
+			case MP_LINTSRC:
+				mpt += sizeof(struct mpc_config_lintsrc);
+				count += sizeof(struct mpc_config_lintsrc);
+				break;
+			default:
+				count = mpc->mpc_length;
+				break;
+		}
+	}
+	/* 
+	 * Paranoia: Allocate one extra of both the number of busses and number
+	 * of irqs, and make sure that we have at least 4 interrupts per PCI
+	 * slot.  But some machines do not report very many busses, so we need
+	 * to fall back on the older defaults.
+	 */
+	++num_bus;
+	max_mp_busses = max(num_bus, MAX_MP_BUSSES);
+	if (num_irq < (4 * max_mp_busses))
+		num_irq = 4 * num_bus;	/* 4 intr/PCI slot */
+	++num_irq;
+	max_irq_sources = max(num_irq, MAX_IRQ_SOURCES);
+	
+	count = (max_mp_busses * sizeof(int)) * 4;
+	count += (max_irq_sources * sizeof(struct mpc_config_intsrc));
+	bus_data = alloc_bootmem(count);
+	if (!bus_data) {
+		printk(KERN_ERR "SMP mptable: out of memory!\n");
+		return 0;
+	}
+	mp_bus_id_to_type = (int *)&bus_data[0];
+	mp_bus_id_to_node = (int *)&bus_data[(max_mp_busses * sizeof(int))];
+	mp_bus_id_to_local = (int *)&bus_data[(max_mp_busses * sizeof(int)) * 
2];
+	mp_bus_id_to_pci_bus = (int *)&bus_data[(max_mp_busses * sizeof(int)) * 
3];
+	mp_irqs = (struct mpc_config_intsrc *)&bus_data[(max_mp_busses * sizeof
(int)) * 4];
+	memset(mp_bus_id_to_pci_bus, -1, max_mp_busses);
+
 	/*
 	 *	Now process the configuration blocks.
 	 */
+	count = sizeof(*mpc);
+	mpt = ((unsigned char *)mpc)+count;
 	while (count < mpc->mpc_length) {
 		switch(*mpt) {
 			case MP_PROCESSOR:
diff -Nru a/include/asm-i386/io_apic.h b/include/asm-i386/io_apic.h
--- a/include/asm-i386/io_apic.h	Wed Mar  6 22:11:01 2002
+++ b/include/asm-i386/io_apic.h	Wed Mar  6 22:11:02 2002
@@ -97,7 +97,7 @@
 extern int mp_irq_entries;
 
 /* MP IRQ source entries */
-extern struct mpc_config_intsrc mp_irqs[MAX_IRQ_SOURCES];
+extern struct mpc_config_intsrc *mp_irqs;
 
 /* non-0 if default (table-less) MP configuration */
 extern int mpc_default_type;
diff -Nru a/include/asm-i386/mpspec.h b/include/asm-i386/mpspec.h
--- a/include/asm-i386/mpspec.h	Wed Mar  6 22:11:02 2002
+++ b/include/asm-i386/mpspec.h	Wed Mar  6 22:11:02 2002
@@ -197,11 +197,11 @@
 	MP_BUS_PCI,
 	MP_BUS_MCA
 };
-extern int mp_bus_id_to_type [MAX_MP_BUSSES];
-extern int mp_bus_id_to_node [MAX_MP_BUSSES];
-extern int mp_bus_id_to_local [MAX_MP_BUSSES];
+extern int *mp_bus_id_to_type;
+extern int *mp_bus_id_to_node;
+extern int *mp_bus_id_to_local;
+extern int *mp_bus_id_to_pci_bus;
 extern int quad_local_to_mp_bus_id [NR_CPUS/4][4];
-extern int mp_bus_id_to_pci_bus [MAX_MP_BUSSES];
 
 extern unsigned int boot_cpu_physical_apicid;
 extern unsigned long phys_cpu_present_map;
@@ -210,11 +210,9 @@
 extern void get_smp_config (void);
 extern int nr_ioapics;
 extern int apic_version [MAX_APICS];
-extern int mp_bus_id_to_type [MAX_MP_BUSSES];
 extern int mp_irq_entries;
-extern struct mpc_config_intsrc mp_irqs [MAX_IRQ_SOURCES];
+extern struct mpc_config_intsrc *mp_irqs;
 extern int mpc_default_type;
-extern int mp_bus_id_to_pci_bus [MAX_MP_BUSSES];
 extern int mp_current_pci_id;
 extern unsigned long mp_lapic_addr;
 extern int pic_mode;
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Comment 8 Matt Domsch 2002-03-07 10:15:50 EST
Created attachment 47741 [details]
mptable.txt
Comment 9 Michael K. Johnson 2002-03-07 16:34:07 EST
Arjan, this is for the slimmerlot, a critical target for Pensacola...
Comment 10 Bob Matthews 2002-03-08 15:39:04 EST
2.4.9-26beta.40 smp kernel boots on this machine.
Comment 11 Matt Domsch 2002-03-08 15:41:59 EST
Can't wait to get my grubby leettle hands on it.
Comment 12 Matt Domsch 2002-03-08 19:34:58 EST
This kernel works now on both Merlot and Discovery (though I still don't know 
why Discovery started working).  Closing "rawhide".  Thanks for the quick work!
Comment 13 Matt Domsch 2002-03-11 11:02:29 EST
Arjan, this fix is now slated for Pensacola, and I assume Hampton as well.  
What about future 7.2 errata kernels (if/when you release one?)  Dell plans on 
factory installing 7.2 on these platforms, and since this fix is required, 
would like to see this in an official errata kernel (when you produce it - I'm 
not requesting a specific timeframe yet).
Comment 14 Michael K. Johnson 2002-03-28 13:57:22 EST
Re-opening until we release an errata kernel with this patch.
Comment 15 Matt Domsch 2003-02-13 13:14:10 EST
7.3 kernel included a fix; also present in Pensacola.  Closing.

Note You need to log in before you can comment on or make changes to this bug.