Bug 241314 - RHEL 5 hugemem equivalent (4G addressable per process)?
RHEL 5 hugemem equivalent (4G addressable per process)?
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: Deployment_Guide (Show other bugs)
5.0
All Linux
urgent Severity high
: rc
: ---
Assigned To: Douglas Silas
Michael Hideo
https://www.redhat.com/docs/manuals/e...
: Documentation
Depends On:
Blocks: 499522
  Show dependency treegraph
 
Reported: 2007-05-24 21:54 EDT by Michael Hideo
Modified: 2016-06-17 17:13 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-11-26 16:41:33 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Michael Hideo 2007-05-24 21:54:39 EDT
> kernel-PAE (only for i686 systems) — This package offers the following
> key configuration options (in addition to the options already enabled
> for the kernel package): 
> 
>     Support for over 4GB of RAM (up to 64GB for the x86) 
> 
>     PAE (Physical Address Extension) or 3-level paging on x86 processors
> that support PAE 
> 
>     4GB/4GB split: 4GB of virtual address space for the kernel and
> almost 4GB for each user process on x86 systems 
> 
>
https://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Deployment_Guide-en-US/ch-kernel.html#s1-kernel-packages
> 
> So, shouldn't PAE kernel give 4GB per user process?
> ---
> 
> Does this mean our website needs to be updated?
> 

Ouch, yes, that part of the website is wrong, wrong, wrong.  We don't
have the 4G/4G split in RHEL-5 at all.  As Dave said, the solution is
either to use 64-bit or go with RHEL-4.  We should definitely get the
website fixed.
Comment 2 Michael Hideo 2007-10-22 22:50:09 EDT
Removing automation notification
Comment 3 Don Domingo 2008-01-31 20:10:10 EST
erroneous slab of text already removed in RHEl5.2 and RHEL5.1 source. 

also, removed listitem mentioning 4GB/4GB split in RHEL5.0.

setting this bug as MODIFIED. 
Comment 5 Akemi Yagi 2008-11-11 15:21:25 EST
It looks like the 4GB/4GB blurb still exists in Deployment_Guide 5.2-11.el5 ?
Comment 7 Michael Hideo 2009-01-14 18:21:56 EST
David, it will. There are some activities that will get the desktop and the website docs in sync. ETA is 1-2 weeks. - Mike
Comment 9 Douglas Silas 2009-07-28 16:14:04 EDT
I am trying to fix a number of high pri/sev bugs filed against the Deployment Guide. I see the erroneous section quoted above in current online instances of the Deployment Guide, but I'm unsure what the original correction/patch was. Can anyone provide me with the correct text for this section?

Here is the current incorrect text:
[SNIP]
kernel-PAE (only for i686 systems) — This package offers the following key configuration options (in addition to the options already enabled for the kernel package):

    *
      Support for over 4GB of RAM (up to 16GB for the x86)
    *
      PAE (Physical Address Extension) or 3-level paging on x86 processors that support PAE
    *
      4GB/4GB split: 4GB of virtual address space for the kernel and almost 4GB for each user process on x86 systems 

kernel-PAE-devel — Contains the kernel headers and makefiles required to build modules against the kernel-PAE package
[/SNIP]

As regards the correct text for this section, I know the following:

* kernel-pae (and kernel-pae-devel) are only available for i686.
* kernel-pae provides a kernel with support for >4GB RAM and up to 16GB.

I'm surmising that RHEL5 doesn't have the 4GB/4GB feature (which changes the virtual memory behavior of PAE kernels), but I'm not sure of the reason for this comment: "as Dave said, the solution is either to use 64-bit or go with RHEL-4." Can anyone provide the reason why that is the solution so that I can fix the source and update the online DGs?

Thanks,
Silas
Comment 10 Chris Lalancette 2009-07-29 06:28:48 EDT
(In reply to comment #9)
> I am trying to fix a number of high pri/sev bugs filed against the Deployment
> Guide. I see the erroneous section quoted above in current online instances of

Awesome, I would love to get this years-old bug fixed and closed.

> the Deployment Guide, but I'm unsure what the original correction/patch was.
> Can anyone provide me with the correct text for this section?
> 
> Here is the current incorrect text:
> [SNIP]
> kernel-PAE (only for i686 systems) — This package offers the following key
> configuration options (in addition to the options already enabled for the
> kernel package):
> 
>     *
>       Support for over 4GB of RAM (up to 16GB for the x86)
>     *
>       PAE (Physical Address Extension) or 3-level paging on x86 processors that
> support PAE
>     *
>       4GB/4GB split: 4GB of virtual address space for the kernel and almost 4GB
> for each user process on x86 systems 
> 
> kernel-PAE-devel — Contains the kernel headers and makefiles required to build
> modules against the kernel-PAE package
> [/SNIP]
> 
> As regards the correct text for this section, I know the following:
> 
> * kernel-pae (and kernel-pae-devel) are only available for i686.
> * kernel-pae provides a kernel with support for >4GB RAM and up to 16GB.
> 
> I'm surmising that RHEL5 doesn't have the 4GB/4GB feature (which changes the
> virtual memory behavior of PAE kernels), but I'm not sure of the reason for
> this comment: "as Dave said, the solution is either to use 64-bit or go with
> RHEL-4." Can anyone provide the reason why that is the solution so that I can
> fix the source and update the online DGs?

Yes, basically everything you say above it true.  Let me see if I can explain the whole situation (everything below deals strictly with i686 unless I say otherwise):

The original i686 only had 32-bits with which to address the memory in a machine.  Because 2^32 == 4GB, the original i686 can only address up to 4GB of memory.

Sometime later on, Intel introduced the PAE extensions for i686.  This means that although the processor still only runs 32-bit code, it can use up to 36-bits to address memory (they added more bits later on, but that's not super-relevant).  So newer i686 processor could address 2^36 bits of memory, or 64GB.

However, the linux kernel has limitations that makes it essentially impossible to run with 64GB of memory reliably (if you want more details about why, I can dig it up).  Therefore, while running the linux kernel on an i686 PAE machine, you can really only use up to about 16GB of memory reliably.

To work around that limitation, RHEL-4 (and RHEL-3) had a new mode called "hugemem".  Among other things, it allowed the kernel to reliably address all 64GB of memory.

The problem is that the hugemem patches were never accepted upstream.  Because of that, and because of the fact that 64-bit machines were so prevalent by the time RHEL-5 was released, we decided to drop the hugemem variant from RHEL-5.  So that leads us to the following situation:

RHEL-4: i686 - no PAE, no hugemem patches, can address up to 4GB memory
        i686-smp - PAE, no hugemem patches, can reliably run with around 16GB
        i686-hugemem - PAE, hugemem patches, can reliably run with 64GB

RHEL-5: i686 - no PAE, no hugemem patches, can address up to 4GB of memory
        i686-PAE - PAE, no hugemem patches, can reliably run with around 16GB

So, in summary, if customers need to use > 16GB of memory, the absolute best suggestion is to use RHEL-5 x86_64, which suffers from none of these limitations.  A secondary idea is to use RHEL-4 i686 hugemem, but as RHEL-4 gets closer to end-of-life (still a few years off, but hardware enablement will slow and stop soon), this becomes less and less a good idea.

Does all of this make sense?  If not, please feel free to ask me for additional clarification, since I would really like to see this fixed (and closed).

Chris Lalancette
Comment 11 Douglas Silas 2009-08-21 12:40:30 EDT
Thanks for your very explanative comment Chris. We should be able to close this and update the online books in the next few days. Based on your description, here is my suggested text (which will replace the original snippet above):

[snip]
kernel-PAE (only for i686 systems) — This package offers the following key configuration option (in addition to the options already enabled for the kernel package):

    * PAE (Physical Address Extension) support for systems with more than 4GB of RAM, and reliably up to 16GB.
      
[IMPORTANT:] Physical Address Extension allows x86 processors to address up to 64GB of physical RAM, but due to differences between the Red Hat Enterprise Linux 4 and 5 kernels, only Red Hat Enterprise Linux 4 (with kernel-PAE package) is able to reliably address all 64GB of memory. However, the x86_64 kernel does not suffer from these limitations, and is the suggested Red Hat Enterprise Linux 5 architecture to use with machines with large-memory systems. 

kernel-PAE-devel — Contains the kernel headers and makefiles required to build modules against the kernel-PAE package
[snip]

Assuming that is all correct, the other consideration would be whether we need to explicitly state that customers with i686 systems who want to address >16GB should use the RHEL4 kernel-PAE packages. That is implicit in the text now. Comments or suggestions?
Comment 12 Chris Lalancette 2009-08-24 04:46:04 EDT
(In reply to comment #11)
> Thanks for your very explanative comment Chris. We should be able to close this
> and update the online books in the next few days. Based on your description,
> here is my suggested text (which will replace the original snippet above):

Actually, I completely left out one important area of discussion, which is the reason for part of the title for this bug "(4G addressable per process)".  Above I mentioned that the hugemem variant allowed you to access up to 64GB of memory "among other things".  The other thing it allowed, in particular, is for userspace to address 4G of memory per process, instead of the usual 3G.  This is another benefit that some customers found useful.  There are also a couple of typos in the above paragraph.  I'll reproduce what I think the re-written paragraph should look like below:

[snip]
kernel-PAE (only for i686 systems) — This package offers the following key
configuration option (in addition to the options already enabled for the kernel
package):

    * PAE (Physical Address Extension) support for systems with more than 4GB
of RAM, and reliably up to 16GB.

[IMPORTANT:] Physical Address Extension allows x86 processors to address up to
64GB of physical RAM, but due to differences between the Red Hat Enterprise
Linux 4 and 5 kernels, only Red Hat Enterprise Linux 4 (with kernel-hugemem
package) is able to reliably address all 64GB of memory. Additionally, the Red Hat Enterprise Linux 5 PAE variant does not allow 4G of addressable memory per-process like the Red Hat Enterprise Linux 4 kernel-hugemem variant does.  However, the x86_64 kernel does not suffer from any of these limitations, and is the suggested Red Hat Enterprise Linux 5 architecture to use with large-memory systems. 

kernel-PAE-devel — Contains the kernel headers and makefiles required to build
modules against the kernel-PAE package
[snip]

Note the changes: fix up the discussion to talk about "kernel-hugemem" for RHEL-4, add the sentence about 4G addressable per process, and fix the last sentence in the paragraph to be a little more clear.

Chris Lalancette
Comment 13 Douglas Silas 2009-08-24 17:42:57 EDT
Thanks for the corrections & clarifications Chris. Was trying to write the text while a bit tired.

Fixed now in 5.4 -r 21313 as per your text with exceptions "with [the] kernel-hugemem package" and "4G" -> "4GB".
Comment 14 Chris Lalancette 2009-08-25 02:22:53 EDT
Excellent, thank you for moving this along!

Chris Lalancette
Comment 15 Douglas Silas 2009-09-07 20:40:49 EDT
The revised text is now in the Deployment Guide linked off of redhat.com/docs, but due to a formatting bug (admonitions require titles, or else CSS/fonts are off), the text in the "Important" box is cut off.

Note that you can highlight the text and read all of it.

I am not closing this until I update the online copy again, hopefully soon.
Comment 17 Chris Lalancette 2009-10-22 11:01:31 EDT
Douglas,
     It looks like this was synced to the external site (which is good).  I did notice one problem, though; the text that you added seems to be showing up in a box that is too small to show it, so it's flowing underneath the rest of the text.  I'm looking at it with Firefox 3.0.14 on F-10.  Can you take a look and possibly fix it?  Once that is fixed, we can probably close out this bug.

Thanks,
Chris Lalancette

Note You need to log in before you can comment on or make changes to this bug.