I have RedHat 7.0 professional server installed on two new servers. Both servers have 1024 MB of RAM (4 x 256 MB sticks). On previous versions of Red Hat (specifically 6.2) free showed the total Mem: as 1036192. free on Red 7.0 shows a total of 906360. top does as well. A cat of /proc/meminfo shows: total: used: free: shared: buffers: cached: Mem: 928112640 356429824 571682816 32776192 270729216 33497088 Swap: 1085652992 0 1085652992 MemTotal: 906360 kB MemFree: 558284 kB MemShared: 32008 kB Buffers: 264384 kB Cached: 32712 kB BigTotal: 0 kB BigFree: 0 kB SwapTotal: 1060208 kB SwapFree: 1060208 kB Any ideas? Kevin Goddard systemadmin
The 2.2.x kernels have an option to support either 1Gb or 2Gb of memory. 1Gb is actually about 900 Mb (the number you are seeing) so recompiling the kernel with support for 2Gb of ram would solve your problem. It would make sense for Red Hat to ship a default enterprise kernel with this option enabled though.
I believe the enterprise kernel has the 4Gig support from Andrea included. My guess is that this is not the kernel being run. In which case the installer made a possibly poor choice.
I am experiencing this same problem on a system with 2GB of RAM. It is running the enterprise kernel. When this system starts to tap into the 2nd GB, it locks solid. I, too, thought that the enterprise kernel would address more than 1GB of RAM without doing anything special. I even added: append="mem=2048M" to my lilo.conf, re-ran lilo, rebooted, and still no change. Here are some outputs: Linux daa20493wwx001.datareturn.com 2.2.16-22enterprise #1 SMP Tue Aug 22 16:29:32 EDT 2000 i686 unknown [root@daa20593wwx001 /root]# free -t total used free shared buffers cached Mem: 2074272 1031256 1043016 1009428 651648 169304 -/+ buffers/cache: 210304 1863968 Swap: 1052592 836 1051756 Total: 3126864 1032092 2094772 [root@daa20593wwx001 /root]# ls -al /proc/kcore -r-------- 1 root root 939528192 Jan 21 18:59 /proc/kcore As you can see, free reports all 2GB of RAM, but kcore only maps out the 1st GB. I did tinker with the kernel configs and rebuilt all the kernel packages with the 2GB option enabled, and WAS able to physically address the 2nd GB of RAM, but the system this is on can use upwards of 4GB and, since I don't have a 4GB kernel config option, I am left wondering if I can make it see beyond the 2GB limit. I will aquire an additional 2 GB of RAM, test this again and post more info. My server hardware is: Compaq Proliant 5500R Dual PIII Xeon 500 2GB RAM (ECC) Compaq 3200 RAID (which I reported a problem on during install, and haven't heard anything about it since: http://bugzilla.redhat.com/bugzilla/show_bug.cgi? id=21780 )
Saw that you posted this, so I thought I'd add it: [root@daa20593wwx001 /root]# cat /proc/meminfo total: used: free: shared: buffers: cached: Mem: 2124054528 1076674560 1047379968 1072283648 617181184 240381952 Swap: 1077854208 856064 1076998144 MemTotal: 2074272 kB MemFree: 1022832 kB MemShared: 1047152 kB Buffers: 602716 kB Cached: 234748 kB BigTotal: 1179644 kB BigFree: 1019980 kB SwapTotal: 1052592 kB SwapFree: 1051756 kB
I finally managed to track down the additional 2GB of RAM, but I couldn't retain the server for extended testing. We recompiled the kernel for 2GB. We found out that we had to use the i386 compiled SMP kernel with the i686 compiled Enterprise kernel. I found it strange that I HAD to install both the Enterprise kernel and the SMP kernels. If I didn't install the SMP kernel and tried to only use the Enterprise kernel, I would get kernel panics, and if we attempted to use the i686 compiled version of the SMP kernel, it would hang on reading the initrd image. After successfully having the system see and physically map 2GB, we decided to test it at the mythical 4GB point...and found that it is truly mythical. Here are the outputs of procinfo along with the size of /proc/kcore. Linux 2.2.16-22enterprise (root@kernel) (gcc egcs-2.91.66) #1 2CPU [kernel] Memory: Total Used Free Shared Buffers Cached Mem: 3988340 31868 3956472 8436 6728 11988 Swap: 526296 0 526296 Bootup: Thu Feb 1 14:57:11 2001 Load average: 0.15 0.14 0.06 1/20 490 user : 0:00:01.64 0.6% page in : 10950 disk 1: 2554r 918w nice : 0:00:00.00 0.0% page out: 2557 system: 0:00:17.77 6.3% swap in : 1 idle : 0:04:22.81 93.2% swap out: 0 uptime: 0:02:20.96 context : 9855 irq 0: 14111 timer irq 8: 1 rtc irq 1: 220 keyboard irq 9: 33 ncr53c8xx irq 2: 0 cascade [4] irq 10: 30 ncr53c8xx irq 3: 3 irq 11: 3453 ida0 irq 4: 3 irq 13: 1 fpu irq 5: 200 eth0 irq 14: 14 ide0 irq 6: 3 As you can see, procinfo (and I'm sure that /proc/meminfo would have similar stats), at least _sees_ all 4GB of RAM. However, only 2GB are physcally mapped, as you can see by listing the size of /proc/kcore: -r-------- 1 root root 2013270016 Feb 1 14:59 /proc/kcore I had to return the hardware I was testing against, so I didn't have an oportunity to see of the 2.4 kernel had the same limitations. If I don't hear from anyone regarding the status on this, I'll go ahead and open it as a separate bug report.
For >2GB of memory, the kernel will see only 2Gb, and the rest is "anonymous" memory. It's still used, just not by the kernel directly.
If this memory isn't mapped by the kernel, I don't understand how processes can operate in it. We've tested server performance with a 1GB compiled kernel under 2GB, and when we approached the threshold of the beginning of the 2GB point, the server started spitting back Virtual Memory errors and promptly froze. Asside from showing the memory present in /proc/meminfo, having more memory than the kernel will physically address is like cutting a room in half with a seamless glass wall. You can see all the space of the room on both sides of the glass, but you can't use all of it. This idea of "anonymous" doesn't make any sense. What processes can use this memory space? It certainly isn't anything that we need. I couldn't get Apache to utilize it. I still hit the glass ceiling at the 1GB barrier. When we compiled for 2GB and installed a total of 4GB, we had a 2GB glass ceiling. /proc/meminfo happily reported 4GB of RAM, but nothing running on the server could utilize it. Does the 2.4 kernel have these same limitations? Are we all frozen at the 2GB point? Are there config options in the kernel that I can tell it I have 4GB? If this "anonymous" memory is really used, can you please list processes/services/programs that can utilize it?
Shared memory is one example of it. The kernel doesn't map it in the kernel virtual memory table, put processes can and do. The Red Hat 2.2 kernels have the "highmem" patches originally done by SAP. They allow the use of the "high" (>2Gb, upto 4Gb) memory for some purposes, but the kernel cannot see it directly. The 2.4.0 kernel can use upto 64Gb of RAM on x86 hardware, but even there the kernel can only address 1 or 2 Gb directly, there rest is done indirectly. In the 2.4 kernel, the memory > 2Gb is also used for diskcache and the like. Per process limitations are still 3Gb, but several processes together can use more.
If that is true, then in theory, processes like Apache and other daemons can utilize the higher memory via shared pools, but doing things like creating a 2GB RAMdrive on a 4GB server (which we tried to do for testing) will fail, because it tries to directly attach itself to the mapped RAM. Am I on the right track with this? It didn't look like Apache was really using all the available RAM...only the mapped RAM. "Out of the box," without recompiling, how much RAM will the 2.2.16enterprise kernel address via shared memory pools? What can I look for to determine if a process is tapping into the memory that surpasses the mapped portion?
re part1: Yes, Apache and other daemons can use the higher memory. Remember that most malloc() uses in .c are actually implemented using this "shared" (or "anonymous" in kernel terms) memory and will benifit from your RAM. Your ramdisk cannot use it because the kernel can see only the 2Gb (depending on what you chose in the config) and a ramdisk is kernel memory, not "anonymous" or "shared userspace" memory. Out of the box, the enterprise kernel supports the 4Gb, with the note that the upper part of that is "high" memory.
I'm closing this as NOTABUG because the code is working as intended.