Bug 222483 - HTS: Memory test fails. Cannot allocate memory
HTS: Memory test fails. Cannot allocate memory
Status: CLOSED CURRENTRELEASE
Product: Red Hat Hardware Certification Program
Classification: Red Hat
Component: Test Suite (tests) (Show other bugs)
5
All Linux
medium Severity high
: ---
: ---
Assigned To: Greg Nichols
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-01-12 15:32 EST by Gregory Feiner
Modified: 2008-07-16 17:56 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-02-05 10:45:22 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
test results (864.84 KB, application/octet-stream)
2007-01-12 15:32 EST, Gregory Feiner
no flags Details
output of cat /proc/meminfo (702 bytes, text/plain)
2007-01-12 15:32 EST, Gregory Feiner
no flags Details
cpu info. (1021 bytes, text/plain)
2007-01-24 10:27 EST, Gregory Feiner
no flags Details
xm demsg output (5.65 KB, text/plain)
2007-01-24 10:27 EST, Gregory Feiner
no flags Details
threaded_memtest log file (62.22 KB, application/octet-stream)
2007-01-30 16:42 EST, Gregory Feiner
no flags Details

  None (edit)
Description Gregory Feiner 2007-01-12 15:32:14 EST
Description of problem:
Memory test fails due to being unable allocate all the memory it calculated.  
The system in question has 2GB of RAM.  According to messages from the virt 
test, 1GB is min, and 2GB is recommended so I would expect the memory test to 
work.

Version-Release number of selected component (if applicable): 5.0-15


How reproducible:
Every time.

Steps to Reproduce:
1. Install OS 32bit 12-18-2006 build (RC5 snapshot) with 2GB RAM
2. Install kit
3. Run the memory test.
  
Actual results:
fails 

Expected results:
pass

Additional info:
Comment 1 Gregory Feiner 2007-01-12 15:32:15 EST
Created attachment 145485 [details]
test results
Comment 2 Gregory Feiner 2007-01-12 15:32:58 EST
Created attachment 145487 [details]
output of cat /proc/meminfo
Comment 3 Greg Nichols 2007-01-24 10:21:35 EST
What hardware is under test?

- Thanks!
Comment 4 Gregory Feiner 2007-01-24 10:24:20 EST
ProLiant DL320 G5.  2GB RAM using 4x512MB.
Comment 5 Gregory Feiner 2007-01-24 10:27:15 EST
Created attachment 146410 [details]
cpu info.
Comment 6 Gregory Feiner 2007-01-24 10:27:38 EST
Created attachment 146411 [details]
xm demsg output
Comment 7 Greg Nichols 2007-01-30 15:21:17 EST
Please install HTS 5.0 R21, and run the failing part of 
the test manually, ala:

> cd /usr/share/hts/tests/memory
> make threaded_memtest
> ./threaded_memtest -m100% -v

and attach the output log.

Thanks!
Comment 8 Gregory Feiner 2007-01-30 16:42:39 EST
Created attachment 146966 [details]
threaded_memtest log file
Comment 9 Gregory Feiner 2007-01-30 16:44:34 EST
I attached the log file. I could not figure out where the test binary puts its 
logfile (if it even creates one).  So I captured output by redirecting the 
console output to a file.  If this is not sufficient, let me know where the 
threaded_memtest binary puts its log file.
Comment 10 Gregory Feiner 2007-01-30 16:46:07 EST
Looking at the log file, it looks like the test ran just fine.  However, 
running the test the normal way (hts certify --test=memory) results in the 
same old mmap failure.
Comment 11 Jay Turner 2007-01-30 17:02:36 EST
I'm seeing the same behavior on a T60p laptop with 2G RAM.  The threaded_memtest
run completes successfully, but running the "hts certify --test memory --noinfo"
command fails.
Comment 12 Jay Turner 2007-01-31 07:54:26 EST
Some additional information . . . I'm able to directly run "./MEMORY2" on the
box successfully, which leads me to conclude that something is getting hosed
with the HTS environment itself and the way things are being run.
Comment 13 Jay Turner 2007-01-31 09:52:08 EST
And lastly, some strace output . . . appears that the mmap in one of the 4
threads is what's failing:

set_robust_list(0x40815be0, 0xc)        = 0
futex(0x804b464, 0x5 /* FUTEX_??? */, 1) = 1
sched_setaffinity(0, 128,  { 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 })         = 0
mmap2(NULL, 531103744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) 
 = -1 ENOMEM (Cannot allocate memory)
dup(2)                                  = 3
fcntl64(3, F_GETFL)                     = 0x1 (flags O_WRONLY)
close(3)                                = 0
write(2, "mmap: Cannot allocate memory\n", 29) = 29
write(1, "basename is threaded_memtest\nsiz"..., 155) = 155
exit_group(1)                           = ?
Comment 15 Jay Turner 2007-02-02 07:41:52 EST
With -22 I'm still getting the failure running under hts and am still able to
run MEMORY2 directly with success.  I'm also seeing different behavior than
described above:

# hts certify --test memory --noinfo
loaded configuration /var/hts/config.xml
loaded plan /var/hts/plan.xml
loaded results /var/hts/results.xml
running only test memory 
running memory on /org/freedesktop/Hal/devices/computer
mkdir -p /tmp/hts-memory-imAjoz/mnt/tests/HTS/hts/memory
cp -a testinfo.desc runtest.sh Makefile  threaded_memtest.c MEMORY2 Makefile
/tmp/hts-memory-imAjoz/mnt/tests/HTS/hts/memory
cp: warning: source file `Makefile' specified more than once
install -m 0755 runtest.sh /tmp/hts-memory-imAjoz/mnt/tests/HTS/hts/memory
make OUTPUTFILE=/var/log/hts/runs/20/memory/output.log RUNMODE=normal
UDI=/org/freedesktop/Hal/devices/computer run
cc -Wall -lpthread threaded_memtest.c -o threaded_memtest
chmod a+x ./runtest.sh
./runtest.sh
/tmp/hts-memory-imAjoz/mnt/tests/HTS/hts/memory/MEMORY2
Running ./MEMORY2:
Total system RAM: 2026MB
Guessing available memory: 1478MB available
Starting multithreaded memtest - dipping into swap
mmap: Cannot allocate memory
Warning: memsize > free_mem. You will probably hit swap.
Detected 2 processors.
RAM: 89.1% free (1805M/2026M)
Testing 2026M RAM for 10 seconds using 4 threads:
thread 0: mapping 506M RAM
thread 1: mapping 506M RAM
thread 2: mapping 506M RAM
thread 3: mapping 506M RAM
errno: 22: Invalid argument
Free memory 1183M
...finished running ./MEMORY2, exit code=1
recovered exit code=1
rhts-report-result /HTS/hts/memory FAIL /var/log/hts/runs/20/memory/output.log 
saveOutput: /var/log/hts/runs/20/memory/output.log
Return value was 0

# cat /var/log/hts/runs/20/memory/output.log
Running ./MEMORY2:
Total system RAM: 2026MB
Guessing available memory: 1478MB available
Starting multithreaded memtest - dipping into swap
mmap: Cannot allocate memory
Warning: memsize > free_mem. You will probably hit swap.
Detected 2 processors.
RAM: 89.1% free (1805M/2026M)
Testing 2026M RAM for 10 seconds using 4 threads:
thread 0: mapping 506M RAM
thread 1: mapping 506M RAM
thread 2: mapping 506M RAM
thread 3: mapping 506M RAM
errno: 22: Invalid argument
Free memory 1183M
...finished running ./MEMORY2, exit code=1
Comment 17 Jay Turner 2007-02-02 14:08:54 EST
Ah, I think I failed to post the rest of my comment.  Yes, running via hts is
still failing, but I'm able to run MEMORY2 directly without any problems:

# ./MEMORY2 
Total system RAM: 2026MB
Guessing available memory: 1265MB available
Starting multithreaded memtest - dipping into swap
Warning: memsize > free_mem. You will probably hit swap.
Detected 2 processors.
RAM: 28.3% free (574M/2026M)
Testing 2026M RAM for 10 seconds using 4 threads:
thread 0: mapping 506M RAM
thread 1: mapping 506M RAM
thread 2: mapping 506M RAM
thread 3: mapping 506M RAM
thread 2: mapping complete
thread 0: mapping complete
thread 3: mapping complete
thread 1: mapping complete
thread 2: test start
thread 0: test start
thread 3: test start
thread 1: test start
thread 1 finished.
thread 0 finished.
thread 3 finished.
thread 2 finished.
thread 2 unmapping and exiting
thread 1 unmapping and exiting
thread 0 unmapping and exiting
thread 3 unmapping and exiting
Runtime was 10.32s
thread 0: 980 loops
thread 1: 909 loops
thread 2: 923 loops
thread 3: 978 loops
Total loops per second: 367.41
Testing complete.
Comment 18 Greg Nichols 2007-02-02 16:49:23 EST
The problem is that the test tries to map 2GB of memory, but there is
no garentee that 2GB of contiguous address space will be available for
an i386, given that /proc/sys/vm/overcommit_memory defaults to 0

Fixed R23 - changed the test to use multiple processes allocating 1GB
for all i386 machines.
Comment 20 Jay Turner 2007-02-05 07:56:25 EST
Confirm that 5.0-23 is working for me as well.  One thing I noticed is that it
appears some debugging output is still enabled with -23.  Throwing back into
assigned to get this looked at.

Testing memory bandwidth using 1024MB of RAM.
0.000512 29569.49
0.001024 30734.15
0.002048 31284.20
0.004096 31413.03
0.008192 31715.49
0.016384 31776.02
0.032768 30734.00
0.065536 12700.13
0.131072 12647.51
0.262144 12631.14
0.524288 12640.26
536.87 4159.63
1073.74 4138.95
0.000512 28334.79
0.001024 29533.12
0.002048 30220.10
0.004096 30559.88
0.008192 30588.93
0.016384 30829.44
0.032768 29930.01
0.065536 9777.93
0.131072 9780.43
0.262144 9783.27
0.524288 9775.83
536.87 1403.25
1073.74 1421.77
0.000512 9063.78
0.001024 9181.37
0.002048 9065.83
0.004096 9267.36
0.008192 9105.18
0.016384 9268.29
0.032768 8814.78
0.065536 6626.23
0.131072 6771.42
0.262144 6663.06
0.524288 6763.16
536.87 1427.83
1073.74 1446.36
0.000512 3197.46
0.001024 5297.83
0.002048 7887.62
0.004096 10408.91
0.008192 12427.04
0.016384 13751.12
0.032768 14290.21
0.065536 11309.94
0.131072 11472.96
0.262144 11528.05
0.524288 11475.91
536.87 2188.30
1073.74 2187.76
0.000512 22349.63
0.001024 24381.86
0.002048 25378.99
0.004096 25950.06
0.008192 26263.64
0.016384 25322.36
0.032768 6615.53
0.065536 6567.80
0.131072 6610.88
0.262144 6617.87
0.524288 6124.86
536.87 1194.67
0.000512 2657.12
0.001024 4387.69
0.002048 6508.28
0.004096 8565.31
0.008192 10230.00
0.016384 10721.68
0.032768 6405.33
0.065536 6499.14
0.131072 6553.60
0.262144 6553.60
0.524288 5453.22
536.87 1254.44
Comment 21 Greg Nichols 2007-02-05 10:44:42 EST
Moving to Verified. - The output is from lmbench, the package used for memory
testing.

Note You need to log in before you can comment on or make changes to this bug.