Bug 803292 - some generators (at least /dev/{u,}random) are broken
Summary: some generators (at least /dev/{u,}random) are broken
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dieharder
Version: 17
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: Jiri Hladky
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-03-14 11:44 UTC by Sami Farin
Modified: 2014-02-26 13:51 UTC (History)
2 users (show)

Fixed In Version: dieharder-3.31.1-9.fc19
Clone Of:
Environment:
Last Closed: 2013-08-01 17:04:47 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Fixed source code for /dev/urandom (501) generator (1.37 KB, text/x-csrc)
2012-03-16 00:27 UTC, Jiri Hladky
no flags Details

Description Sami Farin 2012-03-14 11:44:26 UTC
Description of problem:

seems like some initialization glitch

...
diehard_count_1s_byt|   0|    256000|     100|0.67593396|  PASSED  
 diehard_parking_lot|   0|     12000|     100|0.00000000|  FAILED  
    diehard_2dsphere|   2|      8000|     100|0.00000000|  FAILED  
    diehard_3dsphere|   3|      4000|     100|0.00000000|  FAILED  
     diehard_squeeze|   0|    100000|     100|0.00000000|  FAILED  
        diehard_sums|   0|       100|     100|0.00000000|  FAILED  
        diehard_runs|   0|    100000|     100|0.36067702|  PASSED  
...

$ dieharder -g 501 -d diehard_parking_lot -c ' ' -D default -D histogram -D description
#=============================================================================#
#            dieharder version 3.31.1 Copyright 2003 Robert G. Brown          #
#=============================================================================#
   rng_name     rands/second    Seed    
   /dev/urandom   3.53e+06    147345349 
#==================================================================
#             Diehard Parking Lot Test (modified).
# This tests the distribution of attempts to randomly park a
# square car of length 1 on a 100x100 parking lot without
# crashing.  We plot n (number of attempts) versus k (number of
# attempts that didn't "crash" because the car squares 
# overlapped and compare to the expected result from a perfectly
# random set of parking coordinates.  This is, alas, not really
# known on theoretical grounds so instead we compare to n=12,000
# where k should average 3523 with sigma 21.9 and is very close
# to normally distributed.  Thus (k-3523)/21.9 is a standard
# normal variable, which converted to a uniform p-value, provides
# input to a KS test with a default 100 samples.
#==================================================================
#=============================================================================#
#                         Histogram of test p-values                          #
#=============================================================================#
# Bin scale = 0.100000
#    120|    |    |    |    |    |    |    |    |    |    |
#       |    |    |    |    |    |    |    |    |    |    |
#    108|    |    |    |    |    |    |    |    |    |    |
#       |    |    |    |    |    |    |    |    |    |    |
#     96|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#     84|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#     72|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#     60|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#     48|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#     36|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#     24|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#     12|****|    |    |    |    |    |    |    |    |    |
#       |****|    |    |    |    |    |    |    |    |    |
#       |--------------------------------------------------
#       | 0.1| 0.2| 0.3| 0.4| 0.5| 0.6| 0.7| 0.8| 0.9| 1.0|
#=============================================================================#
#=============================================================================#
        test_name    ntup  tsamples  psamples   p-value  Assessment
#=============================================================================#
 diehard_parking_lot    0      12000      100 0.00000000   FAILED  


$ dieharder -g 200 -d diehard_parking_lot -c ' ' -D default -D histogram -D description < /dev/urandom 
#=============================================================================#
#            dieharder version 3.31.1 Copyright 2003 Robert G. Brown          #
#=============================================================================#
   rng_name     rands/second    Seed    
stdin_input_raw   1.16e+07   2908051647 
#==================================================================
#             Diehard Parking Lot Test (modified).
# This tests the distribution of attempts to randomly park a
# square car of length 1 on a 100x100 parking lot without
# crashing.  We plot n (number of attempts) versus k (number of
# attempts that didn't "crash" because the car squares 
# overlapped and compare to the expected result from a perfectly
# random set of parking coordinates.  This is, alas, not really
# known on theoretical grounds so instead we compare to n=12,000
# where k should average 3523 with sigma 21.9 and is very close
# to normally distributed.  Thus (k-3523)/21.9 is a standard
# normal variable, which converted to a uniform p-value, provides
# input to a KS test with a default 100 samples.
#==================================================================
#=============================================================================#
#                         Histogram of test p-values                          #
#=============================================================================#
# Bin scale = 0.100000
#     20|    |    |    |    |    |    |    |    |    |    |
#       |    |    |    |    |    |    |    |    |    |    |
#     18|    |    |    |    |    |    |    |    |    |    |
#       |    |    |    |    |    |****|    |    |    |    |
#     16|    |    |    |    |    |****|    |    |    |    |
#       |    |    |    |    |    |****|    |****|    |    |
#     14|    |    |    |    |    |****|    |****|    |    |
#       |    |    |    |    |    |****|    |****|    |    |
#     12|    |    |    |    |    |****|    |****|    |    |
#       |    |****|    |    |    |****|    |****|****|    |
#     10|    |****|****|    |    |****|    |****|****|    |
#       |    |****|****|    |    |****|****|****|****|    |
#      8|    |****|****|    |    |****|****|****|****|****|
#       |****|****|****|****|    |****|****|****|****|****|
#      6|****|****|****|****|    |****|****|****|****|****|
#       |****|****|****|****|****|****|****|****|****|****|
#      4|****|****|****|****|****|****|****|****|****|****|
#       |****|****|****|****|****|****|****|****|****|****|
#      2|****|****|****|****|****|****|****|****|****|****|
#       |****|****|****|****|****|****|****|****|****|****|
#       |--------------------------------------------------
#       | 0.1| 0.2| 0.3| 0.4| 0.5| 0.6| 0.7| 0.8| 0.9| 1.0|
#=============================================================================#
#=============================================================================#
        test_name    ntup  tsamples  psamples   p-value  Assessment
#=============================================================================#
 diehard_parking_lot    0      12000      100 0.24520972   PASSED  


Version-Release number of selected component (if applicable):
3.31.1-5

How reproducible:
100%

Steps to Reproduce:
1. dieharder -a -g 500 or -g 501
2. dieharder -a -g 200 < /dev/urandom
3. compare
  
Actual results:
inconsistent results with -g 500 and -g 501

Expected results:
same as in step 2.

Additional info:

Comment 1 Jiri Hladky 2012-03-15 01:08:59 UTC
I have reproduced the problem on different boxes running different OSes. It seems like a bug in the software.

I will try to investigate it further and contact developers if needed. I will keep you posted.

Thanks
Jirka

Comment 2 Jiri Hladky 2012-03-16 00:23:45 UTC
Hi,

my findings are that
-g 500 and -g 501 works fine on 32-bit Linux systems

but failing on all 64-bit OSes.

I have dived into the source code and it seems I have found a route cause. Check the files
libdieharder/rng_dev_random.c
libdieharder/rng_dev_urandom.c
etc

static unsigned long int
dev_urandom_get (void *vstate)
will return 64-bit integer on 64-bit linux but UINT_MAX is just 32-bit integer.

I would propose to review all files where UINT_MAX is used and consider using

#include <stdint.h>
uint32_t
UINT32_MAX

instead of

unsigned long int
and UINT_MAX respectively.

Fixed file libdieharder/rng_dev_urandom.c is attached. 

I have informed the upstream and I will wait for their reaction to decide the next actions.

Thanks
Jirka

Comment 3 Jiri Hladky 2012-03-16 00:27:09 UTC
Created attachment 570454 [details]
Fixed source code for /dev/urandom (501) generator

Fixed file libdieharder/rng_dev_urandom.c

Added:
#include <stdint.h>

Changed unsigned long int => uint32_t
Changed UINT_MAX => UINT32_MAX

Comment 4 Fedora End Of Life 2013-07-04 05:49:55 UTC
This message is a reminder that Fedora 17 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 17. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '17'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 17's end of life.

Bug Reporter:  Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 17 is end of life. If you 
would still like  to see this bug fixed and are able to reproduce it 
against a later version  of Fedora, you are encouraged  change the 
'version' to a later Fedora version prior to Fedora 17's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 5 Fedora End Of Life 2013-08-01 17:04:51 UTC
Fedora 17 changed to end-of-life (EOL) status on 2013-07-30. Fedora 17 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 6 Fedora Update System 2014-01-04 04:12:00 UTC
dieharder-3.31.1-9.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/dieharder-3.31.1-9.fc19

Comment 7 Fedora Update System 2014-01-04 04:20:56 UTC
dieharder-3.31.1-9.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/dieharder-3.31.1-9.fc20

Comment 8 Jiri Hladky 2014-01-06 21:47:57 UTC
Fixed in F19 and F20

Comment 9 Fedora Update System 2014-02-26 13:47:32 UTC
dieharder-3.31.1-9.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2014-02-26 13:51:54 UTC
dieharder-3.31.1-9.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.