Bug 12818 - bash fails to process large here-documents
bash fails to process large here-documents
Status: CLOSED RAWHIDE
Product: Red Hat Linux
Classification: Retired
Component: bash (Show other bugs)
6.0
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Bernhard Rosenkraenzer
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2000-06-22 15:30 EDT by steve moss
Modified: 2008-05-01 11:37 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2000-06-22 15:30:07 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description steve moss 2000-06-22 15:30:05 EDT
The bash shell fails to process here documents in the form of :

uudecode <<'EOF'
begin 644 ./tmpfile3
......<uuencoded contents>
..........

EOF

when the contents approach 7-8 GB. Using the testcase below fails on the 
here document file using the bash shell but passes using the zsh. The 
testcase creates a large file, uuencodes it, verifies, then create a file 
containing the uuencoded file wrapped around the uudecode here document 
construct. This is where the bash shell hangs. 

-----Cut here-------
#!/bin/sh
  # -*-Shell-script-*-

  size_iter=$1

  if [ "$size_iter" = "" ]; then
      echo ERROR: number of iterations not specified
      exit 1
  fi



  tmp1=./tmpfile1
  tmp2=./tmpfile2
  tmp3=./tmpfile3
  tmp4=./tmpfile4

  echo
  echo
  echo Creating a large $tmp1 file of carriage return codes via  
  echo concatentaion...

  echo >$tmp1
  num_dups=0
  while expr $num_dups \< $size_iter >/dev/null 2>&1; do
      cat $tmp1 $tmp1 > $tmp2
      cp $tmp2 $tmp1
      echo "duplication # $num_dups"
      num_dups=`expr $num_dups + 1`
  done
  # no need to keep $tmp2 around to consume space:
  rm -f $tmp2
  echo $tmp1 filesize is ...
  ls -l $tmp1

  echo
  echo
  echo Now uuencoding $tmp1 into ${tmp3}.uu that will create into $tmp3 
  echo when uudecoded...
  cat $tmp1 | uuencode $tmp3 >${tmp3}.uu


  echo
  echo
  echo Embedding $tmp3 as a here-document
  cat /dev/null >$tmp4
  echo "#!/bin/sh" >>$tmp4
  echo "" >>$tmp4
  # note single quoted EOF keyword disables expansion of shell variables 
  #inside of heredoc
  echo "uudecode  <<'EOF'" >>$tmp4
  cat < ${tmp3}.uu   >>$tmp4
  echo "EOF" >>$tmp4

  echo
  echo
  echo Double checking uuencoded file ${tmp3}.uu decodes correctly
  rm -f $tmp3
  uudecode ${tmp3}.uu

 if diff $tmp1 $tmp3 >/dev/null 2>&1; then
     echo PASS: $tmp1 is not different from $tmp3 when uudecoded
 else
     echo FAIL: $tmp1 is different from $tmp3 when uudecoded
     exit 1
 fi


 echo
 echo
 echo Creating the $tmp3 file using the $tmp4 shell script ...
 rm -f $tmp3
 /bin/sh $tmp4


 echo
 echo
 echo There should be no difference between $tmp1 and $tmp3 files
 if diff $tmp1 $tmp3 >/dev/null 2>&1; then
     echo PASS: $tmp1 is not different from $tmp3 when uudecoded from here 
     echo doc
 else
     echo FAIL: $tmp1 is different from $tmp3 when uudecoded from here doc
     exit 1
 fi
Comment 1 Bernhard Rosenkraenzer 2000-06-26 14:04:29 EDT
This is fixed in bash 2.x.

Note You need to log in before you can comment on or make changes to this bug.