Bug 12818

Summary: bash fails to process large here-documents
Product: [Retired] Red Hat Linux Reporter: steve moss <steve_moss>
Component: bashAssignee: Bernhard Rosenkraenzer <bero>
Status: CLOSED RAWHIDE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 6.0CC: steve_moss
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2000-06-22 19:30:07 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description steve moss 2000-06-22 19:30:05 UTC
The bash shell fails to process here documents in the form of :

uudecode <<'EOF'
begin 644 ./tmpfile3
......<uuencoded contents>
..........

EOF

when the contents approach 7-8 GB. Using the testcase below fails on the 
here document file using the bash shell but passes using the zsh. The 
testcase creates a large file, uuencodes it, verifies, then create a file 
containing the uuencoded file wrapped around the uudecode here document 
construct. This is where the bash shell hangs. 

-----Cut here-------
#!/bin/sh
  # -*-Shell-script-*-

  size_iter=$1

  if [ "$size_iter" = "" ]; then
      echo ERROR: number of iterations not specified
      exit 1
  fi



  tmp1=./tmpfile1
  tmp2=./tmpfile2
  tmp3=./tmpfile3
  tmp4=./tmpfile4

  echo
  echo
  echo Creating a large $tmp1 file of carriage return codes via  
  echo concatentaion...

  echo >$tmp1
  num_dups=0
  while expr $num_dups \< $size_iter >/dev/null 2>&1; do
      cat $tmp1 $tmp1 > $tmp2
      cp $tmp2 $tmp1
      echo "duplication # $num_dups"
      num_dups=`expr $num_dups + 1`
  done
  # no need to keep $tmp2 around to consume space:
  rm -f $tmp2
  echo $tmp1 filesize is ...
  ls -l $tmp1

  echo
  echo
  echo Now uuencoding $tmp1 into ${tmp3}.uu that will create into $tmp3 
  echo when uudecoded...
  cat $tmp1 | uuencode $tmp3 >${tmp3}.uu


  echo
  echo
  echo Embedding $tmp3 as a here-document
  cat /dev/null >$tmp4
  echo "#!/bin/sh" >>$tmp4
  echo "" >>$tmp4
  # note single quoted EOF keyword disables expansion of shell variables 
  #inside of heredoc
  echo "uudecode  <<'EOF'" >>$tmp4
  cat < ${tmp3}.uu   >>$tmp4
  echo "EOF" >>$tmp4

  echo
  echo
  echo Double checking uuencoded file ${tmp3}.uu decodes correctly
  rm -f $tmp3
  uudecode ${tmp3}.uu

 if diff $tmp1 $tmp3 >/dev/null 2>&1; then
     echo PASS: $tmp1 is not different from $tmp3 when uudecoded
 else
     echo FAIL: $tmp1 is different from $tmp3 when uudecoded
     exit 1
 fi


 echo
 echo
 echo Creating the $tmp3 file using the $tmp4 shell script ...
 rm -f $tmp3
 /bin/sh $tmp4


 echo
 echo
 echo There should be no difference between $tmp1 and $tmp3 files
 if diff $tmp1 $tmp3 >/dev/null 2>&1; then
     echo PASS: $tmp1 is not different from $tmp3 when uudecoded from here 
     echo doc
 else
     echo FAIL: $tmp1 is different from $tmp3 when uudecoded from here doc
     exit 1
 fi

Comment 1 Bernhard Rosenkraenzer 2000-06-26 18:04:29 UTC
This is fixed in bash 2.x.