Bug 177842

Summary: for..done loop is slow
Product: [Fedora] Fedora Reporter: Need Real Name <lsof>
Component: bashAssignee: Tim Waugh <twaugh>
Status: CLOSED NOTABUG QA Contact: Ben Levenson <benl>
Severity: medium Docs Contact:
Priority: medium    
Version: 4   
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2006-01-16 14:04:58 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Need Real Name 2006-01-15 13:07:42 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050923 Epiphany/1.6.5

Description of problem:
Hello,
While setting up a text box to see how fast mbox might be, I ran the following command:

$ for i in $(seq 123456); do
 touch $i
done

Using a 5 digit sequence number is fast.
From the top usage, I can see that memory allocation or bash is growing extrememly slowly, so it is doing something.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
x

Expected Results:  123456 files isn't really very many these days. bash should be able to handle it without resorting to a higher level language.

Additional info:

Comment 1 Tim Waugh 2006-01-16 13:48:31 UTC
Correcting summary: this isn't to do with bash arrays.

Comment 2 Tim Waugh 2006-01-16 14:04:58 UTC
Almost all of the time is spent dealing with 'touch $i', and there isn't a lot
we can do to make that faster:

# strace -c bash
bash-3.1# for i in $(seq 123456); do
>  touch $i
> done
exit
bash-3.1# exit
exit
Process 17604 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 49.92   15.467247          63    246914    123457 waitpid
 44.93   13.921620         113    123457           clone
  3.63    1.124318           1   1234652           rt_sigprocmask
  0.78    0.242362           2    123458           pipe
  0.34    0.105153           0    370405           ioctl
  0.30    0.091958           0    246938           close
  0.05    0.016355           0    123457           sigreturn
  0.04    0.012871           0    123459           setpgid
  0.00    0.000189           0      4481           mremap
  0.00    0.000147           0      5976           read
  0.00    0.000036           0       266           time
  0.00    0.000000           0        53           write
  0.00    0.000000           0        20         1 open
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           getpid
  0.00    0.000000           0         9         1 access
  0.00    0.000000           0         1           dup
  0.00    0.000000           0        49           brk
  0.00    0.000000           0         1           dup2
  0.00    0.000000           0         1           getppid
  0.00    0.000000           0         1           getpgrp
  0.00    0.000000           0        11           munmap
  0.00    0.000000           0         1           uname
  0.00    0.000000           0         4           mprotect
  0.00    0.000000           0         1           _llseek
  0.00    0.000000           0       130           rt_sigaction
  0.00    0.000000           0         1           getrlimit
  0.00    0.000000           0        24           mmap2
  0.00    0.000000           0        23        10 stat64
  0.00    0.000000           0        17           fstat64
  0.00    0.000000           0         1           getuid32
  0.00    0.000000           0         1           getgid32
  0.00    0.000000           0         1           geteuid32
  0.00    0.000000           0         1           getegid32
  0.00    0.000000           0         9         1 fcntl64
  0.00    0.000000           0         1           set_thread_area
  0.00    0.000000           0         2           socket
  0.00    0.000000           0         2         2 connect
------ ----------- ----------- --------- --------- ----------------
100.00   30.982256               2603830    123472 total

Seems like you want to re-write your script as: 'seq 123456 | xargs touch'.

Comment 3 Need Real Name 2006-01-16 14:12:50 UTC
Thanks for this. I thought it was seq playing up since no files were being
created by touch. an strace to the exiting bash process didn't work, but I'll
try an strace to a new bash next time.