Bug 474115

Summary: command hang in ksh and cpu workload is high
Product: [Fedora] Fedora EPEL Reporter: Michal Hlavinka <mhlavink>
Component: mkshAssignee: Robert Scheck <redhat-bugzilla>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: el5CC: redhat-bugzilla, tg
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: 36-2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-12-02 22:08:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Michal Hlavinka 2008-12-02 10:25:34 UTC
+++ This bug was initially created as a clone of Bug #474114 +++

Description of problem:
command hang in ksh and cpu workload is high

Version-Release number of selected component (if applicable):
mksh-36-1.fc10.x86_64

How reproducible:


Steps to Reproduce:
1. create a script like belowing:
# cat hang.sh 
#!/bin/ksh
alias ls="ls"
i=`ls`
echo $i

2. issuce the script
  # ./hang.sh

Actual Results:
  script hangs while executing  i=`ls`, and the cpu workload is nearly 100%

Expected Results:
   The script runs smoothly.


Additional Information:

# ps -e | grep hang
21776 pts/3    00:00:27 hang.sh

# strace -p 21776 
brk(0xa057000)                          = 0xa057000
brk(0xa078000)                          = 0xa078000
brk(0xa099000)                          = 0xa099000
brk(0xa0ba000)                          = 0xa0ba000
brk(0xa0db000)                          = 0xa0db000
brk(0xa0fc000)                          = 0xa0fc000
brk(0xa11d000)                          = 0xa11d000
...

# ltrace -p 21776 
__rawmemchr(0x9480688, 0, 0xbff7e808, 0x8049a6a, 0x9fb71a8)          =
0x948068a 
memset(0x80758a2, '\000', 63)                                        =
0x80758a2 
strcmp("ls", "ls")                                                   = 0
__ctype_b_loc()                                                      =
0xb7fdda84
__rawmemchr(0x9480688, 0, 0xbff7e808, 0x8049a6a, 0x9fb7210)          =
0x948068a 
memset(0x80758a2, '\000', 63)                                        =
0x80758a2 
strcmp("ls", "ls")                                                   = 0
__ctype_b_loc()                                                      =
0xb7fdda84
__rawmemchr(0x9480688, 0, 0xbff7e808, 0x8049a6a, 0x9fb7278)          =
0x948068a 
memset(0x80758a2, '\000', 63)                                        =
0x80758a2 
strcmp("ls", "ls")                                                   = 0
__ctype_b_loc()                                                      =
0xb7fdda84
__rawmemchr(0x9480688, 0, 0xbff7e808, 0x8049a6a, 0x9fb72e0)          =
0x948068a 
memset(0x80758a2, '\000

WARNING: when you try to reproduce this, mksh starts to consume more and more memory very quickly, so ulimit -m and ulimit -v to something lower than unlimited is very useful.

Comment 1 Michal Hlavinka 2008-12-02 10:28:23 UTC
my mistake, strace and ltrace are originally from pdksh, but this bug is easily reproducible with mksh

Comment 2 Robert Scheck 2008-12-02 10:29:43 UTC
Ehm, is it possible, that you're mixing up ksh and mksh? You wrote ksh above,
not mksh...

Comment 3 Thorsten Glaser 2008-12-02 11:16:48 UTC
Here's an excerpt from the mksh(1) manual page:

  Aliases
     There are two types of aliases: normal command aliases and tracked
     aliases. Command aliases are normally used as a short hand for a long or
     often used command. The shell expands command aliases (i.e. substitutes
     the alias name for its value) when it reads the first word of a command.
     An expanded alias is re-processed to check for more aliases. If a command
     alias ends in a space or tab, the following word is also checked for
     alias expansion. The alias expansion process stops when a word that is
     not an alias is found, when a quoted word is found, or when an alias word
     that is currently being expanded is found.

The crucial part here is: "An expanded alias is re-processed..."

So, first and foremost, this is not a bug in the shell, but a bug
in the script.

Interestingly enough, when aliasing "ls" to "ls", simply typing "ls"
on the shell command line still works, whereas typing "i=$(ls)" hangs.

I also suspect you could get me with the last sentence quoted above,
which _seems_ to try to prevent recursion loops. But I just tested
with pdksh, the ancestor, which exhibits the same behaviour, so the
solution to this problem is probably along the lines of "don't do
that then".

I will keep this in mind but do not promise a change in behaviour.
If you have a patch for mksh ready, I will however look at it.
It does not seem to be dependent on flags like "set -o trackall",
which is the only thing besides interactivity that came to my mind.

Comment 4 Robert Scheck 2008-12-02 11:21:04 UTC
*** Bug 474114 has been marked as a duplicate of this bug. ***

Comment 5 Michal Hlavinka 2008-12-02 11:26:32 UTC
(In reply to comment #2)
> Ehm, is it possible, that you're mixing up ksh and mksh? You wrote ksh above,
> not mksh...

no, it's really mksh. I have ksh->mksh symlink


(In reply to comment #3)
1) ...when a quoted word is found..., there is alias ls="ls", so it's quoted
2) it works for ls but not for `ls`, it's a bug IMO
3) even if it is shell script's bug, mksh should write error as if you try to use not_existing_command or read from not_existing_file. Eat all memory is definitely a bug.

Comment 6 Thorsten Glaser 2008-12-02 11:37:00 UTC
 1) ...when a quoted word is found..., there is alias ls="ls", so it's quoted

It's not quoted. The double quotes are stripped by the command line
parser before the "alias" builtin sees the result.

 2) it works for ls but not for .s, it's a bug IMO

If it happens to work for "ls" despite not being specified to work
at all (the re-expansion clause), it's just lucky. If it then does
not happen to work for "$(ls)", this is just being once but not
twice lucky.

 3) even if it is shell script's bug, mksh should write error as if you try to
 use not_existing_command or read from not_existing_file. Eat all memory is
 definitely a bug.

I'd rather call it "undesired behaviour", since it's a bug in your
script which merely exploits it.

But, as I said, let me see if I can tend to it, okay?
You're of course allowed to send in a patch *smiles*

Comment 7 Michal Hlavinka 2008-12-02 11:43:20 UTC
(In reply to comment #3)
> .. But I just tested
> with pdksh, the ancestor, which exhibits the same behaviour...

Yes

https://bugzilla.redhat.com/show_bug.cgi?id=473528

> I will keep this in mind but do not promise a change in behaviour.
> If you have a patch for mksh ready, I will however look at it.

I'm going to make patch for pdksh (once it's approved - I'm assigned to the pdksh's bug). I've just thought to let you know, there is the same bug with mksh that should be fixed.

Comment 8 Thorsten Glaser 2008-12-02 12:09:22 UTC
Okay, I think I got the cause of the bug.

Try to replace
| i=$(ls)
with
| i=$(ls )
and it vanishes. The source is closed to SEOF early
if there is no trailing space... will fix that soonish.

Comment 9 Michal Hlavinka 2008-12-02 12:25:03 UTC
yes, I can confirm that

Comment 10 Thorsten Glaser 2008-12-02 12:40:43 UTC
Fixed upstream in commitid 10049352BD82DFF462F
Thanks for reporting the problem. Feel free to take the patch
and apply it to the Redhat-local pdksh version (although mksh
is positioned in the market as pdksh successor, seeing that it
has not been under active development since 1999 or so).

Comment 11 Michal Hlavinka 2008-12-02 12:49:55 UTC
(In reply to comment #10)
> ... although mksh
> is positioned in the market as pdksh successor, seeing that it
> has not been under active development since 1999 or so

Yes, I know, but it's not my decision

Thanks for fast fix,
Michal

Comment 12 Michal Hlavinka 2008-12-02 18:21:10 UTC
(In reply to comment #10)
> Fixed upstream in commitid 10049352BD82DFF462F

where can I find that repository? Is it http://www.mirbsd.org/cvs.cgi/src/bin/mksh/ ? Because I'm unable to connect to that server (time-out)

Comment 13 Thorsten Glaser 2008-12-02 18:55:33 UTC
Yes, it is, but the box is currently fscking, sorry.
Once it's up and the next rsync from the master repo
is done, you'll be able to access it from there.

Comment 14 Fedora Update System 2008-12-02 22:05:03 UTC
mksh-36-2.fc9 has been submitted as an update for Fedora 9.
http://admin.fedoraproject.org/updates/mksh-36-2.fc9

Comment 15 Fedora Update System 2008-12-02 22:05:14 UTC
mksh-36-2.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/mksh-36-2.fc10

Comment 16 Fedora Update System 2008-12-02 22:05:54 UTC
mksh-36-2.fc8 has been submitted as an update for Fedora 8.
http://admin.fedoraproject.org/updates/mksh-36-2.fc8

Comment 17 Robert Scheck 2008-12-02 22:08:49 UTC
Package: mksh-36-2.fc11 Tag: dist-f11 Status: complete Built by: robert
Package: mksh-36-2.fc10 Tag: dist-f10-updates-candidate Status: complete Built by: robert
Package: mksh-36-2.fc9 Tag: dist-f9-updates-candidate Status: complete Built by: robert
Package: mksh-36-2.fc8 Tag: dist-f8-updates-candidate Status: complete Built by: robert
877 (mksh): Build on target fedora-5-epel succeeded.
878 (mksh): Build on target fedora-4-epel succeeded.

Comment 18 Fedora Update System 2008-12-07 04:17:55 UTC
mksh-36-2.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2008-12-07 04:18:19 UTC
mksh-36-2.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 20 Fedora Update System 2008-12-07 04:29:32 UTC
mksh-36-2.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2008-12-14 23:52:36 UTC
mksh-36b-1.fc8 has been submitted as an update for Fedora 8.
http://admin.fedoraproject.org/updates/mksh-36b-1.fc8

Comment 22 Fedora Update System 2008-12-14 23:52:45 UTC
mksh-36b-1.fc9 has been submitted as an update for Fedora 9.
http://admin.fedoraproject.org/updates/mksh-36b-1.fc9

Comment 23 Fedora Update System 2008-12-14 23:52:55 UTC
mksh-36b-1.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/mksh-36b-1.fc10

Comment 24 Fedora Update System 2008-12-18 00:31:19 UTC
mksh-36b-1.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 Fedora Update System 2008-12-18 00:33:43 UTC
mksh-36b-1.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 26 Fedora Update System 2008-12-18 00:40:34 UTC
mksh-36b-1.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.