Bug 443889 - getopts function does not work properly when used in a function that is invoked with back quotes
getopts function does not work properly when used in a function that is invok...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: ksh (Show other bugs)
5.1
x86_64 Linux
low Severity medium
: rc
: ---
Assigned To: Michal Hlavinka
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-04-23 17:15 EDT by Serge Bonin
Modified: 2009-09-02 05:11 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-02 05:11:32 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Serge Bonin 2008-04-23 17:15:59 EDT
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)

Description of problem:
The getopts function does not seem to work properly when used in a function that is invoked with back quotes. The OPTIND variable is not incremented on subsequent calls to the getopts function; only the first call works properly. Also, the getopts function stops recognizing arguments after subsequent calls to getopts using back quotes.


Version-Release number of selected component (if applicable):
ksh-20060214-1.4

How reproducible:
Always


Steps to Reproduce:

Let's say we define a function using getopts as follows:

#!/bin/ksh

test_function()
{
   typeset COMM_LCL_OPTION

   OPTIND=1
   while getopts :ci: COMM_LCL_OPTION; do
      case ${COMM_LCL_OPTION} in
         c)
            echo " *** Function received argument -c"
            ;;
         i)
            echo " *** Function received argument -i ${OPTARG}"
            ;;
      esac
   done
   echo " OPTIND: ${OPTIND}"
   echo " ARGS: ${*}"
   shift $((${OPTIND} - 1))

   echo " Any remaining argument in function call: ${*}"
}

If this function is called as follows:

echo "Executing function:"
echo " First call"
RET=`test_function -c -i arg1 -iarg2 call1`
echo "$RET"
echo " Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"

The first call will execute properly but all subsequent will not increment the OPTIND variable as it should:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: -c -i arg1 -iarg2 call2

If this function is called directly instead of using back quotes, the results are in line with what is expected:

echo "Executing function directly:"
echo " First call"
test_function -c -i arg1 -iarg2 call1
echo "$RET"
echo " Second call"
test_function -c -i arg1 -iarg2 call2
echo "$RET"

As seen in the output, the OPTIND variable is properly incremented:

Executing functioni directly:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2

However, if a call is made to the function with back quotes first and then directly to the function:

echo "Executing function:"
echo " First call"
RET=`test_function -c -i arg1 -iarg2 call1`
echo "$RET"
echo " Second call"
test_function -c -i arg1 -iarg2 call2
echo "$RET"

The getopts function does not work properly on subsequent calls, even if these calls are directly done to the function:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: -c -i arg1 -iarg2 call2

However, this does not seem to affect sub scripts that are called after a back quoted function:

echo "Executing sub program:"
echo " First call"
RET=`test_function -c -i arg1 -iarg2 call1`
echo "$RET"
echo " Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"
echo " Third call"
RET=`./check_sub_getopts.ksh -c -i arg1 -iarg2 call3`
echo "$RET"

The check_sub_getopts.ksh script contains the exact same code as the test_function function stated above. The result is as follows:

Executing sub program:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: -c -i arg1 -iarg2 call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3

This script was tested on OSF1, AIX, Solaris, and Unix services for Windows. On all of these OSs the getopts functions would increment the OPTIND variable correctly (its value would go up to 5 on each call to the test_function using back quotes or calling it directly).

After further testing, I had discovered that it is not only the OPTIND variable that is not properly incremented, but the getopts function does not even detect arguments anymore. This can be seen with the following calls:

echo "Executing function:"
echo " First call"
test_function -c -i arg1 -iarg2 call1
echo " Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"
echo " Third call"
test_function -c -i arg1 -iarg2 call3
echo " Forth call"
RET=`test_function -c -i arg1 -iarg2 call4`
echo "$RET"
echo " Fifth call"
RET=`./check_sub_getopts.ksh -c -i arg1 -iarg2 call5`
echo "$RET"

The output is:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: -c -i arg1 -iarg2 call3
   Forth call
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: -c -i arg1 -iarg2 call4
   Fifth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call5
      Any remaining argument in function call: call5


We can see that in the third and forth call, no matter if the function was called directly or with back quotes, the getopts function does not even detected the presence of the -c and -i arguments. Good news is that at least the external shell has the correct getopts info (fifth call).

If I run the exact same script in bash, I do not get the same result:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4
   Fifth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call5
      Any remaining argument in function call: call5

As we can see, in bash, the OPTIND and arguments seem to be processed correctly. 

Actual Results:
Results of the last test:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: -c -i arg1 -iarg2 call3
   Forth call
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: -c -i arg1 -iarg2 call4
   Fifth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call5
      Any remaining argument in function call: call5


Expected Results:
What is expected:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4
   Fifth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call5
      Any remaining argument in function call: call5


Additional info:
This bug was entered on http://bugs.centos.org/view.php?id=2803 and they refered it to this site.
Comment 1 Tomas Smetana 2008-04-24 03:22:28 EDT
There is one thing I don't understand in the function body:

   OPTIND=1

What's this line good for?  OPTIND is set automatically by the shell...

The behaviour is otherwise quite interesting...  I don't know if it's a bug or
if changing OPTIND value explicitly couldn't erase it's special meaning
(unsetting it does).
Comment 2 Serge Bonin 2008-04-24 10:08:56 EDT
Thanks to your help in my issue reported in bug #431448, the problem goes away 
when I use the 'function test_function' format instead of 'test_function()'. 

I believe this changes the scope of the variables used by getopts in the 
function. This used to work in ksh88 but it changed in ksh93.

I did have to remove the OPTIND=1 in the test_function for this to work under 
ksh. However, the reason I had it there in the first place, which used to work 
under ksh88, is that bash does not work properly if it isn't there. 

I can demonstrate this, if a call the test_function in this sequence:

echo "Executing function:"
echo "   First call"
test_function -c -i arg1 -iarg2 call1
echo "   Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"
echo "   Third call"
RET=`test_function -c -i arg1 -iarg2 call3`
echo "$RET"
echo "   Forth call"
test_function -c -i arg1 -iarg2 call4

This is the result of ksh using the 'function test_function' format without 
OPTIND:

The following should always give the same result:
Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4

We now see that the getopts function is reacting normally on multiple calls.

If I run the same thing with bash, I get:

The following should always give the same result:
Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4

As we see, OPTIND seems to be set to the value of the previous call to getopts. 
If I put back the OPTIND=1 statement before the getopts function and re-execute 
it in bash, I get the expected result:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4

If I execute the modified bash script in ksh, I don’t get the same result:

The following should always give the same result:

The following should always give the same result:
Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: -c -i arg1 -iarg2 call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: -c -i arg1 -iarg2 call4

For some reason, ksh increments the value of OPTIND in the first and second 
call but stops doing it in the third and forth call to the function.

If I put a typeset to the OPTIND variable in the test_function, none of the 
calls work in ksh but the all work well under bash.


I just moved all of my scripts from ksh88 to bash as bash seems to be more 
compatible with the ‘mistakes’ I did in ksh88 (getopts and typeset local 
variables in function() name format) in respect to what is now done in ksh93. 
Do you believe that bash will also move in the direction that ksh93 has gone to 
or will bash continue to behave in this manner?
Comment 3 Tomas Smetana 2008-09-05 05:49:47 EDT
Finally I'm sure that this is a bug.  I've tried to patch it but without success yet.
Comment 10 errata-xmlrpc 2009-09-02 05:11:32 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-1256.html

Note You need to log in before you can comment on or make changes to this bug.