Bug 245529

Summary: coolkey hangs in C_Initialize() if pthreads library is not linked
Product: Red Hat Enterprise Linux 5 Reporter: Geoffrey Elgey <geoff.elgey>
Component: coolkeyAssignee: Bob Relyea <rrelyea>
Status: CLOSED ERRATA QA Contact: desktop-bugs <desktop-bugs>
Severity: low Docs Contact:
Priority: low    
Version: 5.0CC: aakkiang, ckannan, jgalipea, shaines, shillman, syeghiay
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-01-27 15:11:24 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Test program for reprocing the bug none

Description Geoffrey Elgey 2007-06-25 04:59:43 UTC 
Description of problem:

Calls to C_Initialize() will hang after a syslog() call, if the application has
not been linked with pthreads library.

This has been observed using the libcoolkeypk11.so that is included with RHEL5,
but is in fact reproducible with all versions of coolkey and on RHEL4 as well.

Attached is a test program that may be used to reproduce the problem.

Steps to Reproduce:

1. Build the attached test program, but do not link it with pthreads:

  $ gcc -c -I<path to PKCS#11 header files> test_pkcs11_syslog.c
  $ gcc -o test-pkcs11-syslog test-pkcs11-syslog.o -ldl

2. Run the program without using the "--syslog" option

  $ ./test-pkcs11-syslog /usr/lib/pkcs11/libcoolkeypk11.so
  ## calling C_Initialize() ... ok
  ## calling C_Finalize() ... ok

3. Run the program using the "--syslog" option. This should cause the program to
hang:

  $ ./test-pkcs11-syslog --syslog /usr/lib/pkcs11/libcoolkeypk11.so
  ## calling syslog ... ok
  ## calling C_Initialize() ... 

4. Now rebuild the program, this time linking in the pthread library:

  $ gcc -o test-pkcs11-syslog test-pkcs11-syslog.o -ldl -lpthread

5. Rerun both tests. Neither should hang:

  $ ./test-pkcs11-syslog /usr/lib/pkcs11/libcoolkeypk11.so
  ## calling C_Initialize() ... ok
  ## calling C_Finalize() ... ok

  $ ./test-pkcs11-syslog /usr/lib/pkcs11/libcoolkeypk11.so
  ## calling syslog ... ok
  ## calling C_Initialize() ... ok
  ## calling C_Finalize() ... ok

It seems that a call to syslog() is tickling some thread-related code, which
causes a problem when C_Initialize() eventually calls down into pcsc-lite. I
have tried the latest version of pcsc-lite but the problem still occurs.

A workaround is that any application that uses coolkey should have pthreads
library linked, or coolkey itself should link in the pthreads library. I added a
"AC_CHECK_LIB(pthread, ...)" line into coolkey's configure.in script, but there
is an m4 macro in pcsc-lite that may be more useful.
Comment 1 Geoffrey Elgey 2007-06-25 04:59:43 UTC 
Created attachment 157729 [details]
Test program for reprocing the bug
Comment 2 RHEL Program Management 2008-06-09 22:01:38 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 5 RHEL Program Management 2009-03-26 16:48:50 UTC 
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 6 Chandrasekar Kannan 2009-08-05 19:32:25 UTC 
per bug council, proposing for rhel 5.5
Comment 12 Asha Akkiangady 2010-01-15 22:41:58 UTC 
Verified.

Performed following steps on a RHEL 5.5 (combination of server/client i386/x86_64), coolkey works fine with C_Initialize() when pthreads library is not linked.

1. Install coolkey build available with the fix for this issue (coolkey -1.1.0-14.el5.{Arch}).

2. Save the test program attached in this bug (test_pkcs11_syslog.c) into a directory.

3. Install gcc, nspr-devel and nss-devel rpms available for RHEL 5.5.

4. Build the attached test program, but do not link it with pthreads:

$ gcc -c -I<path to PKCS#11 header files> -I <path to prtypes header files> test_pkcs11_syslog.c

$ gcc -o test-pkcs11-syslog test-pkcs11-syslog.o -ldl

5. Run the program without using the "--syslog" option

$ ./test-pkcs11-syslog /usr/lib/pkcs11/libcoolkeypk11.so
## calling C_Initialize() ... ok
## calling C_Finalize() ... ok

6. Run the program using the "--syslog" option. Program executes successfully.

$ ./test-pkcs11-syslog --syslog /usr/lib/pkcs11/libcoolkeypk11.so
## calling syslog ... ok
## calling C_Initialize() ... ok
## calling C_Finalize() ... ok

7. Now rebuild the program, this time linking in the pthread library:

  $ gcc -o test-pkcs11-syslog test-pkcs11-syslog.o -ldl -lpthread

8. Rerun both tests. Both of them executes fine:

$ ./test-pkcs11-syslog /usr/lib/pkcs11/libcoolkeypk11.so
## calling C_Initialize() ... ok
## calling C_Finalize() ... ok

$ ./test-pkcs11-syslog --syslog /usr/lib/pkcs11/libcoolkeypk11.so
## calling syslog ... ok
## calling C_Initialize() ... ok
## calling C_Finalize() ... ok
Comment 14 errata-xmlrpc 2010-01-27 15:11:24 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0068.html