Bug 1164304

Summary: Upstream unit tests loads the installed shared libraries instead the ones from the build
Product: Red Hat Enterprise Linux 7 Reporter: Patrik Kis <pkis>
Component: krb5Assignee: Robbie Harwood <rharwood>
Status: CLOSED ERRATA QA Contact: Patrik Kis <pkis>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: dpal, ghudson
Target Milestone: rcKeywords: Patch
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: https://github.com/krb5/krb5/pull/340
Whiteboard:
Fixed In Version: krb5-1.13.2-2.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1281348 (view as bug list) Environment:
Last Closed: 2015-11-19 05:12:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
proposed patch none

Description Patrik Kis 2014-11-14 15:35:25 UTC
Description of problem:
krb5-1.12.2/src/lib/kadm5/unit-test fails (segfaults) when krb5-pkinit is installed on the sysytem.
Monitoring via audit showed that unit tests from this directory loads pkinit.so from the installed package and not the one that was built.
On top of that, monitoring showed that libs from the installed krb5-libs (whatever version installed) are loaded too.
This questions the effectiveness of upstream testing.

Version-Release number of selected component (if applicable):
krb5-libs-1.12.2-8.el7

How reproducible:
always

Steps to Reproduce:

# rpm -qa krb5\*
krb5-devel-1.11.3-49.el7.ppc64
krb5-libs-1.11.3-49.el7.ppc64
krb5-pkinit-1.11.3-49.el7.ppc64
#
# rpm -ivh krb5-1.12.2-8.el7.src.rpm 
... snip ...
# rpmbuild -bc ~/rpmbuild/SPECS/krb5.spec 
... snip ...
#
# cd ~/rpmbuild/BUILD/krb5-1.12.2/src/
# make runenv.py
LD_LIBRARY_PATH=`echo -L./lib | sed -e "s/-L//g" -e "s/ /:/g"`; export LD_LIBRARY_PATH;  \
for i in LD_LIBRARY_PATH; do \
	eval echo 'env['\\\'$i\\\''] = '\\\'\$$i\\\'; \
done > pyrunenv.vals
echo "proxy_tls_impl = 'openssl'" >> pyrunenv.vals
echo 'env = {}' > runenv.py
cat pyrunenv.vals >> runenv.py
# cd lib/kadm5/unit-test/
# make check
.. snip ...
	KINIT=../../../clients/kinit/kinit \
	KDESTROY=../../../clients/kdestroy/kdestroy \
	KADMIN_LOCAL=../../../kadmin/cli/kadmin.local \
	PRIOCNTL_HACK=0 VALGRIND="" \
	
WARNING: Couldn't find the global config file.
WARNING: Couldn't find tool init file
Test Run By root on Fri Nov 14 10:30:35 2014
Native configuration is powerpc64-redhat-linux-gnu

		=== api tests ===

Schedule of variations:
    unix

Running target unix
Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target.
Using ./config/unix.exp as tool-and-target-specific interface file.
Running ./api.2/crte-policy.exp ...
FAIL: create-policy 1: eof
ERROR: create-policy 1: unexpected failure in init
ERROR: create-policy 2: unexpected failure in init
ERROR: create-policy 3: unexpected failure in init

Comment 1 Nalin Dahyabhai 2014-11-14 15:36:50 UTC
Created attachment 957636 [details]
proposed patch

Comment 2 Patrik Kis 2014-11-14 16:31:50 UTC
I can confirm that the patch solves the issue with pkinit.so form krb5-1.11 package. After applying it the test stop failing and the pkinit.so from the installed package is not loaded.

However, I see that libraries from the installed krb5-libs are still used. Although, this not causes any test failure, IMO the libraries from build should be used. Is my assumption correct? Should we maybe open a new case for this?


Here is what I see:

# for i in `rpm -ql krb5-libs | grep so`; do auditctl -w $i -p warx -k KRB5_LIBS; done
# pwd
/root/rpmbuild/BUILD/krb5-1.12.2/src/lib/kadm5/unit-test
# make check
... snip ...
# echo $?
0
#
# ausearch -i -k KRB5_LIBS
----
type=PATH msg=audit(11/14/2014 11:16:04.131:375149) : item=0 name=/lib64/libkadm5srv_mit.so.8 inode=76534412 dev=fd:00 mode=file,755 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:lib_t:s0 objtype=NORMAL 
type=CWD msg=audit(11/14/2014 11:16:04.131:375149) :  cwd=/root/rpmbuild/BUILD/krb5-1.12.2/src/lib/kadm5/unit-test 
type=SYSCALL msg=audit(11/14/2014 11:16:04.131:375149) : arch=ppc64 syscall=open success=yes exit=3 a0=0x3fffa7ae0bb2 a1=O_RDONLY|O_CLOEXEC a2=0x10021d44740 a3=0x8 items=1 ppid=53254 pid=53260 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts1 ses=92 comm=kdb5_util exe=/root/rpmbuild/BUILD/krb5-1.12.2/src/kadmin/dbutil/kdb5_util subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=KRB5_LIBS 
----
type=PATH msg=audit(11/14/2014 11:16:04.241:375150) : item=0 name=/usr/lib64/krb5/plugins/kdb/db2.so inode=100665753 dev=fd:00 mode=file,755 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:lib_t:s0 objtype=NORMAL 
type=CWD msg=audit(11/14/2014 11:16:04.241:375150) :  cwd=/root/rpmbuild/BUILD/krb5-1.12.2/src/lib/kadm5/unit-test 
type=SYSCALL msg=audit(11/14/2014 11:16:04.241:375150) : arch=ppc64 syscall=open success=yes exit=4 a0=0x10021d8d470 a1=O_RDONLY|O_CLOEXEC a2=0x3fffa83c5c50 a3=0x0 items=1 ppid=53254 pid=53260 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts1 ses=92 comm=kdb5_util exe=/root/rpmbuild/BUILD/krb5-1.12.2/src/kadmin/dbutil/kdb5_util subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=KRB5_LIBS 
----
type=PATH msg=audit(11/14/2014 11:16:04.241:375151) : item=0 name=/lib64/libkadm5srv_mit.so.8 inode=76534412 dev=fd:00 mode=file,755 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:lib_t:s0 objtype=NORMAL 
type=CWD msg=audit(11/14/2014 11:16:04.241:375151) :  cwd=/root/rpmbuild/BUILD/krb5-1.12.2/src/lib/kadm5/unit-test 
type=SYSCALL msg=audit(11/14/2014 11:16:04.241:375151) : arch=ppc64 syscall=open success=yes exit=4 a0=0x3fffa7990bb2 a1=O_RDONLY|O_CLOEXEC a2=0x10021d8d4a0 a3=0x8 items=1 ppid=53254 pid=53260 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts1 ses=92 comm=kdb5_util exe=/root/rpmbuild/BUILD/krb5-1.12.2/src/kadmin/dbutil/kdb5_util subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=KRB5_LIBS 
----

... and many more such a logs ...

Comment 3 Patrik Kis 2015-04-03 14:35:32 UTC
Just a short summary about the consequences of this issue.

This problem causing that upstream tests are loading libs from the installed packages instead of the built (tested) ones so the tests are not really testing the release that they suppose to. This is the case especially on automated test/build systems which have older version of krb5 installed than what is actually built/tested.
There is an easy workaround, though (build the new package, install it, and run again the tests), but the issue make troubles on automated systems.

Comment 6 Roland Mainz 2015-05-22 17:36:27 UTC
Fixed in krb5-1.13.2-2.el7 (and Fedora krb5-1.13.2-1.fc23 and krb5-1.13.2-1.fc22) ...

... marking bug as MODIFIED.

Comment 10 Robbie Harwood 2015-10-01 23:07:06 UTC
Opened pull request upstream.

Comment 11 Greg Hudson 2015-10-05 15:40:18 UTC
Regarding comment 2: the krb5 test suite uses the profile variable db_module_dirs to cause KDB modules to be loaded from the build tree instead of the install tree.  The semantics of that variable are kind of dumb: the specified values *and* the install tree location are concatenated together in order, all matching shared objects from those directories are loaded, and then only the first object (which should be the one from the build tree) is actually used.

So, we aren't actually using the db2.so from the install tree, but we are loading it, which looks alarming.  This and a few other problems should get cleaned up, but there's no issue with us testing the wrong code during "make check".

Comment 12 Patrik Kis 2015-10-06 08:08:14 UTC
(In reply to Greg Hudson from comment #11)
> Regarding comment 2: the krb5 test suite uses the profile variable
> db_module_dirs to cause KDB modules to be loaded from the build tree instead
> of the install tree.  The semantics of that variable are kind of dumb: the
> specified values *and* the install tree location are concatenated together
> in order, all matching shared objects from those directories are loaded, and
> then only the first object (which should be the one from the build tree) is
> actually used.
> 
> So, we aren't actually using the db2.so from the install tree, but we are
> loading it, which looks alarming.  This and a few other problems should get
> cleaned up, but there's no issue with us testing the wrong code during "make
> check".

Thanks Greg for the explanation.

Comment 14 errata-xmlrpc 2015-11-19 05:12:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-2154.html