Bug 1462425 - Some patterns are ignored with trace-children-skip in valgrind 3.13
Some patterns are ignored with trace-children-skip in valgrind 3.13
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: valgrind (Show other bugs)
26
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Mark Wielaard
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-17 10:06 EDT by Lukas Slebodnik
Modified: 2017-06-19 08:38 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-06-19 04:23:03 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
tarball with minimal reproducer (3.70 KB, application/x-gzip)
2017-06-17 10:06 EDT, Lukas Slebodnik
no flags Details

  None (edit)
Description Lukas Slebodnik 2017-06-17 10:06:28 EDT
Created attachment 1288560 [details]
tarball with minimal reproducer

Description of problem:
We run unit tests with valgrind to catch typical errors in C language :-)
We use trace-children-skip to ignore errors in system binaries because different distributions have different bugs. We have automake project and libtool sometimes generate bash wrapper for binary in .libs so it can re relinked/recompiled. And the latest upgrade of valgrind to 1.13 causes problems with such wrappers. 

Version-Release number of selected component (if applicable):
dh$ rpm -q valgrind
valgrind-3.13.0-1.fc26.x86_64

How reproducible:
Deterministic

Steps to Reproduce:
1. //unpack attached tarball
   tar -xvzf valgrind_reproducer.tar.gz
2. //install gcc and valgrind
   dnf install -y --setopt=debuglevel=0 --setopt=errorlevel=0 gcc valgrind
3. //run reproducer
   cd valgrind_reproducer/
   bash run_me.sh

Actual results:
FAILED

Expected results:
PASSED

Additional info:
run_me.sh is shell script for simplification of following steps extracted from automake project. Shell wrapper can be probably simplified as well. 

set -o nounset -o pipefail -o errexit

rm -rf .libs test_child_common dummy_child.o *.log
mkdir -p .libs
gcc -Wall -Wextra -g -c dummy_child.c
gcc -Wall -Wextra -g dummy_child.o -o .libs/lt-dummy-child
gcc -Wall -Wextra -g test_child_common.c -o test_child_common
valgrind --leak-check=full \
         --log-file=test_child_common.%p.valgrind.log \
         --leak-check=full \
         --trace-children=yes \
         --trace-children-skip='*/bin/*,*/sbin/*' \
         --gen-suppressions=all \
         --verbose \
         ./test_child_common

ERRORS=`grep ERROR *.log | grep -v "0 errors" | wc -l || true`

if [ "$ERRORS" -ne 0 ]; then
    echo FAILED
    exit 1
else
    echo PASSED
fi
Comment 1 Lukas Slebodnik 2017-06-17 10:09:00 EDT
Bug is also in RC versions of valgrind

sh]# dnf downgrade -y --setopt=debuglevel=0 --setopt=errorlevel=0 https://kojipkgs.fedoraproject.org//packages/valgrind/3.13.0/0.2.RC1.fc27/x86_64/valgrind-3.13.0-0.2.RC1.fc27.x86_64.rpm
sh# rpm -q valgrind
valgrind-3.13.0-0.2.RC1.fc27.x86_64
sh# bash run_me.sh 
FAILED

sh# dnf downgrade -y --setopt=debuglevel=0 --setopt=errorlevel=0 https://kojipkgs.fedoraproject.org//packages/valgrind/3.13.0/0.1.RC1.fc27/x86_64/valgrind-3.13.0-0.1.RC1.fc27.x86_64.rpm
sh# rpm -q valgrind
valgrind-3.13.0-0.1.RC1.fc27.x86_64
sh# bash run_me.sh 
FAILED


But if works with 3.12


sh# dnf downgrade -y --setopt=debuglevel=0 --setopt=errorlevel=0 https://kojipkgs.fedoraproject.org//packages/valgrind/3.12.0/8.fc27/x86_64/valgrind-3.12.0-8.fc27.x86_64.rpm
sh# rpm -q valgrind
valgrind-3.12.0-8.fc27.x86_64
sh# bash run_me.sh 
PASSED
Comment 2 Lukas Slebodnik 2017-06-17 10:14:38 EDT
BTW reproducer will pass if ",./dummy-child" is appended to trace-children-skip

-         --trace-children-skip='*/bin/*,*/sbin/*' \
+         --trace-children-skip='*/bin/*,*/sbin/*,./dummy-child' \

but adding more problematic shell wrappers to trace-children-skip is just a workaround for regression in valgrind. And it will not be ideal to add more wrappers to the list.
Comment 3 Mark Wielaard 2017-06-18 04:14:01 EDT
What seems to happen is that interpreter names (#!/bin/sh in this case for ./dummy-child) was never ignored by --trace-children-skip. But a bug in --log-file has been fixed that now correctly creates a log files. So previously with older valgrind you were missing some log files. But with newer valgrind you get proper log files for all processes ran. I suspect this might be https://bugs.kde.org/show_bug.cgi?id=162848

You can check this by removing --log-file=test_child_common.%p.valgrind.log and instead use 2>&1 | less to show all output. You will see there are ERRORs for some of the dummy-child/sh leaks.

Or you can just see that the number of .log files after running with a newer valgrind is larger than with an older valgrind.
Comment 4 Mark Wielaard 2017-06-18 10:40:58 EDT
BTW. When using libtool the best way to run a program under valgrind is:
$ libtool --mode=execute valgrind ./dummy-child

That will setup the environment (LD_LIBRARY_PATH, etc.) and the executable first and then run valgrind. That way there is no need for any shell execution under valgrind.
Comment 5 Lukas Slebodnik 2017-06-19 01:58:49 EDT
(In reply to Mark Wielaard from comment #4)
> BTW. When using libtool the best way to run a program under valgrind is:
> $ libtool --mode=execute valgrind ./dummy-child
> 
> That will setup the environment (LD_LIBRARY_PATH, etc.) and the executable
> first and then run valgrind. That way there is no need for any shell
> execution under valgrind.

You did not get a point.

We already use libtool in such way. But "./dummy-child" is not executed directly because it is used within other test "./test_child_common."
Comment 6 Mark Wielaard 2017-06-19 02:46:31 EDT
(In reply to Lukas Slebodnik from comment #5)
> (In reply to Mark Wielaard from comment #4)
> > BTW. When using libtool the best way to run a program under valgrind is:
> > $ libtool --mode=execute valgrind ./dummy-child
> > 
> > That will setup the environment (LD_LIBRARY_PATH, etc.) and the executable
> > first and then run valgrind. That way there is no need for any shell
> > execution under valgrind.
> 
> You did not get a point.
> 
> We already use libtool in such way. But "./dummy-child" is not executed
> directly because it is used within other test "./test_child_common."

OK. In that case, if you don't want valgrind to run dummy-child (and report issues in it) you will have to skip it.

Do you agree with Comment #3 that this is just a fix in 3.13.0 so that you get more accurate log files? Or do you still believe there is a regression in 3.13.0 compared to earlier versions?
Comment 7 Lukas Slebodnik 2017-06-19 03:41:07 EDT
(In reply to Mark Wielaard from comment #6)
> (In reply to Lukas Slebodnik from comment #5)
> > (In reply to Mark Wielaard from comment #4)
> > > BTW. When using libtool the best way to run a program under valgrind is:
> > > $ libtool --mode=execute valgrind ./dummy-child
> > > 
> > > That will setup the environment (LD_LIBRARY_PATH, etc.) and the executable
> > > first and then run valgrind. That way there is no need for any shell
> > > execution under valgrind.
> > 
> > You did not get a point.
> > 
> > We already use libtool in such way. But "./dummy-child" is not executed
> > directly because it is used within other test "./test_child_common."
> 
> OK. In that case, if you don't want valgrind to run dummy-child (and report
> issues in it) you will have to skip it.
> 
> Do you agree with Comment #3 that this is just a fix in 3.13.0 so that you
> get more accurate log files? Or do you still believe there is a regression
> in 3.13.0 compared to earlier versions?

I tried original reproducer and not the reduced/simplified one and I can confirt that I got less log files with "--log-file=test_child_common.%p.valgrind.log " and I can see errors without "--log-file".

Thank you very much for provided details.
Feel free to close as not a bug.
Comment 8 Lukas Slebodnik 2017-06-19 03:48:05 EDT
BTW Do you have an idea how to track errors in ".libs/lt-dummy-child" but not in libtool/shell wrapper "./dummy-child"?

Because --trace-children-skip='*/bin/*,*/sbin/*,./dummy-child' will skip everything.
Comment 9 Mark Wielaard 2017-06-19 04:21:15 EDT
(In reply to Lukas Slebodnik from comment #8)
> BTW Do you have an idea how to track errors in ".libs/lt-dummy-child" but
> not in libtool/shell wrapper "./dummy-child"?
> 
> Because --trace-children-skip='*/bin/*,*/sbin/*,./dummy-child' will skip
> everything.

If you cannot arrange for the shell wrapper to be called through libtool --mode=execute valgrind then maybe you could run valgrind on the installed binaries? Or since you already split the log files you might want to ignore those log files that have Command: wrapper and only process those that have Command: .libs/lt-?
Comment 10 Mark Wielaard 2017-06-19 04:23:03 EDT
As discussed above this isn't really a bug, but caused by the fact that a bug was fixed so that there are now more/accurate .log files with --trace-children=yes and --log-file=...%p...
Comment 11 Lukas Slebodnik 2017-06-19 08:38:30 EDT
(In reply to Mark Wielaard from comment #9)
> (In reply to Lukas Slebodnik from comment #8)
> > BTW Do you have an idea how to track errors in ".libs/lt-dummy-child" but
> > not in libtool/shell wrapper "./dummy-child"?
> > 
> > Because --trace-children-skip='*/bin/*,*/sbin/*,./dummy-child' will skip
> > everything.
> 
> If you cannot arrange for the shell wrapper to be called through libtool
> --mode=execute valgrind

I do not want to run "exec" every time with valgrind and It would be complicated to detect within unit test whether it was executed with valgrind and then execute   
"./dummy-child" with valgrind as well. Moreover it would be complicated to pass all valgrind arguments from "unit test" to child process.

> then maybe you could run valgrind on the installed
> binaries? Or since you already split the log files you might want to ignore
> those log files that have Command: wrapper and only process those that have
> Command: .libs/lt-?

I think much simpler would be add all leaks/errors from shell wrapper to the suppression file.
In this particular case, I do not care much about errors in "./dummy-child". So I will add it to  "trace-children-skip". I was jsut curious about other ways how to solve it with valgrind.

Anyway, thank you very much for your time.
I really appreciate fast response.

Note You need to log in before you can comment on or make changes to this bug.