Bug 709561

Summary: Object missing from generated dependency file with -MTfoo.d
Product: [Fedora] Fedora Reporter: Kieron Briggs <kbriggs>
Component: ccacheAssignee: Ville Skyttä <ville.skytta>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 14CC: jakub, jan.kratochvil, joel, ville.skytta
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-01-19 08:25:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kieron Briggs 2011-06-01 02:35:01 UTC
Description of problem:

Without ccache, gcc with -o foo.o -MTfoo.d (no space between -MT and filename) generates dependencies for both foo.o and foo.d. With ccache 3.1.4, foo.o is missing from the dependencies, leading to incorrect builds.


Version-Release number of selected component (if applicable):

ccache-3.1.4 in Fedora 14 and 15

How reproducible:

Always.


Steps to Reproduce:
1. Write a trivial C/C++ source file, e.g. foo.cc:
    #include <cstdio>
    void foo() {printf("Hello, world\n");}

2. Compile with dependency generation, with ccache:
    g++ -Wall -Werror -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc

3. Observe the contents of foo.d:
    foo.d: foo.cc
   There is no dependency for foo.o on foo.cc; this will lead to incorrect builds.

4. Remove foo.[do] and recompile with ccache disabled:
    CCACHE_DISABLE= g++ -Wall -Werror -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc

5. Observe the contents of foo.d:
    foo.d foo.o: foo.cc
   The dependency is correctly speficied.
  

Actual results:

foo.d: foo.cc


Expected results:

foo.d foo.o: foo.cc


Additional info:

This appears to have been fixed upstream in the 3.1.5 release. See http://gitweb.samba.org/?p=ccache.git;a=commit;h=8c3039d676d71564e98d50f1ca8ba029d1efa46c "Be bug compatible with GCC for -MTarg and -MQarg"

Comment 1 Ville Skyttä 2011-06-01 05:40:11 UTC
I don't seem to be able to reproduce on F-14, ccache produces the results you mentioned, but without ccache the results are the same:

$ cat foo.cc 
#include <cstdio>
void foo() {printf("Hello, world\n");}
$ which g++
/usr/lib64/ccache/g++

$ g++ -Wall -Werror -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc
$ cat foo.d 
foo.d: foo.cc

$ rm foo.[do]

$ CCACHE_DISABLE= g++ -Wall -Werror -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc
$ cat foo.d 
foo.d: foo.cc

$ rm foo.[do]

$ /usr/bin/g++ -Wall -Werror -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc
$ cat foo.d
foo.d: foo.cc

Anyway, ccache 3.1.5 is already in Rawhide.

Comment 2 Kieron Briggs 2011-06-01 06:45:04 UTC
With 3.1.5 in Rawhide, is it likely that it will also be pushed to F-14 and F-15? (the rawhide package actually installs fine into F-15 anyway, but not into F-14 due to a glibc dependency)

It's odd that you see a different result without ccache. If anything, your GCC appears to be behaving slightly differently to both mine and the ccache maintainer's...

Note that for some reason GCC seems to only include the .o in the dependency if there is no space between -MT and the filename. So it is mostly an issue for Eclipse-CDT-generated builds that rely on that behaviour.

Without ccache even being installed:

$ which g++
/usr/bin/g++
$ rpm -q gcc-c++ ccache
gcc-c++-4.5.1-4.fc14.x86_64
package ccache is not installed
$ g++ --version
g++ (GCC) 4.5.1 20100924 (Red Hat 4.5.1-4)
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ rm -f foo.[od]
$ g++ -Wall -Werror -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc
$ cat foo.d
foo.d foo.o: foo.cc

Comment 3 Ville Skyttä 2011-06-01 21:28:06 UTC
(In reply to comment #2)
> With 3.1.5 in Rawhide, is it likely that it will also be pushed to F-14 and
> F-15?

Most definitely if it fixes a problem people frequently run into - I gather the CDT one you mentioned is one, but I'd like to understand the problem better first.

> It's odd that you see a different result without ccache. If anything, your GCC
> appears to be behaving slightly differently to both mine and the ccache
> maintainer's...

...which I suppose could be because I forgot that the box in front of me has sort of a hybrid F-14/F-15 setup at the moment, and gcc and friends are from F-15.

Anyway I tested on some other machines, and was able to reproduce your results with a real vanilla F-14.  But all my attempts to reproduce on F-15 and Rawhide failed, foo.d always ends up containing just "foo.d: foo.cc" with vanilla g++ (ccache not even installed, and tested also with inserting spaces like -MF foo.d and -MT foo.d, no difference).

I'm afraid my (non-)expertise is not up to the task that I could say if this is a bug in F-15+ g++ (4.6.0).  Thoughts?  Have you tested on F-15+?

(Cc'ing upstream for comments/thoughts/info.)

Comment 4 Kieron Briggs 2011-06-02 06:20:44 UTC
On vanilla F-15 (g++ 4.6.0) I do NOT get the .o included in the dependencies, with or without ccache 3.1.4, and with or without the space between -MT and filename.

So it looks like there is a change in behaviour in GCC 4.6.0. I guess whether it should be considered a bug/regression in GCC is debatable, since I don't think the previous behaviour was documented, but it's certainly unfortunate since it breaks existing makefiles that depend on that behaviour.

Installing ccache-3.1.5-1.fc16.x86_64.rpm from rawhide onto F-15 restores the previous behaviour of -MT without spaces, which is what I want, but is arguably now inconsistent with the underlying compiler... argh.

Comment 5 Ville Skyttä 2011-06-17 20:37:27 UTC
Cc'ing gcc maintainer for comments:

Is the behavior change described in previous comments between g++ 4.5.1 and 4.6.0 intentional?  I didn't find anything related in http://gcc.gnu.org/gcc-4.6/changes.html on a quick look.

If it's intentional, I'm afraid the related change introduced in ccache 3.1.5 should probably not be done when the invoked compiler is from the gcc >= 4.6.0 family (not sure if/how feasible this is to implement).

Comment 6 Ville Skyttä 2011-12-03 13:54:11 UTC
Jakub, Joel, any comments?

Comment 7 Ville Skyttä 2011-12-03 15:54:07 UTC
Here's the summary of my tests:

EL5:
- g++ 4.1.2 adds foo.o with both "-MT foo.d" and "-MTfoo.d"
- ccache 2.4 adds foo.o with "-MT foo.d" but not with "-MTfoo.d" 

EL6:
- g++ 4.4.4 adds foo.o with "-MT foo.d" but not with "-MTfoo.d"
- ccache 3.1.4 does not add foo.o with either

F-14:
- g++ 4.5.1 adds foo.o with "-MT foo.d" but not with "-MTfoo.d"
- ccache 3.1.4 does not add foo.o with either

F-15:
- g++ 4.6.1 does not add foo.o with either
- ccache 3.1.4 does not add foo.o with either

F-16:
- g++ 4.6.2 does not add foo.o with either
- ccache 3.1.6 adds foo.o with "-MT foo.d" but not with "-MTfoo.d"

Rawhide:
- g++ 4.6.2 does not add foo.o with either
- ccache 3.1.6 adds foo.o with "-MT foo.d" but not with "-MTfoo.d"

So with what we have in the repos at the moment, only on F-15 g++ and ccache behave similarly wrt this; on all other distros there are differences.

Comment 8 Jan Kratochvil 2011-12-03 16:09:32 UTC
It has changed in FSF GCC:
g++ (GCC) 4.5.4 20111128 (prerelease)
->
g++ (GCC) 4.6.3 20111128 (prerelease)
(I do not know now by which specific commit.)

FSF gcc45-root/bin/g++ -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc;cat foo.d
foo.d foo.o: foo.cc
FSF gcc46-root/bin/g++ -c -o foo.o -MMD -MP -MFfoo.d -MTfoo.d foo.cc;cat foo.d
foo.d: foo.cc
FSF gcc45-root/bin/g++ -c -o foo.o -MMD -MP -MF foo.d -MT foo.d foo.cc;cat foo.d
foo.d: foo.cc
FSF gcc46-root/bin/g++ -c -o foo.o -MMD -MP -MF foo.d -MT foo.d foo.cc;cat foo.d
foo.d: foo.cc

I find as a clear fix -MTfoo.d behaves now the same as -MT foo.d .

That foo.o is missing there also seems to be a fix from the FSF GCC point of view, it was fixed by:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12448

So both ccache and Eclipse-CDT should be fixed.

Comment 9 Ville Skyttä 2011-12-03 17:15:06 UTC
I suppose "fixing" ccache would be a matter of reverting this patch, or maybe just the first hunk of it?

http://gitweb.samba.org/?p=ccache.git;a=commit;h=8c3039d676d71564e98d50f1ca8ba029d1efa46c

I say "fixing" because doing that would make ccache inconsistent with older gcc versions (< 4.6.0?).  Still waiting for comments from Joel (ccache upstream); with the info in comment 8 I believe the gcc POV is now clear.

Comment 10 Joel Rosdahl 2011-12-17 21:21:55 UTC
Sorry about the extended ping time.

I'm glad the bug was fixed in GCC, but I don't see any way of handling the situation acceptably in ccache (short of introducing a "compiler feature/bug detection" mechanism, which is something I'd like to do sometime but it's unlikely to happen in the near future). Sigh. Yes, I guess that reverting parts of 8c3039d676d71564e98d50f1ca8ba029d1efa46c would be the least bad thing to do.

(A side-note: it seems that the GCC bug was fixed 2007-03-13, so how come the fix hasn't shown up until GCC 4.6?)

Comment 11 Fedora Update System 2011-12-17 22:53:15 UTC
ccache-3.1.6-2.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/ccache-3.1.6-2.el6

Comment 12 Fedora Update System 2012-01-02 19:53:48 UTC
ccache-3.1.6-2.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 13 Fedora Update System 2012-01-08 20:41:50 UTC
ccache-3.1.7-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/ccache-3.1.7-1.fc16

Comment 14 Fedora Update System 2012-01-19 01:30:59 UTC
ccache-3.1.7-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Ville Skyttä 2012-01-19 08:25:59 UTC
F-15+ and EL-6 now have it so that gcc and ccache produce the same results wrt this; I don't intend to address this for EL-5's ccache 2.x.