Bug 219892

Summary: make 3.81 problem with implicit rules
Product: [Fedora] Fedora Reporter: Philippe Rigault <prigault>
Component: makeAssignee: Petr Machata <pmachata>
Status: CLOSED WONTFIX QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 6CC: mnewsome
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-02-16 13:28:18 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
the makefile in question
none
reproducer none

Description Philippe Rigault 2006-12-16 00:10:48 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.0.8) Gecko/20061107 Fedora/1.5.0.8-1.fc6 Firefox/1.5.0.8

Description of problem:
I encounter a strange behaviour of make (fc6 make-3.81) that I did not see on previous (fc5 make-3.80) versions.

This happens while building an RPM for the NCBI C toolkit (which has quite a convoluted build system which had to be portable during the old proprietary Unix era).

What happens is the following: during the build process 
- The first call of 'make' fails to build a target (ncbimain.o) because it does not find an implicit rule
- Calling 'make' subsequently succeeds, this time an implicit rule is found.

On fc5, make succeeds the first time.

Version-Release number of selected component (if applicable):
make-3.81-1.1

How reproducible:
Always


Steps to Reproduce:
I could not produce a simple testcase, so here is how to reproduce the behaviour on the real thing. The make process is driven by a C-shell script, so it is a bit complicated:

1. Get http://genome.ulaval.ca/bugtraces/ncbi_c_rpmbuild.tar.gz
2. untar it from the root of the rpmbuild tree
3. make sure lesstif-devel is installed (buildrequires)
4. To see the verbose debug output of make, uncomment line 56 of the spec file (to apply patch2)
5. Start the build: rpmbuild -bb ncbi_c.spec




Actual Results:
Build fails with

<snip>
    Considering target file `libncbi.a'.
     File `libncbi.a' does not exist.
      Considering target file `ncbimain.o'.
       File `ncbimain.o' does not exist.
       Looking for an implicit rule for `ncbimain.o'.
       Trying pattern rule with stem `ncbimain'.
       Trying implicit prerequisite `ncbimain.c'.
       Trying pattern rule with stem `ncbimain'.
       Trying implicit prerequisite `ncbimain.cc'.
       Trying pattern rule with stem `ncbimain'.
       Trying implicit prerequisite `ncbimain.C'.
<snip>
       No implicit rule found for `ncbimain.o'.
       Finished prerequisites of target file `ncbimain.o'.
      Must remake target `ncbimain.o'.
make: *** No rule to make target `ncbimain.o', needed by `libncbi.a'.  Stop.

Now, in order to start make a second time
 a) go to the build directory (RPMBUILD_ROOT/BUILD/ncbi/build)
 b) start make again (with the right options, the same as the shell script would have used):

make -d CFLAGS1="-DNDEBUG -O2 -march=opteron -c " LDFLAGS1="-O2 -march=opteron" OTHERLIBS="-lm" SHELL="/bin/sh" LCL="lnx" RAN="ranlib" AR="ar" CC="gcc -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -fPIC" LIB30=libncbicn3d.a LIB28=libvibgif.a LIB4=libvibrant.a LIB20=libncbidesk.a LIB45=libddvlib.a VIBFLAG="-I/usr/include -L/usr/lib64 -DWIN_MOTIF" VIBLIBS="-L/usr/lib64 -lXmu -lXm -lXt -lSM -lICE -lXext -lXp -lX11 -ldl" all

<snip>
    Considering target file `libncbi.a'.
     File `libncbi.a' does not exist.
      Considering target file `ncbimain.o'.
       File `ncbimain.o' does not exist.
       Looking for an implicit rule for `ncbimain.o'.
       Trying pattern rule with stem `ncbimain'.
       Trying implicit prerequisite `ncbimain.c'.
       Found an implicit rule for `ncbimain.o'.
        Pruning file `ncbimain.c'.
       Finished prerequisites of target file `ncbimain.o'.
      Must remake target `ncbimain.o'.
gcc -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -fPIC -DNDEBUG -O2 -march=opteron -c  -I../include -I/usr/include -L/usr
/lib64 -DWIN_MOTIF ncbimain.c
Putting child 0x006f7bf0 (ncbimain.o) PID 15210 on the chain.
Live child 0x006f7bf0 (ncbimain.o) PID 15210
Reaping winning child 0x006f7bf0 PID 15210
Removing child 0x006f7bf0 PID 15210 from chain.
      Successfully remade target file `ncbimain.o'.
<snip>





Expected Results:
Build succeed the first time around.

Additional info:
PS: If you want to see it build successfully on fc5, you have to replace lesstif by openmotif.

PPS: There is also an explicit pattern rule in the makefile:

.c.o :
        $(CC) $(CFLAGS) $<

Comment 1 Philippe Rigault 2006-12-16 00:13:42 UTC
Created attachment 143836 [details]
the makefile in question

Comment 2 Philippe Rigault 2006-12-16 00:37:50 UTC
Note: the tar file is not compressed despite its .gz extension.

Comment 3 Petr Machata 2006-12-19 12:44:05 UTC
By any chance, do you know if the Makefile uses paralel builds in some way?  And
doesn't the problem go away when you simply reexec toplevel make?  There is
another bug for make, bug 212111, that has somewhat similar symptom.

Comment 4 Philippe Rigault 2006-12-19 21:59:54 UTC
>By any chance, do you know if the Makefile uses paralel builds in some way?
No, it does not.
If I explicitly add the "-j1" option, I get the same results.

>doesn't the problem go away when you simply reexec toplevel make?
I assume you mean reexec in a "clean" state, not just after make has failed 
(in which case the rule is found).
No, it does not go away if you reexec make just after 'make clean'.
In other words, this always happens:
- make clean
- make [make_options] -> FAIL (no rule found)
- make [make_options] -> SUCCESS (rule found)


Comment 5 Petr Machata 2007-02-15 19:37:52 UTC
Created attachment 148139 [details]
reproducer

The problem boils down to this reproducer.
What I think happens is that make doesn't take into account the files that are
created during the course of make execution itself.  During the second
execution, the files are already here, and hence make finds the implicit chain.


Furthermore this only holds for implicit rules, be they old-style (.b.o) or
new-style (%.o: %.b).  When the rule is made explicit, it works.

Session snapshot:

$ make clean
rm -f *.[bo]
$ make
touch a1.b
make: *** No rule to make target `a1.o', needed by `all'.  Stop.
$ make
touch a1.b
cp a1.b a1.o

Comment 6 Philippe Rigault 2007-02-15 20:10:23 UTC
Thanks for the nice testcase. Yes, this is exactly what I observed too 
(affects files created by make, and only implicit rules). Explicit rules are 
unpractical in my case (many targets), so my current workaround is to use 
several consecutive calls to make in my rpmbuild setup.

Shouldn't this bug be filed upstream ? ( i.e 
http://savannah.gnu.org/bugs/?group=make )



Comment 7 Petr Machata 2007-02-16 13:28:18 UTC
I grepped through upstream bug tracker, and found this:
  http://savannah.gnu.org/bugs/?func=detailitem&item_id=15583

"""
You are seeing this problem because you are hiding information from make. You
have a rule, "copies", that creates three different .c files. However, only one
.c file lists "copies" as a prerequisite. Even if all of them listed it, make
doesn't know that invoking the rule "copies" once creates all three source
files. Because make caches directory information internally, it doesn't realize
that files were created as a side-effect of invoking the "copies" target.
"""

This is pretty much what we see.  The workaround is to mark the dependencies
explicitly:

all: a1.o
.b.o:; cp $*.b $*.o
a1.b:; touch a1.b
clean:; rm -f *.[bo]
.PHONY: clean all
.SUFFIXES: .b

The reason lies in the fact that make doesn't update the contents of
directories.  There was a bug reported on this:
  https://savannah.gnu.org/bugs/?443

Also in that bug, the reason that is stated is that make doesn't know about the
files you are about to create.  I don't know what's required by posix, or
whether this issue is covered at all.  Chances are, what GNU make does is allowed.

I'm closing this bug as wontfix anyway... directory caching is reportedly too
great a performance boost to lose for several obscure makefiles.