Bug 1377341 - out-of-tree builds generate XDR headers and source files in the original directory
Summary: out-of-tree builds generate XDR headers and source files in the original dire...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: build
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kaleb KEITHLEY
QA Contact:
URL:
Whiteboard:
Depends On: 1330604 1369124
Blocks: 1279164 1374278 1374280
TreeView+ depends on / blocked
 
Reported: 2016-09-19 13:11 UTC by Kaleb KEITHLEY
Modified: 2017-03-06 17:26 UTC (History)
4 users (show)

Fixed In Version: glusterfs-3.10.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1330604
Environment:
Last Closed: 2017-03-04 07:48:41 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Kaleb KEITHLEY 2016-09-19 13:11:48 UTC
.c and .h files are blindly regenerated. If you run `make` followed by `make install` the `make install` will redundantly recompile everything because the unnecessarily regenerated header files are newer than the prior builds .o files


+++ This bug was initially created as a clone of Bug #1330604 +++

Description of problem:
out-of-tree builds are expected to not modify or add files in the original source tree. Our rpc/xdr/src/Makefile.am and the xdrgen script generate files in the wrong directory.

How reproducible:
100%

Steps to Reproduce:
1. checkout a git repository, say $SOURCES
2. run ./autogen.sh in $SOURCES (just like a 'make dist' tarball)
3. cd into a different directory, say $BUILD
4. run "$SOURCES/configure && make" from $BUILD
5. find newly generated files under $SOURCES/rpc/xdr/src

Actual results:
new generated files under $SOURCES/rpc/xdr/src

Expected results:
newly generated files should be created under $BUILD/rpc/xdr/src

--- Additional comment from Vijay Bellur on 2016-04-26 17:46:02 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#1) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-04-26 18:28:45 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#2) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-04-26 18:40:25 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#3) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-04-26 18:54:39 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#4) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-04-27 09:38:18 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#5) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-05-02 14:12:26 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds creates XDR files in the wrong directory) posted (#6) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-05-02 14:49:01 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds creates XDR files in the wrong directory) posted (#7) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-05-03 09:08:58 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#8) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-05-03 14:00:33 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#9) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-05-03 14:13:32 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#10) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-06-13 11:17:53 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#11) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-06-13 13:03:06 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#12) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Vijay Bellur on 2016-08-17 14:37:34 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#13) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-08-22 08:53:20 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#14) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-09-15 12:04:55 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#15) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-09-15 21:21:19 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#16) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-09-16 03:07:32 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#17) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-09-16 08:37:30 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#18) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-09-17 10:08:04 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#19) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-09-17 22:27:16 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#20) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

--- Additional comment from Worker Ant on 2016-09-18 02:31:20 EDT ---

REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#21) for review on master by Niels de Vos (ndevos@redhat.com)

--- Additional comment from Worker Ant on 2016-09-18 12:34:41 EDT ---

COMMIT: http://review.gluster.org/14085 committed in master by Niels de Vos (ndevos@redhat.com) 
------
commit e38dff5b4e0f0a25db664810fc3617eac44673ce
Author: Kaleb S KEITHLEY <kkeithle@redhat.com>
Date:   Tue Apr 26 17:04:04 2016 -0400

    build: out-of-tree builds generates files in the wrong directory
    
    And minor cleanup of a few of the Makefile.am files while we're
    at it.
    
    Rewrite the make rules to do what xdrgen does. Now we can get rid
    of xdrgen.
    
    Note 1. netbsd6's sed doesn't do -i. Why are we still running
    smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
    as it is.
    
    Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
    libglusterfs? A cut-and-paste mistake? It has no references to
    symbols in libglusterfs.
    
    Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
    regex that matches the same lines as the _extended_ regex
    "/#(ifndef|define|endif)/". To match the extended regex sed needs to
    be run with -r on Linux; with -E on *BSD. However NetBSD's and
    FreeBSD's sed helpfully also provide -r for compatibility. Using a
    basic regex avoids having to use a kludge in order to run sed with
    the correct option on OS X.
    
    Note 4. Not copying the bit of xdrgen that inserts copyright/license
    boilerplate. AFAIK it's silly to pretend that machine generated
    files like these can be copyrighted or need license boilerplate.
    The XDR source files have their own copyright and license; and
    their copyrights are bound to be more up to date than old
    boilerplate inserted by a script. From what I've seen of other
    Open Source projects -- e.g. gcc and its C parser files generated
    by yacc and lex -- IIRC they don't bother to add copyright/license
    boilerplate to their generated files.
    
    It appears that it's a long-standing feature of make (SysV, BSD,
    gnu) for out-of-tree builds to helpfully pretend that the source
    files it can find in the VPATH "exist" as if they are in the $cwd.
    rpcgen doesn't work well in this situation and generates files
    with "bad" #include directives.
    
    E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
    you get an #include directive in the generated .c file like this:
    
      ...
      #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
      ...
    
    which (obviously) results in compile errors on out-of-tree build
    because the (generated) header file doesn't exist at that location.
    Compared to `rpcgen ./glusterfs3-xdr.x` where you get:
    
      ...
      #include "glusterfs3-xdr.h"
      ...
    
    Which is what we need. We have to resort to some Stupid Make Tricks
    like the addition of various .PHONY targets to work around the VPATH
    "help".
    
    Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
    looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
    And don't delete the -I$(top_builddir)/rpc/xdr/... bits
    
    Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
    BUG: 1330604
    Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
    Reviewed-on: http://review.gluster.org/14085
    Tested-by: Niels de Vos <ndevos@redhat.com>
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Niels de Vos <ndevos@redhat.com>

Comment 1 Worker Ant 2016-09-19 14:13:58 UTC
REVIEW: http://review.gluster.org/15530 (build: rpc/xdr .c and .h files are regenerated unnecessarily) posted (#2) for review on master by Kaleb KEITHLEY (kkeithle@redhat.com)

Comment 2 Worker Ant 2016-09-20 11:45:49 UTC
COMMIT: http://review.gluster.org/15530 committed in master by Kaleb KEITHLEY (kkeithle@redhat.com) 
------
commit c5426a13ad28fb2c6f0ed62404dbe60ea0dfaad2
Author: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Date:   Mon Sep 19 09:13:09 2016 -0400

    build: rpc/xdr .c and .h files are regenerated unnecessarily
    
    .c and .h files are blindly (re)generated. If you run `make`
    followed by a second `make` or `make install` the subsequent
    `make` will unnecessarily recompile everything because of the
    redundant regeneration of the .c and .h files are now newer
    than the prior builds .o files
    
    Change-Id: I7e477bcdcc20869e144ada7e6d91e7221b8ee71f
    BUG: 1377341
    Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
    Reviewed-on: http://review.gluster.org/15530
    Smoke: Gluster Build System <jenkins@build.gluster.org>
    Reviewed-by: Niels de Vos <ndevos@redhat.com>
    Tested-by: Niels de Vos <ndevos@redhat.com>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Anoop C S <anoopcs@redhat.com>

Comment 3 Shyamsundar 2017-03-06 17:26:11 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report.

glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.