Bug 1330604

Summary: out-of-tree builds generate XDR headers and source files in the original directory
Product: [Community] GlusterFS Reporter: Niels de Vos <ndevos>
Component: buildAssignee: Kaleb KEITHLEY <kkeithle>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: mainlineCC: anoopcs, bugs, kkeithle
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.10.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1369124 1377341 (view as bug list) Environment:
Last Closed: 2017-03-04 07:49:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1369124    
Bug Blocks: 1279164, 1374278, 1374280, 1377341    

Description Niels de Vos 2016-04-26 14:42:59 UTC
Description of problem:
out-of-tree builds are expected to not modify or add files in the original source tree. Our rpc/xdr/src/Makefile.am and the xdrgen script generate files in the wrong directory.

How reproducible:
100%

Steps to Reproduce:
1. checkout a git repository, say $SOURCES
2. run ./autogen.sh in $SOURCES (just like a 'make dist' tarball)
3. cd into a different directory, say $BUILD
4. run "$SOURCES/configure && make" from $BUILD
5. find newly generated files under $SOURCES/rpc/xdr/src

Actual results:
new generated files under $SOURCES/rpc/xdr/src

Expected results:
newly generated files should be created under $BUILD/rpc/xdr/src

Comment 1 Vijay Bellur 2016-04-26 21:46:02 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#1) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 2 Vijay Bellur 2016-04-26 22:28:45 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#2) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 3 Vijay Bellur 2016-04-26 22:40:25 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#3) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 4 Vijay Bellur 2016-04-26 22:54:39 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#4) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 5 Vijay Bellur 2016-04-27 13:38:18 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generate XDR files in the original directory) posted (#5) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 6 Vijay Bellur 2016-05-02 18:12:26 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds creates XDR files in the wrong directory) posted (#6) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 7 Vijay Bellur 2016-05-02 18:49:01 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds creates XDR files in the wrong directory) posted (#7) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 8 Vijay Bellur 2016-05-03 13:08:58 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#8) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 9 Vijay Bellur 2016-05-03 18:00:33 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#9) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 10 Vijay Bellur 2016-05-03 18:13:32 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#10) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 11 Vijay Bellur 2016-06-13 15:17:53 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#11) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 12 Vijay Bellur 2016-06-13 17:03:06 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#12) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 13 Vijay Bellur 2016-08-17 18:37:34 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#13) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 14 Worker Ant 2016-08-22 12:53:20 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#14) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 15 Worker Ant 2016-09-15 16:04:55 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#15) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 16 Worker Ant 2016-09-16 01:21:19 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#16) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 17 Worker Ant 2016-09-16 07:07:32 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#17) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 18 Worker Ant 2016-09-16 12:37:30 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#18) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 19 Worker Ant 2016-09-17 14:08:04 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#19) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 20 Worker Ant 2016-09-18 02:27:16 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#20) for review on master by Kaleb KEITHLEY (kkeithle)

Comment 21 Worker Ant 2016-09-18 06:31:20 UTC
REVIEW: http://review.gluster.org/14085 (build: out-of-tree builds generates files in the wrong directory) posted (#21) for review on master by Niels de Vos (ndevos)

Comment 22 Worker Ant 2016-09-18 16:34:41 UTC
COMMIT: http://review.gluster.org/14085 committed in master by Niels de Vos (ndevos) 
------
commit e38dff5b4e0f0a25db664810fc3617eac44673ce
Author: Kaleb S KEITHLEY <kkeithle>
Date:   Tue Apr 26 17:04:04 2016 -0400

    build: out-of-tree builds generates files in the wrong directory
    
    And minor cleanup of a few of the Makefile.am files while we're
    at it.
    
    Rewrite the make rules to do what xdrgen does. Now we can get rid
    of xdrgen.
    
    Note 1. netbsd6's sed doesn't do -i. Why are we still running
    smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
    as it is.
    
    Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
    libglusterfs? A cut-and-paste mistake? It has no references to
    symbols in libglusterfs.
    
    Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
    regex that matches the same lines as the _extended_ regex
    "/#(ifndef|define|endif)/". To match the extended regex sed needs to
    be run with -r on Linux; with -E on *BSD. However NetBSD's and
    FreeBSD's sed helpfully also provide -r for compatibility. Using a
    basic regex avoids having to use a kludge in order to run sed with
    the correct option on OS X.
    
    Note 4. Not copying the bit of xdrgen that inserts copyright/license
    boilerplate. AFAIK it's silly to pretend that machine generated
    files like these can be copyrighted or need license boilerplate.
    The XDR source files have their own copyright and license; and
    their copyrights are bound to be more up to date than old
    boilerplate inserted by a script. From what I've seen of other
    Open Source projects -- e.g. gcc and its C parser files generated
    by yacc and lex -- IIRC they don't bother to add copyright/license
    boilerplate to their generated files.
    
    It appears that it's a long-standing feature of make (SysV, BSD,
    gnu) for out-of-tree builds to helpfully pretend that the source
    files it can find in the VPATH "exist" as if they are in the $cwd.
    rpcgen doesn't work well in this situation and generates files
    with "bad" #include directives.
    
    E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
    you get an #include directive in the generated .c file like this:
    
      ...
      #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
      ...
    
    which (obviously) results in compile errors on out-of-tree build
    because the (generated) header file doesn't exist at that location.
    Compared to `rpcgen ./glusterfs3-xdr.x` where you get:
    
      ...
      #include "glusterfs3-xdr.h"
      ...
    
    Which is what we need. We have to resort to some Stupid Make Tricks
    like the addition of various .PHONY targets to work around the VPATH
    "help".
    
    Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
    looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
    And don't delete the -I$(top_builddir)/rpc/xdr/... bits
    
    Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
    BUG: 1330604
    Signed-off-by: Kaleb S KEITHLEY <kkeithle>
    Reviewed-on: http://review.gluster.org/14085
    Tested-by: Niels de Vos <ndevos>
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Niels de Vos <ndevos>

Comment 23 Shyamsundar 2017-03-06 17:20:39 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report.

glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html
[2] https://www.gluster.org/pipermail/gluster-users/