Bug 1450947 - Autoconf leaves unexpanded variables in path names of non-shell-script text files
Summary: Autoconf leaves unexpanded variables in path names of non-shell-script text f...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: build
Version: 3.10
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kaleb KEITHLEY
QA Contact:
URL:
Whiteboard:
: 1450588 (view as bug list)
Depends On: 1444228
Blocks: glusterfs-3.10.3
TreeView+ depends on / blocked
 
Reported: 2017-05-15 12:44 UTC by Kaleb KEITHLEY
Modified: 2017-06-06 06:08 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.10.3
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1444228
Environment:
Last Closed: 2017-06-06 06:08:20 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Kaleb KEITHLEY 2017-05-15 12:44:54 UTC
+++ This bug was initially created as a clone of Bug #1444228 +++

Description of problem:

The GlusterFS build system makes use of the autoconf functionality of generating installed versions of certain text files from templates with .in extensions. In some cases it occurs that autotools managed variables make their ways into the template expansions. It can be a problem (but not always). For example, the mount.glusterfs script is produced as the template expansion of mount.glusterfs.in, which contains:

    prefix="@prefix@";
    exec_prefix=@exec_prefix@;
    cmd_line=$(echo "@sbindir@/glusterfs");

that might expand to (depending on configuration):

    prefix="/usr";
    exec_prefix=${prefix};
    cmd_line=$(echo "${exec_prefix}/sbin/glusterfs");

Here the definitions of exec_prefix and cmd_line include variables but it's not a problem, as mount.glusterfs is a shell script and according to shell script syntax, the variables will be substituted with the actual string values upon running the script.

However, there are other cases, like that of glusterd.vol.in / glusterd.vol, 

    option working-directory @GLUSTERD_WORKDIR@

becaming

    option working-directory ${prefix}/var/lib/glusterd

or that of geo-replication/syncdaemon/conf.py.in / /usr/libexec/glusterfs/glusterfind/conf.py,

    GLUSTERFS_LIBEXECDIR = '@GLUSTERFS_LIBEXECDIR@'
    GLUSTERD_WORKDIR = "@GLUSTERD_WORKDIR@"
    
    LOCALSTATEDIR = "@localstatedir@"
    UUID_FILE = "@GLUSTERD_WORKDIR@/glusterd.info"

becoming

    GLUSTERFS_LIBEXECDIR = '${exec_prefix}/libexec/glusterfs'
    GLUSTERD_WORKDIR = "${prefix}/var/lib/glusterd"
    
    LOCALSTATEDIR = "${prefix}/var"
    UUID_FILE = "${prefix}/var/lib/glusterd/glusterd.info"

where the expanded version of the file functions as config file or python script, which does not support shell style variables and these values will be used verbatim.

After some investigation, I identified the following list of *.in files where variables in the expansion are problematic:

- events/src/eventsapiconf.py.in
- extras/geo-rep/schedule_georep.py.in
- extras/glusterd.vol.in
- extras/snap_scheduler/conf.py.in
- geo-replication/src/peer_georep-sshkey.py.in
- geo-replication/src/peer_mountbroker.in
- geo-replication/src/peer_mountbroker.py.in
- geo-replication/syncdaemon/conf.py.in
- tools/glusterfind/glusterfind.in
- tools/glusterfind/src/tool.conf.in

I've done testing with the following configure invocations:

- ./configure
- ./configure --prefix=/usr
- RHEL's configure invocation
- Ubuntu's configure invocation


How reproducible:

Deterministically.


Steps to Reproduce:

In a GlusterFS source tree with configure script present (ie. either shipped by vendor or ./autogen.sh has been run) run the attached glusterdotin.sh script.

The script calls configure with various arguments (according to the above invocation list). For each invocation it shows information about the invocation (the arguments and summary of results) and a optionally a diff excerpt from the diffs of the possibly affected *.in files and their expanded versions. The diff excerpt shall contain those lines which exhibit the issue.


Actual results:

With some configure invocations a diff excerpt is shown.


Expected results:

No diff excerpt should be shown.


Additional info:

Before change https://review.gluster.org/16880 the issue was present, but among the tested configurations it occurred only with "./configure --prefix=/usr", whereby the following files were affected:
 
- events/src/eventsapiconf.py.in
- geo-replication/src/peer_mountbroker.py.in

However, https://review.gluster.org/16880 worsened the situation to a great extent -- after this change only RHEL's configuration featured flawless template expansion, and for the other three configurations a higher number of files (5 to 9) were affected. (Implied: https://review.gluster.org/16880 is not the final fix for Ubuntu build issues that are reported in the assiciated bug, Bug 1430841.)

--- Additional comment from Worker Ant on 2017-04-21 02:24:58 EDT ---

REVIEW: https://review.gluster.org/17096 (ensure autoconf template expansions resolve variables) posted (#1) for review on master by Csaba Henk (csaba)

--- Additional comment from Worker Ant on 2017-04-21 04:30:31 EDT ---

REVIEW: https://review.gluster.org/17096 (build: ensure autoconf template expansions resolve variables) posted (#2) for review on master by Csaba Henk (csaba)

--- Additional comment from Worker Ant on 2017-04-27 11:19:02 EDT ---

COMMIT: https://review.gluster.org/17096 committed in master by Kaleb KEITHLEY (kkeithle) 
------
commit a94b7df7b9308a590ebcac15536d22a386561f10
Author: Csaba Henk <csaba>
Date:   Fri Apr 21 19:11:24 2017 +0200

    build: ensure autoconf template expansions resolve variables
    
    Make sure the following autotools managed variables:
    
    GLUSTERFS_LIBEXECDIR
    GLUSTERD_MISCDIR
    GLUSTERD_VOLFILE
    LOCALSTATEDIR
    
    get expanded to literal paths when the configure
    script generates the installed versions of certain
    text files from *.in templates.
    
    This change is partly implemented by restoring some of
    the "eval echo $variable" style forced expansions in
    configure that were removed in
    If5219cadc51ae316f7ba2e2831d739235c77902d.
    
    BUG: 1444228
    Change-Id: I3b31b1259c5101252bbc37861683894e6eae29e6
    Signed-off-by: Csaba Henk <csaba>
    Reviewed-on: https://review.gluster.org/17096
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Niels de Vos <ndevos>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Tested-by: Kaleb KEITHLEY <kkeithle>

Comment 1 Worker Ant 2017-05-15 13:03:51 UTC
REVIEW: https://review.gluster.org/17298 (build: ensure autoconf template expansions resolve variables) posted (#1) for review on release-3.10 by Kaleb KEITHLEY (kkeithle)

Comment 2 Kaleb KEITHLEY 2017-05-23 11:25:34 UTC
*** Bug 1450588 has been marked as a duplicate of this bug. ***

Comment 3 Worker Ant 2017-05-24 00:16:28 UTC
COMMIT: https://review.gluster.org/17298 committed in release-3.10 by Kaleb KEITHLEY (kkeithle) 
------
commit 7e962090a074f298e021ef2d8f6c1e303c022eaf
Author: Csaba Henk <csaba>
Date:   Fri Apr 21 19:11:24 2017 +0200

    build: ensure autoconf template expansions resolve variables
    
    Make sure the following autotools managed variables:
    
    GLUSTERFS_LIBEXECDIR
    GLUSTERD_MISCDIR
    GLUSTERD_VOLFILE
    LOCALSTATEDIR
    
    get expanded to literal paths when the configure
    script generates the installed versions of certain
    text files from *.in templates.
    
    This change is partly implemented by restoring some of
    the "eval echo $variable" style forced expansions in
    configure that were removed in
    
      mainline BUG: 1444228
      master https://review.gluster.org/17096
    
    Change-Id: I3b31b1259c5101252bbc37861683894e6eae29e6
    BUG: 1450947
    Signed-off-by: Kaleb S. KEITHLEY <kkeithle>
    Signed-off-by: Csaba Henk <csaba>
    Reviewed-on: https://review.gluster.org/17298
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 4 Raghavendra Talur 2017-06-06 06:08:20 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.3, please open a new bug report.

glusterfs-3.10.3 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-June/031399.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.