Bug 1291537

Summary: [RFE] Provide mechanism to spin up reproducible test environment for all developers
Product: [Community] GlusterFS Reporter: Raghavendra Talur <rtalur>
Component: project-infrastructureAssignee: Raghavendra Talur <rtalur>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: mainlineCC: bugs, gluster-infra, mliyazud
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: glusterfs-3.8rc2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-16 13:50:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Raghavendra Talur 2015-12-15 05:39:23 UTC
Description of problem:

There should be a way for developers to obtain/spin up a test environment which
is identical to our upstream test infrastructure.

Comment 1 Vijay Bellur 2015-12-15 05:44:04 UTC
REVIEW: http://review.gluster.org/12753 (tests: Introduce a Vagrant VM based test environment) posted (#6) for review on master by Raghavendra Talur (rtalur)

Comment 2 Vijay Bellur 2015-12-22 14:59:15 UTC
REVIEW: http://review.gluster.org/12753 (tests: Introduce a Vagrant VM based test environment) posted (#7) for review on master by Raghavendra Talur (rtalur)

Comment 3 Vijay Bellur 2015-12-28 08:50:45 UTC
COMMIT: http://review.gluster.org/12753 committed in master by Raghavendra Talur (rtalur) 
------
commit 78305ce9604329250dd4a9f2335587957d71c701
Author: Raghavendra Talur <rtalur>
Date:   Tue Nov 24 18:18:01 2015 +0530

    tests: Introduce a Vagrant VM based test environment
    
    This introduces a mechanism using which a developer
    could easily test the Gluster code in a VM environment.
    
    Also, it will help bring uniformity in the environments
    used by various developers.
    
    How to use:
    1. git checkout -b custom-branch-name
    2. Make changes
    3. Execute ./run-tests-in-vagrant.sh
    
    What happens in the background:
    1. A new directory is created:
        tests/vagrant/vagrant-custom-branch-name
    
       It will serve as the Vagrant dir which has the
       Vagrantfile and related ansible playbooks.
    
       The VM is started using Vagrant and provisioned
       using ansible.
    
    2. The source dir is recursively copied over to the
       VM under /home/vagrant/glusterfs.
    
    3. Gluster is source installed in VM.
    
    What happens in the foreground:
    1. run-tests.sh is executed in VM using ssh and output is displayed
       in the same terminal with option to use ctrl-c to interrupt the test
       midway. The VM would still persist and you could ssh into it.
    
    Also, you can checkout a different branch elsewhere and execute
    run-tests-in-vagrant.sh there to get another VM which would
    execute tests on that code.
    
    If you wish to make some changes in the code, you could:
    a. Change the code in host and run the script again to repeat
       the whole process.
    OR
    b. vagrant ssh into the VM and make the changes in the VM.
    
    Co-authored-by: Kaushal M <kaushal>
    Co-authored-by: Michael Adam <obnox>
    
    Change-Id: Ic87801172c8b614cdecbdf2a765e1b3370a5faf7
    BUG: 1291537
    Signed-off-by: Michael Adam <obnox>
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/12753
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>

Comment 4 Vijay Bellur 2016-01-05 12:45:55 UTC
REVIEW: http://review.gluster.org/13175 (vagrant-test: Exit on critical errors) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 5 Vijay Bellur 2016-01-06 08:43:12 UTC
COMMIT: http://review.gluster.org/13175 committed in master by Raghavendra Talur (rtalur) 
------
commit cae9a5b3a1868a8bae25cd1ba9ebb41d698f39e5
Author: Raghavendra Talur <rtalur>
Date:   Wed Dec 30 18:15:04 2015 +0530

    vagrant-test: Exit on critical errors
    
    There were quite a few places where exiting the script
    made more sense.
    
    More debug messages have been added.
    
    Move back to top directory after the script is complete.
    
    Change-Id: I2a66ee3a68c41a3acd0b7168c56b801fb5567e5f
    BUG: 1291537
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13175
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Michael Adam <obnox>

Comment 6 Vijay Bellur 2016-01-18 09:50:27 UTC
REVIEW: http://review.gluster.org/13251 (vagrant-test: Use pre-baked box for better perf) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 7 Vijay Bellur 2016-01-18 09:51:47 UTC
REVIEW: http://review.gluster.org/13252 (vagrant-test: Autostart vagrant VMs on host boot) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 8 Vijay Bellur 2016-01-25 19:22:04 UTC
REVIEW: http://review.gluster.org/13251 (vagrant-test: Use pre-baked box for better perf) posted (#2) for review on master by Raghavendra Talur (rtalur)

Comment 9 Vijay Bellur 2016-02-04 11:28:14 UTC
REVIEW: http://review.gluster.org/13355 (vagrant-tests: Fix bug when git branch name has a /) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 10 Vijay Bellur 2016-02-06 22:07:56 UTC
COMMIT: http://review.gluster.org/13355 committed in master by Niels de Vos (ndevos) 
------
commit c7d2037b8c6293663280a759a2e447d5aef130bd
Author: Raghavendra Talur <rtalur>
Date:   Thu Feb 4 14:33:54 2016 +0530

    vagrant-tests: Fix bug when git branch name has a /
    
    It is possible for a git branch to have a / in it.
    
    Source copy from host to VM used relative path on
    the assumption that VM vagrant dir will always be
    three levels down the topdir of repo. This assumption
    breaks when git branch has a / in it.
    
    Of the two solutions to fix it:
    a. Mangle the git branch name to not have a /
    b. Accomodate the possibility of have a / in git branch name.
    
    I have chosen b) because that looks cleaner.
    
    Change-Id: I6b71c31da2f5f7c349d6d6882767768b2534d14f
    BUG: 1291537
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13355
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>

Comment 11 Vijay Bellur 2016-02-10 07:54:14 UTC
REVIEW: http://review.gluster.org/13251 (vagrant-test: Use pre-baked box for better perf) posted (#3) for review on master by Raghavendra Talur (rtalur)

Comment 12 Vijay Bellur 2016-02-10 08:36:20 UTC
REVIEW: http://review.gluster.org/13251 (vagrant-test: Use pre-baked box for better perf) posted (#4) for review on master by Raghavendra Talur (rtalur)

Comment 13 Vijay Bellur 2016-02-12 00:10:41 UTC
COMMIT: http://review.gluster.org/13251 committed in master by Jeff Darcy (jdarcy) 
------
commit e29bf0b1f102308f114e04421d80696eebfbf6e3
Author: Raghavendra Talur <rtalur>
Date:   Mon Jan 18 15:06:18 2016 +0530

    vagrant-test: Use pre-baked box for better perf
    
    Using Fedora cloud base leads to lot of downloading
    time for packages and also it is larger in disk size
    once imported into linux.
    
    Change-Id: I2e878e7cb29a79ade4d6e517075ab523062deb53
    BUG: 1291537
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13251
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Jeff Darcy <jdarcy>

Comment 14 Vijay Bellur 2016-02-17 10:12:30 UTC
REVIEW: http://review.gluster.org/13458 (vagrant-test: Install helper packages) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 15 Vijay Bellur 2016-02-18 07:35:26 UTC
REVIEW: http://review.gluster.org/13252 (vagrant-test: Autostart vagrant VMs on host boot) posted (#2) for review on master by Raghavendra Talur (rtalur)

Comment 16 Vijay Bellur 2016-02-18 10:18:54 UTC
REVIEW: http://review.gluster.org/13252 (vagrant-test: Provide option to autostart vm on host boot) posted (#3) for review on master by Raghavendra Talur (rtalur)

Comment 17 Vijay Bellur 2016-02-18 10:28:11 UTC
REVIEW: http://review.gluster.org/13252 (vagrant-test: Provide option to autostart vm on host boot) posted (#4) for review on master by Raghavendra Talur (rtalur)

Comment 18 Vijay Bellur 2016-02-25 18:18:13 UTC
COMMIT: http://review.gluster.org/13458 committed in master by Raghavendra Talur (rtalur) 
------
commit e991d14a3a2aaf19218cbb5993a94de263769cb4
Author: Raghavendra Talur <rtalur>
Date:   Fri Jan 22 12:08:52 2016 +0530

    vagrant-test: Install helper packages
    
    It is useful to have helper packages like gdb, sysstat, valgrind
    installed on the dev machines. The pre-baked box that was merged in
    commit e29bf0b1f102308f114e04421d80696eebfbf6e3 has those packages
    installed.
    
    This patch updates the list of packages installed in the install-pkgs
    role for future reference.
    
    Change-Id: I2ee25d40708a4d29652b5bd40b66e3b1fcd33f0c
    BUG: 1291537
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13458
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>

Comment 19 Vijay Bellur 2016-02-25 18:24:39 UTC
COMMIT: http://review.gluster.org/13252 committed in master by Raghavendra Talur (rtalur) 
------
commit 025eebecefa45c8a3e6eb33fe382dd701ac9c28a
Author: Raghavendra Talur <rtalur>
Date:   Mon Jan 18 14:44:46 2016 +0530

    vagrant-test: Provide option to autostart vm on host boot
    
    If the machines are being used as test machines for active development,
    user might want to have them autostart on host boot. You can use -a as
    short option or --autostart as longoption with run-tests-in-vagrant.sh
    to set the autostart flag on the VM. The autostart value is set to off
    by default.
    
    It is done using virsh until vagrant-libvirt supports it in the API.
    
    Change-Id: I5b53ea6d850c991d548dbac0bb8fadd528cabf41
    BUG: 1291537
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/13252
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>

Comment 20 Vijay Bellur 2016-03-03 20:17:44 UTC
REVIEW: http://review.gluster.org/13597 (vagrant-tests: Fix CFLAGS not passed to configure error) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 21 Mike McCune 2016-03-28 23:30:47 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 22 Vijay Bellur 2016-04-06 14:09:54 UTC
REVIEW: http://review.gluster.org/13904 (tests: improve vagrant usability) posted (#2) for review on master by Raghavendra Talur (rtalur)

Comment 23 Vijay Bellur 2016-04-06 14:32:10 UTC
COMMIT: http://review.gluster.org/13904 committed in master by Jeff Darcy (jdarcy) 
------
commit 21d8a461ae712539f5a0bf7e6889df29750c5256
Author: Jeff Darcy <jdarcy>
Date:   Mon Apr 4 17:23:53 2016 -0400

    tests: improve vagrant usability
    
    The first and most important change is to deal with the fact that
    "localhost" is an IPv6 address in the vagrant box we're using.  This was
    non-obviously behind some of the NFS tests failing to run properly, so
    there's now an Ansible role etc. to make the necessary modifications in
    /etc/hosts.  Also added "-j" to the make command, and changed the
    invocation of "run-tests.sh" so it passes through any arguments instead
    of running all tests every time.
    
    BUG: 1291537
    Change-Id: I5e6125de043c41fdb2866875e999cb1fcd72ba52
    Signed-off-by: Jeff Darcy <jdarcy>
    Reviewed-on: http://review.gluster.org/13904
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra Talur <rtalur>
    Smoke: Gluster Build System <jenkins.com>

Comment 24 Vijay Bellur 2016-04-21 21:47:34 UTC
REVIEW: http://review.gluster.org/14046 (vagrant-test: added more options and centos6 base box) posted (#1) for review on master by Raghavendra Talur (rtalur)

Comment 25 Vijay Bellur 2016-04-22 09:16:49 UTC
REVIEW: http://review.gluster.org/14046 (vagrant-test: added more options and centos6 base box) posted (#2) for review on master by Raghavendra Talur (rtalur)

Comment 26 Vijay Bellur 2016-04-25 14:28:42 UTC
COMMIT: http://review.gluster.org/14046 committed in master by Jeff Darcy (jdarcy) 
------
commit ef3d6fd06cf0712774e721eaf1d0b072f54e0b8a
Author: Raghavendra Talur <rtalur>
Date:   Wed Apr 20 15:02:16 2016 +0530

    vagrant-test: added more options and centos6 base box
    
    Options added are
    
    1. destroy-now option
    Now you can call run-tests-in-vagrant.sh with --destroy-now option to
    destroy the VM corresponding to the git branch and delete the
    corresponding vagrant dir.
    Usage: run-tests-in-vagrant.sh --destroy-now
    
    2. os option
    With --os option you can specify the os to be used for VM.
    Default is fedora and this patch adds support for centos6.
    Usage: run-tests-in-vagrant.sh --os="centos6"
    
    3. destroy-after-test option
    When specified the VM is deleted right after the tests are executed. If
    the script exited before running tests then the VM is left as is for
    further debugging.
    
    4. verbose option
    Many log messages which were mainly added for initial debugging have
    been removed now. All vagrant and compilation outputs have been redirected to
    /dev/null. To enable these logs please use --verbose option.
    
    5. ssh option
    --ssh option now allows one to ssh into the VM from topdir of source
    code thereby avoiding the hassle of cd'ing into the vagrant dir.
    
    Box added
    
    1. This patch adds a centos6 base box which is currently made available
    at atlas.hashicorp.com. Template files for this box are available
    under tests/vagrant/vagrant-template-centos6 directory. The default box is
    moved under vagrant-template-fedora directory to keep up with the naming
    convention.
    
    Enhancements
    
    This patch also modifies the cpu and memory allocation mechanism for
    the VMs. All the cpus on the host are used and 1/4 of the memory
    available on host is used.
    
    Bug Fixes
    1. Fix CFLAGS not passed to configure error
        If CFLAGS is assigned and then configure is called under sudo, the cflags
        are not passed onto configure.
    
    Change-Id: Iee1811a7d2b77785f6167f4706135a8f584a6bbd
    BUG: 1291537
    Signed-off-by: Raghavendra Talur <rtalur>
    Reviewed-on: http://review.gluster.org/14046
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Jeff Darcy <jdarcy>

Comment 27 Niels de Vos 2016-06-16 13:50:45 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user