Bug 1091777

(In reply to Gilles Dubreuil from comment #0)
> Full validation is needed for puppet-gluster module on RHEL7 and Fedora 20
> in order to support RDO on those platforms.

How accurate is the "urgent" severity? When is this needed by?

In any case, already working on this.
Initial WIP feature branch is at:
https://github.com/purpleidea/puppet-gluster/tree/feat/yamldata
Specific commit:
https://github.com/purpleidea/puppet-gluster/commit/fef7461994e48b17b4e1f95ffbb767033c3651ed

More to come shortly...

(In reply to James from comment #1)

> How accurate is the "urgent" severity? When is this needed by?

For RHEL7 which could be out anytime. There is less than 2 months all wrapped up!

RHOS wise it will be available with next RHOS update A4.5 (havana) or the coming RHOS A5.0 (Icehouse).

For RDO it's more tied because it should potentially support RHEL7 as soon as out wit RDO updates being continuous. 

For Fedora, well it's already yesterday :(

Do you have an estimated time for the Alpha - or Beta if you think we're already there?

> In any case, already working on this.
> Initial WIP feature branch is at:
> https://github.com/purpleidea/puppet-gluster/tree/feat/yamldata
> Specific commit:
> https://github.com/purpleidea/puppet-gluster/commit/
> fef7461994e48b17b4e1f95ffbb767033c3651ed
> 
> More to come shortly...

That's great, thanks for the time!

(In reply to Gilles Dubreuil from comment #2)
> (In reply to James from comment #1)
> 
> > How accurate is the "urgent" severity? When is this needed by?
> 
> For RHEL7 which could be out anytime. There is less than 2 months all
> wrapped up!

I will definitely have it ported before two months :)

> 
> RHOS wise it will be available with next RHOS update A4.5 (havana) or the
> coming RHOS A5.0 (Icehouse).
> 
> For RDO it's more tied because it should potentially support RHEL7 as soon
> as out wit RDO updates being continuous. 
> 
> For Fedora, well it's already yesterday :(

Okay. Well I'll look into this now. I'm optimistic :)

> 
> Do you have an estimated time for the Alpha - or Beta if you think we're
> already there?

Puppet-Gluster doesn't really have alpha/beta/rc. It just has git master for now :) I'll hack on this tonight, and we'll see how it looks by tomorrow.

> 
> > In any case, already working on this.
> > Initial WIP feature branch is at:
> > https://github.com/purpleidea/puppet-gluster/tree/feat/yamldata
> > Specific commit:
> > https://github.com/purpleidea/puppet-gluster/commit/
> > fef7461994e48b17b4e1f95ffbb767033c3651ed
> > 
> > More to come shortly...
> 
> That's great, thanks for the time!

NP :)

James

Branch updated and rebased:
https://github.com/purpleidea/puppet-gluster/tree/feat/yamldata

I haven't tested this branch much, but I feel pretty good about it.

It includes a slew of different patches: some that add the multi-distro-based yaml infrastructure, some that add replace static strings with variables, and some that fill data into yaml files.

The what? Right. So I've been wanting to use this clever yaml-in-module technique, as it's quite a smart way to keep distro specific variables out of the code, and into data files (yaml). To add support for your specific distro, drop the distro specific changes into the most common branch of the data/ tree.

This is also a really easy way to use different binaries on a QA/QE setup without having to patch Puppet-Gluster.

Gilles, this patch is dedicated to you:
https://github.com/purpleidea/puppet-gluster/commit/3e7a720528d5f70878e7197a33df50e7a8942d63

Please feel free to test and *report* bugs or missing features here.
More patches to come :)

Cheers!

Hi James,

Initial feedback regarding testing feat/yamldata branch against RHEL7 RC1:

Missing comma
=============
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Syntax error at 'owner'; expected '}' at /usr/share/openstack-puppet/modules/gluster/manifests/params.pp:75 on node f2-glu1.os.tst

Submitted following Pull Request which fixes it:
https://github.com/purpleidea/puppet-gluster/pull/14

I cannot go any further at this stage because I cannot install puppet on the RHEL7 nodes without using EPEL7 repo, in which case the puppet version (puppet-3.5.1-1.el7.noarch) will be too high for the Foreman puppet-server version (puppet-server-3.2.4-3.el6_5.noarch)

(In reply to Gilles Dubreuil from comment #5)
> Hi James,
> 
> Initial feedback regarding testing feat/yamldata branch against RHEL7 RC1:
> 
> Missing comma
> =============
> Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
> Syntax error at 'owner'; expected '}' at
> /usr/share/openstack-puppet/modules/gluster/manifests/params.pp:75 on node
> f2-glu1.os.tst
> 
> Submitted following Pull Request which fixes it:
> https://github.com/purpleidea/puppet-gluster/pull/14
Woops. Good catch. Merged :) First patch! w000t!


> 
> I cannot go any further at this stage because I cannot install puppet on the
> RHEL7 nodes without using EPEL7 repo, in which case the puppet version
> (puppet-3.5.1-1.el7.noarch) will be too high for the Foreman puppet-server
> version (puppet-server-3.2.4-3.el6_5.noarch)

Hm, can you elaborate on this point please?

James,

Thanks for merging that incredible patch! ;)

The puppet issue has nothing to do with puppet-gluster, it's the matter of having the correct puppet version on RHEL7 side (client).
The puppet version on the agent side is supposed to be aligned with the puppet-master side.
Anyway, I currently have 3.4.3 on the client side and 3.2.4 on the foreman/foreman-proxy.
That's not ideal but that seems to works so far, at least I'm not getting the errors/warnings. That said it doesn't mean it's correct although debug mode doesn't anything abnormal either.

From that the next issue I'm, getting:

Error: Failed to apply catalog: Could not find dependent File[/xml.py] for Package[python-argparse] at /usr/share/openstack-puppet/modules/gluster/manifests/xml.pp:27

This seems like a dependency issue, although $vardir is empty instead of something like this "/var/lib/puppet/tmp/gluster/" .

(In reply to Gilles Dubreuil from comment #7)
> James,
> 
> Thanks for merging that incredible patch! ;)
My pleasure :)
Just before I did it, I made an "official release". 0.0.3
It and other code releases are available at:
https://download.gluster.org/pub/gluster/purpleidea/puppet-gluster/
The point being that I feel it's safe to be a bit more cavalier with git master :)
There is a 0.0.3 tag available if you want to point to that for more stable things.

I'll blog about some of these things shortly...

> 
> The puppet issue has nothing to do with puppet-gluster, it's the matter of
> having the correct puppet version on RHEL7 side (client).
> The puppet version on the agent side is supposed to be aligned with the
> puppet-master side.
> Anyway, I currently have 3.4.3 on the client side and 3.2.4 on the
> foreman/foreman-proxy.
> That's not ideal but that seems to works so far, at least I'm not getting
> the errors/warnings. That said it doesn't mean it's correct although debug
> mode doesn't anything abnormal either.
> 
> From that the next issue I'm, getting:
> 
> Error: Failed to apply catalog: Could not find dependent File[/xml.py] for
> Package[python-argparse] at
> /usr/share/openstack-puppet/modules/gluster/manifests/xml.pp:27
> 
> This seems like a dependency issue, although $vardir is empty instead of
> something like this "/var/lib/puppet/tmp/gluster/" .

Sounds like you didn't include puppet::vardir
That's a requirement... It's listed in that old README.pdf i think.
If that's your issue, let me know and I'll document it in the README (txt).

(In reply to James from comment #8)
> > Error: Failed to apply catalog: Could not find dependent File[/xml.py] for
> > Package[python-argparse] at
> > /usr/share/openstack-puppet/modules/gluster/manifests/xml.pp:27
> > 
> > This seems like a dependency issue, although $vardir is empty instead of
> > something like this "/var/lib/puppet/tmp/gluster/" .
> 
> Sounds like you didn't include puppet::vardir
> That's a requirement... It's listed in that old README.pdf i think.
> If that's your issue, let me know and I'll document it in the README (txt).

I'm using the same manifest tested and validated against RHEL6.5+:
https://github.com/redhat-openstack/astapor/pull/150

gluster::vardir is already included in the above 'Gluster Server' hostgroup I'm testing with.

More information:

On the other hosts from the same test set I get slightly different results all related to vardir issue:


f2-glu2
-------
Error: Failed to apply catalog: Could not find dependency File[/var/lib/puppet/tmp/] for File[/var/lib/puppet/tmp/common/] at /usr/share/openstack-puppet/modules/common/manifests/vardir.pp:49
-------

f2-glu3
-------
Error: Failed to apply catalog: Could not find dependency File[/var/lib/puppet/tmp/] for File[/var/lib/puppet/tmp/gluster/] at /usr/share/openstack-puppet/modules/gluster/manifests/vardir.pp:49
-------

On all hosts, '/var/lib/puppet/tmp/' exists.

Could this issue related to BZ#1095985 ?

(In reply to Gilles Dubreuil from comment #10)
> More information:
> 
> On the other hosts from the same test set I get slightly different results
> all related to vardir issue:
> 
> 
> f2-glu2
> -------
> Error: Failed to apply catalog: Could not find dependency
> File[/var/lib/puppet/tmp/] for File[/var/lib/puppet/tmp/common/] at
> /usr/share/openstack-puppet/modules/common/manifests/vardir.pp:49
> -------
> 
> f2-glu3
> -------
> Error: Failed to apply catalog: Could not find dependency
> File[/var/lib/puppet/tmp/] for File[/var/lib/puppet/tmp/gluster/] at
> /usr/share/openstack-puppet/modules/gluster/manifests/vardir.pp:49
> -------
> 
> On all hosts, '/var/lib/puppet/tmp/' exists.
> 
> Could this issue related to BZ#1095985 ?
Sort of.

I realized that the latest puppet versions change a lot of things. I think the parse order is now the execution order in some instances. I think there's an option to choose it... So actually a bunch of stuff might be broken depending on puppet version. I rebased the branch with some new changes, and also I still have some testing to do on my side, because I found some issues, and I've never had a chance to properly test this till recently...

I'll hack on this over the weekend too.

James,

Just to follow up

(In reply from comment #8)

> There is a 0.0.3 tag available if you want to point to that for more stable
> things.
> 

Thanks for the tag.

At the moment OPM (havana and master branches (Icehouse) are pointing to puppet-gluster commit 80c2b13448. Because that's what I tested with at the time.
Will have to test with 0.0.3 corresponding commit to move the submodules HEAD

(In reply from comment #11)
> I'll hack on this over the weekend too.

Any update?

Thanks

(In reply to Gilles Dubreuil from comment #12)
> James,
> 
> Just to follow up
> 
> (In reply from comment #8)
> 
> > There is a 0.0.3 tag available if you want to point to that for more stable
> > things.
> > 
> 
> Thanks for the tag.
> 
> At the moment OPM (havana and master branches (Icehouse) are pointing to
> puppet-gluster commit 80c2b13448. Because that's what I tested with at the
> time.
> Will have to test with 0.0.3 corresponding commit to move the submodules HEAD
> 
> (In reply from comment #11)
> > I'll hack on this over the weekend too.
> 
> Any update?
> 
> Thanks

Hey,

I've been pulled in a lot of different directions for all sorts of new projects.
I tested git master today, and it seemed to work with all the porting changes :)
I also tested it with two disks per host, so the disk stuff seems to be working too.
I didn't test it on Fedora 20 though, although I made some Vagrant boxes for this last night:
https://download.gluster.org/pub/gluster/purpleidea/vagrant/fedora-20/

Please feel free to test those boxes, don't know how well they work yet.
Do you have any issues with this on Fedora 20?

HTH,
James

Ok, I'll try the 'Gluster Server' Hostgroup against against RHEL7RC and Fedora20

James,

The feat/yamldata branch you initially pointed over doesn't seem to have moved much. Could you please clarify what branch is to use against RHEL7/CENTOS7/Fedora20?

Using master in the meantime.

Thanks

(In reply to Gilles Dubreuil from comment #15)
> James,
> 
> The feat/yamldata branch you initially pointed over doesn't seem to have
> moved much. Could you please clarify what branch is to use against
> RHEL7/CENTOS7/Fedora20?

feat/yamldata has been merged to git master for a while now.
It works on CentOS 6.5, but hasn't been tested on F20 yet.
I made some untested vagrant images though:
https://download.gluster.org/pub/gluster/purpleidea/vagrant/fedora-20/

So basically, everything should be ready for you, but I have no time to test anything right now :) Ping me with any bugs. Hopefully zero.

> 
> Using master in the meantime.
> 
> Thanks

Cheers

Current status, please understand some of the following could migrate to new BZs/RFEs:

Dependencies:

1. Currently using gluster epel repo
   For now until RHS repo available for RHEL7 (not found any but I could be wrong)

2. puppet-puppet master module needed for puppet::facter.
   I think we don't need that on RedHat Oses family.
   This could be refactored as type/provider or at least in the meantime with
   platform detection

Issues:

1. The current gluster server cluster fails at mount stage, the following is happening times 3, for each cinder, glance and swift slices. 

-----------
Notice: /Stage[main]/Quickstack::Gluster::Server/Gluster::Brick[f2-glu1.os.tst:/swift]/Mount[/swift]/ensure: ensure changed 'unmounted' to 'mounted'
Error: /Stage[main]/Quickstack::Gluster::Server/Gluster::Brick[f2-glu1.os.tst:/swift]/Mount[/swift]: Could not evaluate: Execution of '/usr/bin/mount /swift' returned 1: mount: can't find UUID=64a2e07c-515f-4d48-bf2e-ca96038261cb
-----------

Now this could be two folded:

Trying to mount the partitions using the device mapper path instead of UUID also fails:
------------
# mount -t xfs /dev/mapper/vg_f2--glu1.os.tst+_cinder-lv_f2--glu1.os.tst+_cinder /cinder
mount: mount /dev/mapper/vg_f2--glu1.os.tst+_cinder-lv_f2--glu1.os.tst+_cinder on /cinder failed: Function not implemented
------------

But digging out messages log from journalctl shows:
-----------
kernel: XFS (dm-2): device supports 512 byte sectors (not 0)
-----------

It seems XFS partitioning fails without any errors or warnings from puppet-gluster. Because the process went quite far, I wrongly assumed the partition was done. :(

Also, thanks Dave Chinner for help about above XFS error:
"That's a regression that recently surfaced. That device does
not contain an XFS filesystem, and where XFS expects to find the
sector size in the superblock, it's finding a zero value.  If should
be returning EINVAL, not ENOSYS."

(In reply to Gilles Dubreuil from comment #18)
> It seems XFS partitioning fails without any errors or warnings from
> puppet-gluster. Because the process went quite far, I wrongly assumed the
> partition was done. :(
> 
> Also, thanks Dave Chinner for help about above XFS error:
> "That's a regression that recently surfaced. That device does
> not contain an XFS filesystem, and where XFS expects to find the
> sector size in the superblock, it's finding a zero value.  If should
> be returning EINVAL, not ENOSYS."

Thanks for tracking that down :) Any change you can refer me to an open bug so that I can cc myself on it?

Cheers

I discovered the test set was flawed with:

1. Two different puppet versions were installed (repectively from RHEL7-EPEL and RHOS5)!
2. The test discs weren't properly wiped (zeroed)

From a clean fresh restart:
1. XFS are properly created 
2. Logical volumes mounted
3. Puppet output is consistent

The process now stops because of failed dependencies with python-argparse package missing, that error was actually displayed during previous runs but I ignored it in favor of XFS false issue:

-------------
...snipped...

Notice: /Stage[main]/Gluster::Xml/File[/var/lib/puppet/tmp/gluster/xml.py]: Dependency Package[python-argparse] has failures: true
Warning: /Stage[main]/Gluster::Xml/File[/var/lib/puppet/tmp/gluster/xml.py]: Skipping because of failed dependencies

...snipped...
Notice: /Stage[main]/Quickstack::Gluster::Server/Gluster::Volume[glance]/Exec[gluster-volume-stuck-glance]: Dependency Package[python-argparse] has failures: true
...etc...

...snipped...
-------------


Full output is available here: http://fpaste.org/106728/14017791/

(In reply to Gilles Dubreuil from comment #20)
> I discovered the test set was flawed with:
> 
> 1. Two different puppet versions were installed (repectively from RHEL7-EPEL
> and RHOS5)!
> 2. The test discs weren't properly wiped (zeroed)

I have two FAQ items about this:

https://github.com/purpleidea/puppet-gluster/blob/master/DOCUMENTATION.md#provisioning-fails-with-cant-open-devsdb1-exclusively

https://github.com/purpleidea/puppet-gluster/blob/master/DOCUMENTATION.md#provisioning-fails-with-cannot-open-devsdb1-device-or-resource-busy

Let me know if you think we need a third and how your error manifested itself...

> 
> From a clean fresh restart:
> 1. XFS are properly created 
> 2. Logical volumes mounted
> 3. Puppet output is consistent
> 
> The process now stops because of failed dependencies with python-argparse
> package missing, that error was actually displayed during previous runs but
> I ignored it in favor of XFS false issue:

Right... So I'm guessing RHEL7 is probably similar to Fedora in that it comes with argparse package in python. I patched this for Fedora20 already...

Can you run:
python
import argparse
# does it work ?

If so, this patch should fix your issue:

https://github.com/purpleidea/puppet-gluster/tree/bug/rhel7-python
https://github.com/purpleidea/puppet-gluster/commit/34b53c59815d6cd195753c76268bc2ea09517af3

If you can confirm about the other changes in RHEl7 that would be great. Look at Fedora.yaml for a decent guess of common changes...

> 
> -------------
> ...snipped...
> 
> Notice: /Stage[main]/Gluster::Xml/File[/var/lib/puppet/tmp/gluster/xml.py]:
> Dependency Package[python-argparse] has failures: true
> Warning: /Stage[main]/Gluster::Xml/File[/var/lib/puppet/tmp/gluster/xml.py]:
> Skipping because of failed dependencies
> 
> ...snipped...
> Notice:
> /Stage[main]/Quickstack::Gluster::Server/Gluster::Volume[glance]/
> Exec[gluster-volume-stuck-glance]: Dependency Package[python-argparse] has
> failures: true
> ...etc...
> 
> ...snipped...
> -------------
> 
> 
> Full output is available here: http://fpaste.org/106728/14017791/

> Right... So I'm guessing RHEL7 is probably similar to Fedora in that it
> comes with argparse package in python. I patched this for Fedora20 already...
> 
> Can you run:
> python
> import argparse
> # does it work ?
Yeah, it does. Which makes sense since RHEL7 is inherited from F18/19

> 
> If so, this patch should fix your issue:
> 
> https://github.com/purpleidea/puppet-gluster/tree/bug/rhel7-python
> https://github.com/purpleidea/puppet-gluster/commit/
> 34b53c59815d6cd195753c76268bc2ea09517af3

That doesn't fix the problem, still searching for python-argparse.

Debugging shows the  ${::gluster::params::package_python_argparse} is not empty and still assigned 'python-argparse', therefore the test in gluster/manifest/xml.pp doesn't fail and it triggers the package installation.

So maybe the YAML layer doesn't work, but not error message are displayed so far.

BTW, using Python to parse xml results from Gluster is, I believe, creating unnecessary dependencies that could be avoided by writing Ruby function. I'll send a email about that separately.


> If you can confirm about the other changes in RHEl7 that would be great.
> Look at Fedora.yaml for a decent guess of common changes...
> 

The yaml (hiera) approach, I think, is good.
But because of above, I'm afraid it's not validated yet.

> > If so, this patch should fix your issue:
> > 
> > https://github.com/purpleidea/puppet-gluster/tree/bug/rhel7-python
> > https://github.com/purpleidea/puppet-gluster/commit/
> > 34b53c59815d6cd195753c76268bc2ea09517af3
> 
> That doesn't fix the problem, still searching for python-argparse.

Please run ls -lAh /etc/puppet/modules # or wherever you puppet modules dir is.

> 
> Debugging shows the  ${::gluster::params::package_python_argparse} is not
> empty and still assigned 'python-argparse', therefore the test in
> gluster/manifest/xml.pp doesn't fail and it triggers the package
> installation.
> 
> So maybe the YAML layer doesn't work, but not error message are displayed so
> far.

It should either work, or you should see a warning.

> 
> BTW, using Python to parse xml results from Gluster is, I believe, creating
> unnecessary dependencies that could be avoided by writing Ruby function.
> I'll send a email about that separately.

I completely disagree. System python is the dependency.

> 
> 
> > If you can confirm about the other changes in RHEl7 that would be great.
> > Look at Fedora.yaml for a decent guess of common changes...
> > 
> 
> The yaml (hiera) approach, I think, is good.
> But because of above, I'm afraid it's not validated yet.

It works for me. I suspect your setup is broken. Please test with puppet-gluster+vagrant.

Created attachment 901944 [details]
Puppet agent log

(In reply to James from comment #23)

> > That doesn't fix the problem, still searching for python-argparse.
> 
> Please run ls -lAh /etc/puppet/modules # or wherever you puppet modules dir
> is.

On foreman the modules are in different places.
What are you looking for?

> > So maybe the YAML layer doesn't work, but not error message are displayed so
> > far.
> 
> It should either work, or you should see a warning.

Apparently it does not. See attached output in debug mode.


> > BTW, using Python to parse xml results from Gluster is, I believe, creating
> > unnecessary dependencies that could be avoided by writing Ruby function.
> > I'll send a email about that separately.
> 
> I completely disagree. System python is the dependency.

This is a separate discussion about the fact the use of Exec should be refactored, whenever possible, with Types/Providers custom resources.

> > > If you can confirm about the other changes in RHEl7 that would be great.
> > > Look at Fedora.yaml for a decent guess of common changes...
> > > 
> > 
> > The yaml (hiera) approach, I think, is good.
> > But because of above, I'm afraid it's not validated yet.
> 
> It works for me. I suspect your setup is broken. Please test with
> puppet-gluster+vagrant.

Vagrant is not supported yet by Quickstack.

(In reply to Gilles Dubreuil from comment #24)

> > > So maybe the YAML layer doesn't work, but not error message are displayed so
> > > far.
> > 
> > It should either work, or you should see a warning.
> 
> Apparently it does not. See attached output in debug mode.
> 
As I said, the bug/rhel7-python still doesn't work. I've no idea why the variable ${::gluster::params::package_python_argparse}" is not empty.
I don't have much experience with hiera.
That's the current blocker.

That said, working around above issue by commenting out the whole above block (from lines 26 to 31) allow the catalog run through:
https://github.com/purpleidea/puppet-gluster/blob/bug/rhel7-python/manifests/xml.pp#L26

Everything seems to be in place, although gluster refuses to connect when querying the volumes. It looks like BZ#1065632.
But after a cycle of 'service glusterd restart', 'puppet agent' sync, and disabling selinux, the volumes are in place.

Created attachment 902748 [details]
Modules list

(In reply to Gilles Dubreuil from comment #26)
> Created attachment 902748 [details]
> Modules list

# ls -la /usr/share/openstack-foreman-installer/puppet/modules /usr/share/openstack-puppet/modules

/usr/share/openstack-foreman-installer/puppet/modules:
total 16
drwxr-xr-x. 4 root root 4096 May 26 01:21 .
drwxr-xr-x. 3 root root 4096 May 26 01:21 ..
drwxr-xr-x. 7 root root 4096 May 26 01:21 ntp
drwxr-xr-x. 5 root root 4096 May 26 01:21 quickstack

/usr/share/openstack-puppet/modules:
total 164
drwxr-xr-x. 41 root root 4096 Jun  5 05:59 .
drwxr-xr-x.  3 root root 4096 Jun  5 02:15 ..
drwxr-xr-x.  8 root root 4096 Jun  5 02:15 apache
drwxr-xr-x.  6 root root 4096 Jun  5 02:15 ceilometer
drwxr-xr-x.  5 root root 4096 Jun  5 02:15 certmonger
drwxr-xr-x.  5 root root 4096 Jun  5 02:15 cinder
drwxr-xr-x.  5 root root 4096 Jun  5 02:15 common
drwxr-xr-x.  4 root root 4096 Jun  5 02:15 concat
drwxr-xr-x.  5 root root 4096 Jun  5 02:15 firewall
drwxr-xr-x.  6 root root 4096 Jun  5 02:15 galera
drwxr-xr-x.  7 root root 4096 Jun  5 02:15 glance
drwxr-xr-x. 11 root root 4096 May 27 00:07 gluster
drwxr-xr-x.  9 root root 4096 Jun  5 02:16 gluster-RPM
drwxr-xr-x.  6 root root 4096 Jun  5 02:16 haproxy
drwxr-xr-x.  6 root root 4096 Jun  5 02:16 heat
drwxr-xr-x.  5 root root 4096 Jun  5 02:16 horizon
drwxr-xr-x.  5 root root 4096 Jun  5 02:16 inifile
drwxr-xr-x.  9 root root 4096 Jun  5 02:16 keystone
drwxr-xr-x.  7 root root 4096 Jun  5 02:16 memcached
drwxr-xr-x.  7 root root 4096 Jun  5 02:16 mongodb
drwxr-xr-x.  8 root root 4096 Jun  5 02:16 mysql
drwxr-xr-x.  6 root root 4096 Jun  5 02:16 neutron
drwxr-xr-x.  7 root root 4096 Jun  5 02:16 nova
drwxr-xr-x.  4 root root 4096 Jun  5 02:16 nssdb
drwxr-xr-x.  7 root root 4096 Jun  5 02:16 openstack
drwxr-xr-x.  6 root root 4096 Jun  5 02:16 pacemaker
drwxr-xr-x.  6 root root 4096 Jun  5 05:59 puppet
drwxr-xr-x.  6 root root 4096 Jun  5 05:58 puppet-RPM
drwxr-xr-x.  5 root root 4096 Jun  5 02:16 qpid
drwxr-xr-x.  8 root root 4096 Jun  5 02:16 rabbitmq
drwxr-xr-x.  7 root root 4096 Jun  5 02:16 rsync
drwxr-xr-x.  7 root root 4096 Jun  5 02:16 ssh
drwxr-xr-x.  8 root root 4096 Jun  5 02:16 staging
drwxr-xr-x.  6 root root 4096 Jun  5 02:16 stdlib
drwxr-xr-x.  9 root root 4096 Jun  5 02:16 swift
drwxr-xr-x.  4 root root 4096 Jun  5 02:16 sysctl
drwxr-xr-x.  4 root root 4096 Jun  5 02:16 tempest
drwxr-xr-x.  5 root root 4096 Jun  5 02:16 vcsrepo
drwxr-xr-x.  4 root root 4096 Jun  5 02:16 vlan
drwxr-xr-x.  5 root root 4096 Jun  5 02:16 vswitch
drwxr-xr-x.  7 root root 4096 Jun  5 02:16 xinetd


As you can see in the above lists you have provided (thank you) there is no "module-data" module. It is essential for you to add it for the multi-os aspect of puppet-gluster to work. In other words, puppet-gluster only knows to avoid installing python-argparse on fedora20 when it knows it's on a different os, which it needs that module for. If you don't use it, it defaults to thinking that you are running on CentOS 6.x (backwards compatibility mode).
use: https://github.com/purpleidea/puppet-module-data/tree/4b3ad1cc239d7831616e69796184e400de0f5fe4

eg:
https://github.com/purpleidea/puppet-module-data

Cheers,
James

Making progress with hiera and in module-data approach!

Added https://github.com/ripienaar/puppet-module-data
to the modules list (/usr/share/openstack-puppet/modules)

But now the catalogue fails entirely:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::AST::Resource failed with error NameError: uninitialized constant Puppet::DataBinding::LookupError at /usr/share/openstack-foreman-installer/puppet/modules/quickstack/manifests/gluster/server.pp:41 on node f2-glu1.os.tst

Above line 41 corresponds to last line of this block:
---------------
  $vip  = ''
  $vrrp =  false

  class {'::gluster::server':
    vip       => $vip,
    vrrp      => $vrrp,
    repo      => false,
    shorewall => false,
  }
--------------

It seems the issue is in gluster::server, but haven't found why.

Also it seems others had a similar issue:

https://github.com/ripienaar/puppet-module-data/issues/6

Actually as soon as the module-data gets into the picture it was breaking every catalogue run.

Long story short, I realigned puppet and hiera versions between clients and server (now puppet 3.4.3 and hiera 1.3.1). This is a Openstack Foreman issue  BZ#1095985

After some cleaning of the out of sync lib/hiera/backend/module_data_backend.rb file, the python-argparse error has disappeared because the yaml data is provided.

We can close this one.

Thanks James for your patience!

PS: I suppose bug/rhel7-python can be merged

Openstack-Puppet-Module will need the module-data submodule to be created.
It will also depends on BZ#1095985

(In reply to Gilles Dubreuil from comment #29)
> Actually as soon as the module-data gets into the picture it was breaking
> every catalogue run.
> 
> Long story short, I realigned puppet and hiera versions between clients and
> server (now puppet 3.4.3 and hiera 1.3.1). This is a Openstack Foreman issue
> BZ#1095985

This sounds like a good idea :)

> 
> After some cleaning of the out of sync
> lib/hiera/backend/module_data_backend.rb file, the python-argparse error has
> disappeared because the yaml data is provided.

I am not sure what you did, and quite frankly, I am afraid. Can you elaborate?

> 
> We can close this one.

Okay.

> 
> Thanks James for your patience!
yw!

> 
> PS: I suppose bug/rhel7-python can be merged
Okay. Done.
https://github.com/purpleidea/puppet-gluster/commit/34b53c59815d6cd195753c76268bc2ea09517af3

Note: I except there are other values in that data file that should be filled in. Not sure if necessary though.

> 
> Openstack-Puppet-Module will need the module-data submodule to be created.
> It will also depends on BZ#1095985

(In reply to James from comment #30)
> > After some cleaning of the out of sync
> > lib/hiera/backend/module_data_backend.rb file,

Please ignore above, that's irrelevent, I had some trace messages and need the file to be refreshed.

> > the python-argparse error has
> > disappeared because the yaml data is provided.
> 
> I am not sure what you did, and quite frankly, I am afraid. Can you
> elaborate?

${::gluster::params::package_python_argparse} wasn't set up for RHEL7
from
https://github.com/purpleidea/puppet-gluster/blob/master/data/tree/RedHat/RedHat/7.0.yaml#L3
and because of this test
https://github.com/purpleidea/puppet-gluster/blob/master/manifests/xml.pp#L26

The catalogue was tryig to install package python-argparse

The puppet-server error, familiar to puppet-gluster yaml structure, was:
-------
puppet-master[17279]: (Scope(Class[Gluster::Params])) Unable to load yaml data/ directory!
-------

> > PS: I suppose bug/rhel7-python can be merged
> Okay. Done.
> https://github.com/purpleidea/puppet-gluster/commit/
> 34b53c59815d6cd195753c76268bc2ea09517af3
> 

Thanks.

> Note: I except there are other values in that data file that should be
> filled in. Not sure if necessary though.

I suppose this one to be reopened if needed

(In reply to Gilles Dubreuil from comment #31)
> (In reply to James from comment #30)
> > > After some cleaning of the out of sync
> > > lib/hiera/backend/module_data_backend.rb file,
> 
> Please ignore above, that's irrelevent, I had some trace messages and need
> the file to be refreshed.
> 
> > > the python-argparse error has
> > > disappeared because the yaml data is provided.
> > 
> > I am not sure what you did, and quite frankly, I am afraid. Can you
> > elaborate?
> 
> ${::gluster::params::package_python_argparse} wasn't set up for RHEL7
> from
> https://github.com/purpleidea/puppet-gluster/blob/master/data/tree/RedHat/
> RedHat/7.0.yaml#L3
> and because of this test
> https://github.com/purpleidea/puppet-gluster/blob/master/manifests/xml.pp#L26
> 
> The catalogue was tryig to install package python-argparse
> 
> The puppet-server error, familiar to puppet-gluster yaml structure, was:
> -------
> puppet-master[17279]: (Scope(Class[Gluster::Params])) Unable to load yaml
> data/ directory!

lol... that's *my* warning message... I added it. I asked you to look for that the other day...

> -------
> 
> > > PS: I suppose bug/rhel7-python can be merged
> > Okay. Done.
> > https://github.com/purpleidea/puppet-gluster/commit/
> > 34b53c59815d6cd195753c76268bc2ea09517af3
> > 
> 
> Thanks.

yw

> 
> > Note: I except there are other values in that data file that should be
> > filled in. Not sure if necessary though.
> 
> I suppose this one to be reopened if needed

Nope. It's perma-closed now. If you have new issues, open new tickets :)