Bug 1337335 - Hiera >= 2.x packaging
Summary: Hiera >= 2.x packaging
Status: CLOSED UPSTREAM
Alias: None
Product: RDO
Classification: Community
Component: distribution
Version: trunk
Hardware: All
OS: Linux
urgent
urgent
Target Milestone: ---
: ---
Assignee: Alan Pevec
QA Contact: Shai Revivo
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-18 21:23 UTC by Emilien Macchi
Modified: 2017-06-29 12:32 UTC (History)
10 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2017-06-29 12:32:53 UTC


Attachments (Terms of Use)

Description Emilien Macchi 2016-05-18 21:23:05 UTC
In TripleO, and OSP Director, we start to have the need of Hiera 2.x packaging (centos7 currently provides Hiera 1.x).
Indeed, some features in Hiera 2.x would be very useful for us to improve our installer (like looking up in an hash to get the value of a key).

I don't know where to post this RFE so I started here, feel free to move it to the right place.

Comment 1 Alan Pevec 2016-05-18 21:32:06 UTC
Not sure how backward compatible hiera 2.x is but 3.0 update did break OpenStack a lot, see bug 1286826

Comment 2 Mike Burns 2016-05-18 21:35:29 UTC
This should be an RDO bug, then cloned into OSP.  moving back to RDO

Comment 3 Giulio Fidente 2016-05-18 21:41:44 UTC
Sample use case for [1] in TripleO [2]

1. https://docs.puppet.com/hiera/2.0/lookup_types.html#lookup-keys
2. https://review.openstack.org/#/c/318344/

Comment 4 Alan Pevec 2016-05-18 21:53:36 UTC
Random facts collection:

* 2.x was short lived: March - June last year, then 3.0.1 was shipped
* 3.0.1 is known to break Packstack and TripleO,
* latest is 3.1.2 which was not tested, let's try that first and see if and what breaks in Packstack and TripleO

Comment 5 Alan Pevec 2016-05-20 16:36:55 UTC
hiera-3.1.2 update is available for testing from a temporary repo:
http://cbs.centos.org/repos/cloud7-openstack-common-pending/x86_64/os/

Comment 7 Giulio Fidente 2016-05-23 15:16:31 UTC
Build in comment #5 worked fine for me on CentOS release 7.2.1511 and tripleo-heat-templates from master so I think it's good.

Comment 8 Alan Pevec 2016-05-23 16:18:25 UTC
Thanks for testing!
hiera-3.1.2-1.el7 has been tagged into cloud7-openstack-common-testing for wider testing across all RDO release in all testing repos for all RDO releases ( http://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-*/ )

Comment 9 John Trowbridge 2016-05-23 18:01:35 UTC
This is definitely breaking tripleo. hiera lookups fail when installing the undercloud:

2016-05-23 16:50:49 - Error: Could not find data item service_certificate in any Hiera data file and no default supplied at /etc/puppet/manifests/puppet-stack-config.pp:211 on node undercloud
2016-05-23 16:50:49 - Error: Could not find data item service_certificate in any Hiera data file and no default supplied at /etc/puppet/manifests/puppet-stack-config.pp:211 on node undercloud
+ rc=1
+ set -e
+ echo 'puppet apply exited with exit code 1'
puppet apply exited with exit code 1
+ '[' 1 '!=' 2 -a 1 '!=' 0 ']'
+ exit 1
[2016-05-23 16:50:49,523] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1]
 
[2016-05-23 16:50:49,523] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1037, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 915, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 450, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1

Downgrading the hiera package resolves the issue.

Comment 10 John Trowbridge 2016-05-23 18:19:40 UTC
Giulio was able to track down the issue to the new hiera not liking nil values.

If I set undercloud_service_certificate = "''" in undercloud.conf, then the service_certificate key in hiera is populated with '' and the undercloud install succeeds.

We do have other cases in TripleO though where nil values end up in hiera. As such, I think it is still best to revert this, and make sure we have resolved those issues before updating.

Comment 11 Giulio Fidente 2016-05-23 18:47:18 UTC
hi John, thanks.

Still investigating on the root cause but the issue in comment #9 seems an instance of a more generic problem with the default values given to hiera() calls, specifically with 3.1 the following:

  puppet apply -e 'notice(hiera("service_certificate", "some_default"))'

does not seem to work as expected, giving:

  Error: Could not find data item service_certificate in any Hiera data file and no default supplied at line 1 on node instack

while it is instead supposed to return "some_default".

Comment 12 Sai Sindhur Malleni 2016-05-23 19:11:40 UTC
So is the recommended method to proceed with installation now, to set undercloud_service_certificate = "''" or to downgrade hiera?

Comment 13 Steven Hardy 2016-05-23 20:30:46 UTC
I think downgrade is the best short-term option pending further investigation?

I posted this which (if I have my exclude syntax correct?) should exclude just the hiera-3* version and fall back to the old version for tripleo-ci:

https://review.openstack.org/#/c/320127/

Comment 14 Alan Pevec 2016-05-23 21:07:52 UTC
http://cbs.centos.org/koji/buildinfo?buildID=11136 hiera 1.3.4 with bumped Epoch:1 was pushed to the common-testing.

Comment 15 Alan Pevec 2016-05-24 09:09:45 UTC
Available in testing repo used by CI jobs:
http://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-mitaka/common/hiera-1.3.4-5.el7.noarch.rpm

Comment 16 Alan Pevec 2016-05-24 09:11:40 UTC
@EmilienM - original issue is back on hold, until nil values issue is resolved in TripleO.

Comment 17 Emilien Macchi 2016-05-24 16:12:41 UTC
It seems like we're not alone,

See:
https://projects.puppetlabs.com/issues/20923
https://github.com/camptocamp/puppet-hiera-undef

I'm wondering if a recent version of Puppet would help us.
I'll take some time to test it.

Comment 18 Christopher Brown 2017-02-15 17:24:35 UTC
I guess this can be closed now. Not an issue for me any longer.


Note You need to log in before you can comment on or make changes to this bug.