Bug 1247684 - [Dell RHOS 7.0 Bugs] Puppet Error 400 for cinder dell_sc multibackend
[Dell RHOS 7.0 Bugs] Puppet Error 400 for cinder dell_sc multibackend
Status: CLOSED CURRENTRELEASE
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-foreman-installer (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 7.0 (Kilo)
Assigned To: Jason Guiditta
yeylon@redhat.com
: OtherQA
Depends On: 1240362
Blocks: 1172300 dellosp7.0bugs
  Show dependency treegraph
 
Reported: 2015-07-28 11:35 EDT by Rajini Karthik
Modified: 2016-04-18 02:51 EDT (History)
21 users (show)

See Also:
Fixed In Version: openstack-foreman-installer-4.0.5-1.el7ost
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1240362
Environment:
Last Closed: 2015-08-06 21:15:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Rajini Karthik 2015-07-28 11:35:46 EDT
Description of problem:
When you make multibackend true and enable dell_sc backend, puppet fails
Seems like there is problem with the puppet script in the multibackend code path

Version-Release number of selected component (if applicable):

How reproducible: Always

Steps to Reproduce:
Select multiple_backends= true and 
backend_dell_sc = true 

Run puppet agent on the controller nodes and the error pops up right away

Actual results:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type quickstack::dellsc::volume at /usr/share/openstack-foreman-installer/puppet/modules/quickstack/manifests/cinder_volume.pp:269 on node r11rh7cntl3.r11linux.rcbd.lab
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run


Expected results:
Puppet run successful and cinde confirgured for multibackend /etc/cinder/cinder.conf

Additional info:
Comment 3 Rajini Karthik 2015-07-28 11:36:30 EDT
This is a blocker
Comment 5 Jason Guiditta 2015-07-28 12:11:21 EDT
(In reply to Rajini Ram from comment #0)
> Description of problem:
> When you make multibackend true and enable dell_sc backend, puppet fails
> Seems like there is problem with the puppet script in the multibackend code
> path
> 
> Version-Release number of selected component (if applicable):
> 
> How reproducible: Always
> 
> Steps to Reproduce:
> Select multiple_backends= true and 
> backend_dell_sc = true 
> 
> Run puppet agent on the controller nodes and the error pops up right away
> 
> Actual results:
> Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
> Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid
> resource type quickstack::dellsc::volume at
> /usr/share/openstack-foreman-installer/puppet/modules/quickstack/manifests/
> cinder_volume.pp:269 on node r11rh7cntl3.r11linux.rcbd.lab
> Warning: Not using cache on failed catalog
> Error: Could not retrieve catalog; skipping run
> 
> 
> Expected results:
> Puppet run successful and cinde confirgured for multibackend
> /etc/cinder/cinder.conf
> 
> Additional info:

This is a pretty simple fix.  Looking at your patch, you can see that here:
https://github.com/redhat-openstack/astapor/commit/860e4d0995abce763968b4c9fa86a2eb0fe353d9#diff-d7deb803d0403ae7d397a87550c2284fR256
The define is called via 'quickstack::dellsc::volume { $last:'

However, further down, your calls, which exists in the file quickstack::dellsc::volume, does not match that in its define name (which it has to).  The define is actually called 'define quickstack::dell_sc_iscsi::volume'[1].  This is incorrect.  I don't care which you would like to call, but the file name and the define name need to match, or it will never work.  This was an oversight when we reviewed the patch, we usually catch these kinds of inconsistencies.  So 1 of 2 things can be done:
1. Change the directory structure to be correct to match your file (which would mean manifests/dell_sc_iscsi/volume.pp)
2. Change the name of the define to match how it is called (making it 'define quickstack::dellsc::volume)

Both are valid options, if you'd like me to make the change for you, just let me know which path you prefer.  If you would rather test and submit a patch yourself, that is fine as well, it can be merged quickly and included in our next build.

[1] https://github.com/redhat-openstack/astapor/commit/860e4d0995abce763968b4c9fa86a2eb0fe353d9#diff-c4e25734a5d439e93ce1e24451146407R1
Comment 7 Rajini Karthik 2015-07-29 10:41:32 EDT
Working on the fix. I changed the name of the define to match the directory structure. There are couple of more issues that I found. Will test it and submit a PR soon
Comment 8 Jason Guiditta 2015-07-29 11:06:46 EDT
In the interest of trying to get this fixed in time for our next release, I have posted a patch implementing #2 above, on comment #5:
https://github.com/redhat-openstack/astapor/pull/553
Comment 9 Jason Guiditta 2015-07-29 11:07:47 EDT
(In reply to Rajini Ram from comment #7)
> Working on the fix. I changed the name of the define to match the directory
> structure. There are couple of more issues that I found. Will test it and
> submit a PR soon

If your patch supersedes mine, we will take that and I will close mine, leaving it for the moment until that is sorted.
Comment 10 Rajini Karthik 2015-07-29 11:11:06 EDT
I just posted one
https://github.com/redhat-openstack/astapor/pull/554
There is a couple of places where the directory structure is not right. Please review.

I'm still testing this, I noticed there is a new error Cinder:Type[0] not found. I think this is due to the multi_volume_types code.  Will be a couple of hourse before I could reproduce this new error. Any idea?
Comment 11 Jason Guiditta 2015-07-29 14:05:58 EDT
Do you have the puppet error?  That would make it easier to see what the context is here. multi_instance_type.pp does indeed call cinder::type, but I don't see anywhere that the name is passed in as '0'.
Comment 12 Rajini Karthik 2015-07-29 14:34:34 EDT
My stamp is still installing. Will update the actual bug soon
Comment 13 Rajini Karthik 2015-07-29 18:57:15 EDT
Error: Could not retrieve catalog from remote server: Could not intern from text/pson: Could not intern from data: Could not find relationship target "Cinder::Type[]"
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Comment 14 Jason Guiditta 2015-07-30 08:36:45 EDT
this looks to me like bad data, or perhaps the puppet code not quite evaluating something the way you expect in the code.  If you can give me sample config values for the sc configuration (best would be your host's yaml from foreman, so I can see exactly how you are attempting to configure this), then I can try to replicate your error, even though I wont be able to make it all the way to a full install.
Comment 15 Rajini Karthik 2015-07-30 12:46:27 EDT
quickstack::pacemaker::cinder::multiple_backends: 'true'
quickstack::pacemaker::cinder::backend_dell_sc: 'true'
quickstack::pacemaker::cinder::backend_dell_sc_name: ['dell_sc_backend']
quickstack::pacemaker::cinder::dell_sc_san_ip: ['192.168.0.10']
quickstack::pacemaker::cinder::dell_sc_san_login: ['Admin']
quickstack::pacemaker::cinder::dell_sc_san_password: ['password']
quickstack::pacemaker::cinder::dell_sc_iscsi_ip_address: ['192.168.0.20']
quickstack::pacemaker::cinder::dell_sc_iscsi_port: ['3260']
quickstack::pacemaker::cinder::dell_sc_ssn: ['64702']
quickstack::pacemaker::cinder::dell_sc_api_port: ['3033']
quickstack::pacemaker::cinder::dell_sc_server_folder: ['server']
quickstack::pacemaker::cinder::dell_sc_volume_folder: ['volume']


Will this help?
Comment 16 Rajini Karthik 2015-07-30 13:04:13 EDT
Something to do with this?
puppet/modules/quickstack/manifests/cinder_volume_types.pp:    quickstack::cinder::multi_instance_type { "dell-sc-${dell_sc_last_index}":
Comment 17 Jason Guiditta 2015-07-30 14:36:03 EDT
Ok, got it.  In quickstack::cinder_volume_types, on line 39, it has:
$dell_sc_last_index = size($backend_dell_sc) - 1

If I change that to:
$dell_sc_last_index = size($backend_dell_sc_name) - 1

(which matches other types) then the error goes away for me.
Comment 18 Rajini Karthik 2015-07-30 15:23:42 EDT
Updated https://github.com/redhat-openstack/astapor/pull/554

Tested that cinder.conf is generated successfully
Comment 19 Jason Guiditta 2015-07-30 16:01:21 EDT
Patch merged
Comment 20 Rajini Karthik 2015-07-30 16:08:13 EDT
 jason,
One odd thing I noticed netapp is add to the list but backend_netapp=false

# A list of backend names to use. These backend names should
# be backed by a unique [CONFIG] group with its options (list
# value)
#enabled_backends=<None>
enabled_backends=dell_sc1,netapp,rbd
Comment 21 Jason Guiditta 2015-07-30 16:23:40 EDT
I see that too, but if this does not cause any error in function, I would propose we look to clean it up in the next release.  I have built with your last fix already.  If this turns out to be a problem, we can moves this back to assigned and figure out why this is happening.
Comment 22 Rajini Karthik 2015-07-30 16:52:35 EDT
Cleaned up the multibackend netapp code section.
https://github.com/redhat-openstack/astapor/pull/555

It is causing errors in cinder volume logs and cinder volume service is starting up etc
Comment 23 Jason Guiditta 2015-08-03 15:45:07 EDT
Second patch merged, to go into next build
Comment 24 Rajini Karthik 2015-08-10 13:05:50 EDT
Verified fix

Note You need to log in before you can comment on or make changes to this bug.