Bug 1324932

Created attachment 1144793 [details]
dell-storage-environment.yaml

hi, I am trying to investigate this but it's not clear the purpose of  dell-storage-environment.yaml, you don't need that to test the user enabled backends and it might even collide with it depending on what is in the other custom template.

Can you please try using only dell-cinder-backends.yaml and paste the exact deploy command you're using?

dell-storage-environment.yaml may not be relevant in this context. We have a number of files environment files we use. We don't have a setup to  just deploy with only the dell-cinder-backends.yaml.

Is there anything else that might help to debug the issue?

Can you provide the deploy command line and the contents of any environment yamls you're passing?

Also, if you can get on Freenode IRC and reach out to us on #tripleo or #rdo, we can likely get to the root of this more quickly.

Command that runs the overcloud deploy, will upload the yamls as attachments

016-04-06 15:41:54.696 20221 INFO openstackclient.shell [ admin None] START with options: ['overcloud', 'deploy', '--debug', '--log-file', '/home/osp_admin/pilot/overcloud_deployment.log', '-t', '
380', '--templates', '/home/osp_admin/pilot/templates/overcloud', '-e', '/home/osp_admin/pilot/templates/overcloud/environments/network-isolation.yaml', '-e', '/home/osp_admin/pilot/templates/netwo
rk-environment.yaml', '-e', '/home/osp_admin/pilot/templates/overcloud/environments/storage-environment.yaml', '-e', '/home/osp_admin/pilot/templates/dell-environment.yaml', '-e', '/usr/share/opens
tack-tripleo-heat-templates/environments/puppet-pacemaker.yaml', '-e', '/home/osp_admin/pilot/templates/dell-cinder-backends.yaml', '--control-flavor', 'control', '--compute-flavor', 'compute', '--
ceph-storage-flavor', 'ceph-storage', '--swift-storage-flavor', 'swift-storage', '--block-storage-flavor', 'block-storage', '--neutron-public-interface', 'bond1', '--neutron-network-type', 'vlan', 
'--neutron-disable-tunneling', '--os-auth-url', 'http://192.168.120.178:5000/v2.0', '--os-project-name', 'admin', '--os-user-id', 'admin', '--os-password', 'd24ec4dbe2e7b48ee57497bb74bf0a7152066c42
', '--control-scale', '3', '--compute-scale', '3', '--ceph-storage-scale', '3', '--ntp-server', '0.centos.pool.ntp.org', '--neutron-network-vlan-ranges', 'physint:201:220,physext', '--neutron-bridg
e-mappings', 'physint:br-tenant,physext:br-ex']

Command that runs the overcloud deploy, will upload the yamls as attachments

016-04-06 15:41:54.696 20221 INFO openstackclient.shell [ admin None] START with options: ['overcloud', 'deploy', '--debug', '--log-file', '/home/osp_admin/pilot/overcloud_deployment.log', '-t', '
380', '--templates', '/home/osp_admin/pilot/templates/overcloud', '-e', '/home/osp_admin/pilot/templates/overcloud/environments/network-isolation.yaml', '-e', '/home/osp_admin/pilot/templates/netwo
rk-environment.yaml', '-e', '/home/osp_admin/pilot/templates/overcloud/environments/storage-environment.yaml', '-e', '/home/osp_admin/pilot/templates/dell-environment.yaml', '-e', '/usr/share/opens
tack-tripleo-heat-templates/environments/puppet-pacemaker.yaml', '-e', '/home/osp_admin/pilot/templates/dell-cinder-backends.yaml', '--control-flavor', 'control', '--compute-flavor', 'compute', '--
ceph-storage-flavor', 'ceph-storage', '--swift-storage-flavor', 'swift-storage', '--block-storage-flavor', 'block-storage', '--neutron-public-interface', 'bond1', '--neutron-network-type', 'vlan', 
'--neutron-disable-tunneling', '--os-auth-url', 'http://192.168.120.178:5000/v2.0', '--os-project-name', 'admin', '--os-user-id', 'admin', '--os-password', 'd24ec4dbe2e7b48ee57497bb74bf0a7152066c42
', '--control-scale', '3', '--compute-scale', '3', '--ceph-storage-scale', '3', '--ntp-server', '0.centos.pool.ntp.org', '--neutron-network-vlan-ranges', 'physint:201:220,physext', '--neutron-bridg
e-mappings', 'physint:br-tenant,physext:br-ex']

Created attachment 1144823 [details]
network-isolation.yaml

Created attachment 1144824 [details]
network-environment.yaml

Created attachment 1144825 [details]
storage-environment.yaml

Created attachment 1144826 [details]
network-environment.yaml

Created attachment 1144827 [details]
dell-environment.yaml

The overcloud eventually times out, after being stuck on Ceph deployment steps

Created attachment 1144835 [details]
Diff of templates

diff -rNu /home/osp_admin/pilot/templates/overcloud /usr/share/openstack-tripleo-heat-templates > diff_of_templates

I was able to deploy using both your dell-cinder-backends.yaml and ceph successfully. If there is a problem I think it is elsewhere.

From the diff you attached I see you're editing the ceph hieradata and I think I spotted a syntax error::

-ceph::profile::params::osds:
-  'sdd/dev/':
-    journal: '/dev/sdb'

should it read '/dev/sdd' ?

Also, it is not necessary to edit the hieradata files so you don't need to copy the templates. Instead, you can use the stock templates (switching all references to /usr/share/openstack-tripleo-heat-templates) and use a custom environment file for those parameters which need to overridden, for example:

parameter_defaults:
  ExtraConfig:
    ceph::profile::params::osd_journal_size: 10000
    ceph::profile::params::osd_pool_default_pg_num: 256
    ceph::profile::params::osds:
      '/dev/sdd':
        journal: '/dev/sdb'

I noticed you're using some custom first boot and post deployment script too 

resource_registry:
    OS::TripleO::NodeUserData: /home/osp_admin/pilot/templates/first-boot.yaml
    OS::TripleO::NodeExtraConfigPost: /home/osp_admin/pilot/templates/post-deploy.yaml

can you attach those two files as well?

This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

Created attachment 1144904 [details]
first-boot.yaml

Created attachment 1144905 [details]
post-deploy.yaml

Created attachment 1144906 [details]
diff_templates_without_cinder_backends

Giulio was correct. The disk formats were old. Fixing them and generating the correct ceph.yaml file solved the issue.

Was able to successfully deploy ceph, dell eqlx and dell storage center backends on my Stamp.

QA stamps and FutureVille 2 stamps pending tests

hi, thanks for the update

Unfortunately the syntax error was in a param value, not in the param name, heat does not know what are the allowed values for each and every parameter so validating this before the deployment is quite complex

Waiting on the results from the other environments.

Per comment 20, closing this bug.   If the issue reappears, please reopen.