Bug 1775283 - multi cellv2 deployment fail with exported data after refactor in https://review.opendev.org/#/c/694115/
Summary: multi cellv2 deployment fail with exported data after refactor in https://rev...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: tripleo-ansible
Version: 16.0 (Train)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 16.0 (Train on RHEL 8.1)
Assignee: Martin Schuppert
QA Contact: Archit Modi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-21 16:30 UTC by Martin Schuppert
Modified: 2020-02-06 14:43 UTC (History)
6 users (show)

Fixed In Version: tripleo-ansible-0.4.1-0.20191203040321.ccad006.el8ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-02-06 14:42:58 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1853472 0 None None None 2019-11-21 16:30:23 UTC
OpenStack gerrit 695464 0 'None' ABANDONED For cell export add filter parameters to GlobalConfigExtraMapData 2021-01-05 20:05:36 UTC
OpenStack gerrit 695805 0 'None' MERGED Make sure we add all_nodes_extra_map_data to all_nodes 2021-01-05 20:05:36 UTC
Red Hat Product Errata RHEA-2020:0283 0 None None None 2020-02-06 14:43:58 UTC

Description Martin Schuppert 2019-11-21 16:30:23 UTC
Description of problem:
When export cell input data from overcloud stack using:

~~~
(undercloud) $ openstack overcloud cell export cell1 -f -o $DIR/cell1-cell-input.yaml

Cell input information exported to scenario2/cell1/cell1-cell-input.yaml.
~~~

The cell deployment fails with memcached_node_ips, ovn_dbs_vip and redis_vip missing in hieradata of the cell nodes:

        "<13>Nov 21 14:13:46 puppet-user: Error: Function lookup() did not find a value for the name 'memcached_node_ips'",
        "2019-11-21 14:13:49,964 WARNING: 17529 -- Retrying running container: nova_libvirt",
        "2019-11-21 14:13:49,964 ERROR: 17529 -- Failed running container for nova_libvirt",
        "2019-11-21 14:13:49,964 INFO: 17529 -- Finished processing puppet configs for nova_libvirt",
        "2019-11-21 14:13:51,304 ERROR: 17532 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', 'container-puppet-neutron', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,neutron_config,ovn_metadata_agent_config', '--env', 'NAME=neutron', '--env', 'HOSTNAME=cell1-compute-9', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--env', 'DEBUG=False', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmp95d05cyv:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-neutron.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', '/lib/modules:/lib/modules:ro', '--volume', '/run/openvswitch:/run/openvswitch:shared,z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'f04-h17-b01-5039ms.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-neutron-server-ovn:20191112.1'] run failed after + mkdir -p /etc/puppet",
        "<13>Nov 21 14:13:42 puppet-user: Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5",
        "<13>Nov 21 14:13:48 puppet-user: (file: /etc/puppet/hiera.yaml)",
        "<13>Nov 21 14:13:48 puppet-user: Warning: Undefined variable '::deploy_config_name'; \\n (file & line not available)",
        "<13>Nov 21 14:13:48 puppet-user: Warning: ModuleLoader: module 'tripleo' has unresolved dependencies - it will only see those that are resolved. Use
'puppet module list --tree' to see information about modules\\n (file & line not available)",
        "<13>Nov 21 14:13:48 puppet-user: Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/5.5/depreacated_language.html\\n (file & line not available)",
        "<13>Nov 21 14:13:48 puppet-user: Error: Function lookup() did not find a value for the name 'ovn_dbs_vip'",
        "2019-11-21 14:13:51,304 WARNING: 17532 -- Retrying running container: neutron",
        "2019-11-21 14:13:51,304 ERROR: 17532 -- Failed running container for neutron",
        "2019-11-21 14:13:51,304 INFO: 17532 -- Finished processing puppet configs for neutron",
        "2019-11-21 14:13:51,304 ERROR: 17527 -- ERROR configuring nova_libvirt",
        "2019-11-21 14:13:51,304 ERROR: 17527 -- ERROR configuring ovn_controller",
        "2019-11-21 14:13:51,304 ERROR: 17527 -- ERROR configuring neutron"

When comparing export data from before refactor we see the following difference:
(undercloud) [stack@f04-h17-b01-5039ms ~]$ diff -u scenario1/cell3/cell3-cell-input.yaml scenario2/cell1/cell1-cell-input.yaml
--- scenario1/cell3/cell3-cell-input.yaml 2019-11-14 15:52:33.763263726 +0000
+++ scenario2/cell1/cell1-cell-input.yaml 2019-11-21 11:57:03.327008542 +0000
@@ -1,20 +1,27 @@
 parameter_defaults:
   AdminPassword: CghqwfI2U23Ru5xoEqDD1QtqI
   AdminToken: g4cT4pcL58U6EPy4WCumsKia3
+ AllNodesExtraMapData:
+ memcached_node_ips:
+ - 192.168.19.129
+ - 192.168.20.73
+ - 192.168.19.103
+ oslo_messaging_notify_node_names:
+ - controller-0.internalapi.osp16.test
+ - controller-1.internalapi.osp16.test
+ - controller-2.internalapi.osp16.test
+ oslo_messaging_notify_short_bootstrap_node_name: controller-0
+ oslo_messaging_rpc_node_names:
+ - controller-0.internalapi.osp16.test
+ - controller-1.internalapi.osp16.test
+ - controller-2.internalapi.osp16.test
+ ovn_dbs_vip: 192.168.17.73
+ redis_vip: 192.168.17.35
   AodhPassword: e5CKmWxWNvJsIhpVB1Q7LkzuS
   BarbicanPassword: K9NuoGxMF2kqKS8ckzyAkzLYU
   BarbicanSimpleCryptoKek: jWLXQwy6UtBssqQ4WM7-DhNtsn2nyiiLNwFdR1H4CiE=
   CeilometerMeteringSecret: SGlVamiH2ukcy6IN76AOJzpwY
   CeilometerPassword: 4XgvINqAONCMi2HBS4E9mG4th
- CephAdminKey: AQDL1stdAAAAABAAg2J0t79NV8BkkCLholYtdg==
- CephClientKey: AQDL1stdAAAAABAAwHviXOtxTTXsteRZGtSJ1A==
- CephClusterFSID: f209c180-05fd-11ea-a6ab-ac1f6bac47f2
- CephDashboardAdminPassword: AQDL1stdAAAAABAAhNUu++wfCQX6flmn7nbo3w==
- CephGrafanaAdminPassword: AQDL1stdAAAAABAAIdtbSzGd/MFjLvxwfF0VBg==
- CephManilaClientKey: AQDL1stdAAAAABAAE3hnhSMmwXw/tgW8cgJzwg==
- CephMdsKey: AQDL1stdAAAAABAA6OqHGEHmTItE/+FGnsmBcA==
- CephMonKey: AQDL1stdAAAAABAABUjMkmQNgRTsNhRMEP08Ig==
- CephRgwKey: AQDL1stdAAAAABAAnA3rEv6wbf6UJaUOr8BlqQ==
   CinderPassword: gmoMM8WCWWwpinJtGch4grdnl
   CongressPassword: WUHDntrSF6gfyLdMxeirORNvI
   DesignatePassword: vOhoCJtziHVuKCiwoQ4hpu83i
@@ -820,31 +827,16 @@
     192.168.65.1 overcloud.osp16.test'
   GlancePassword: ZFosXqtyzZXMRDO3eKPiR5ObR
   GlobalConfigExtraMapData:
- memcached_node_ips:
- - 192.168.19.129
- - 192.168.20.73
- - 192.168.19.103
- oslo_messaging_notify_node_names:
- - controller-0.internalapi.osp16.test
- - controller-1.internalapi.osp16.test
- - controller-2.internalapi.osp16.test
     oslo_messaging_notify_password: tnPGMmBI8cJdyE3pLOnuEql2t
     oslo_messaging_notify_port: 5672
     oslo_messaging_notify_scheme: rabbit
- oslo_messaging_notify_short_bootstrap_node_name: controller-0
     oslo_messaging_notify_use_ssl: 'False'
     oslo_messaging_notify_user_name: guest
- oslo_messaging_rpc_node_names:
- - controller-0.internalapi.osp16.test
- - controller-1.internalapi.osp16.test
- - controller-2.internalapi.osp16.test
     oslo_messaging_rpc_password: tnPGMmBI8cJdyE3pLOnuEql2t
     oslo_messaging_rpc_port: 5672
     oslo_messaging_rpc_scheme: rabbit
     oslo_messaging_rpc_use_ssl: 'False'
     oslo_messaging_rpc_user_name: guest
- ovn_dbs_vip: 192.168.17.73
- redis_vip: 192.168.17.35
   GnocchiPassword: 6o5MyL0FFRVR5l6rY2Z4ETmR8
   HAProxyStatsPassword: cEOrGrSm53RvAsXVRRTIMxOq8
   HeatAuthEncryptionKey: x2JXsZGOszmfNZm85xdA25q7PXOe4z97

Before the refactor https://review.opendev.org/#/c/694115/ of openstack overcloud cell export we added the information to GlobalConfigExtraMapData, now it is added to AllNodesExtraMapData https://review.opendev.org/#/c/694115/1/tripleoclient/export.py

Version-Release number of selected component (if applicable):
python3-tripleoclient-12.3.1-0.20191115200343.315f6d4.el8ost.noarch.rpm

How reproducible:
always

Steps to Reproduce:
1. openstack overcloud cell export cell1 -f -o $DIR/cell1-cell-input.yaml
2. deploy cell using above export
3.

Actual results:


Expected results:


Additional info:

Comment 8 errata-xmlrpc 2020-02-06 14:42:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:0283


Note You need to log in before you can comment on or make changes to this bug.