| Summary: | rhosp-director: Error: Evaluation Error: Illegal relationship operand, can not form a relationship with an Undef Value. A Catalog type is required. at /etc/puppet/modules/pacem aker/manifests/remote.pp:31:6 on node overcloud-networker-0.localdomain | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Alexander Chuzhoy <sasha> |
| Component: | puppet-pacemaker | Assignee: | Michele Baldessari <michele> |
| Status: | CLOSED ERRATA | QA Contact: | pkomarov |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 12.0 (Pike) | CC: | aherr, dbecker, dprince, jjoyce, jschluet, mburns, michele, morazi, rhel-osp-director-maint, slinaber, tvignaud, ushkalim |
| Target Milestone: | rc | Keywords: | Triaged |
| Target Release: | 12.0 (Pike) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | puppet-pacemaker-0.6.1-0.20171015172907.44d3226.el7ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-12-13 22:11:04 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
Ok so there are a couple of things here: 1) OS::TripleO::Services::PacemakerRemote probably does not make any sense in the Networker role anyway as I cannot see any services managed by pacemaker. *But* that is not the real problem here. 2) The real problem is that https://review.openstack.org/#/c/503077/ introduced ensure_resource() with a relationship and that will barf out like this: Error: Evaluation Error: Illegal relationship operand, can not form a relationship with an Undef Value. A Catalog type is required. at /etc/puppet/modules/pacemaker/manifests/remote.pp:31:6 on node nuc2.int.rhx Am not entirely sure as to why it was introduced because remote.pp and pacemaker::corosync are not meant to be included in the same node anyway? I'll take this one and work on a fix but I'd love to hear as to why we moved to ensure_resource() ? Change has merged upstream and there are no branches for this module Successfully deployed OC with composable roles after applying the attached patch to OC image. Martin explained to me why the assumption that remote and corosync were on the same node. It was mainly wrong thinking on a CI patch. So removing needinfo here. Verified. HA composable roles deployments are passing. https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/pidone/view/features/job/DFG-pidone-features-12_director-rhel-virthost-3cont_3data_3mess_1comp_1ceph-ipv4-vxlan-composable_roles/ Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3462 |
rhosp-director: Error: Evaluation Error: Illegal relationship operand, can not form a relationship with an Undef Value. A Catalog type is required. at /etc/puppet/modules/pacem aker/manifests/remote.pp:31:6 on node overcloud-networker-0.localdomain Environment: openstack-tripleo-heat-templates-7.0.0-0.20170913050524.0rc2.el7ost.noarch openstack-puppet-modules-11.0.0-0.20170828113154.el7ost.noarch instack-undercloud-7.4.1-0.20170912115418.el7ost.noarch Steps to reproduce: Try to deploy overcloud with: openstack overcloud deploy \ --templates /usr/share/openstack-tripleo-heat-templates \ --libvirt-type kvm \ --ntp-server clock.redhat.com \ -r /home/stack/roles_data.yaml \ -e /home/stack/virt/internal.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e /home/stack/virt/docker-images.yaml \ -e /home/stack/virt/docker-images-ceph.yaml [root@undercloud-0 ~]# cat /home/stack/roles_data.yaml ############################################################################### # File generated by TripleO ############################################################################### ############################################################################### # Role: ControllerOpenstack # ############################################################################### - name: Controller description: | Controller role that does not contain the database, messaging and networking components. Use in combination with the Database, Messaging and Networker roles. tags: - primary - controller networks: - External - InternalApi - Storage - StorageMgmt - Tenant HostnameFormatDefault: '%stackname%-controller-%index%' ServicesDefault: - OS::TripleO::Services::AodhApi - OS::TripleO::Services::AodhEvaluator - OS::TripleO::Services::AodhListener - OS::TripleO::Services::AodhNotifier - OS::TripleO::Services::AuditD - OS::TripleO::Services::BarbicanApi - OS::TripleO::Services::CACerts - OS::TripleO::Services::CeilometerAgentCentral - OS::TripleO::Services::CeilometerAgentNotification - OS::TripleO::Services::CeilometerApi - OS::TripleO::Services::CeilometerCollector - OS::TripleO::Services::CeilometerExpirer - OS::TripleO::Services::CephExternal - OS::TripleO::Services::CephMds - OS::TripleO::Services::CephMon - OS::TripleO::Services::CephRbdMirror - OS::TripleO::Services::CephRgw - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::CinderApi - OS::TripleO::Services::CinderBackup - OS::TripleO::Services::CinderHPELeftHandISCSI - OS::TripleO::Services::CinderScheduler - OS::TripleO::Services::CinderVolume - OS::TripleO::Services::Collectd - OS::TripleO::Services::Congress - OS::TripleO::Services::Docker - OS::TripleO::Services::Ec2Api - OS::TripleO::Services::Etcd - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::GlanceApi - OS::TripleO::Services::GnocchiApi - OS::TripleO::Services::GnocchiMetricd - OS::TripleO::Services::GnocchiStatsd - OS::TripleO::Services::HAproxy - OS::TripleO::Services::HeatApi - OS::TripleO::Services::HeatApiCfn - OS::TripleO::Services::HeatApiCloudwatch - OS::TripleO::Services::HeatEngine - OS::TripleO::Services::Horizon - OS::TripleO::Services::IronicApi - OS::TripleO::Services::IronicConductor - OS::TripleO::Services::Iscsid - OS::TripleO::Services::Keepalived - OS::TripleO::Services::Kernel - OS::TripleO::Services::Keystone - OS::TripleO::Services::ManilaApi - OS::TripleO::Services::ManilaBackendCephFs - OS::TripleO::Services::ManilaBackendGeneric - OS::TripleO::Services::ManilaBackendIsilon - OS::TripleO::Services::ManilaBackendNetapp - OS::TripleO::Services::ManilaBackendUnity - OS::TripleO::Services::ManilaBackendVNX - OS::TripleO::Services::ManilaBackendVMAX - OS::TripleO::Services::ManilaScheduler - OS::TripleO::Services::ManilaShare - OS::TripleO::Services::Memcached - OS::TripleO::Services::MongoDb - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::NovaApi - OS::TripleO::Services::NovaConductor - OS::TripleO::Services::NovaConsoleauth - OS::TripleO::Services::NovaIronic - OS::TripleO::Services::NovaMetadata - OS::TripleO::Services::NovaPlacement - OS::TripleO::Services::NovaScheduler - OS::TripleO::Services::NovaVncProxy - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::OctaviaApi - OS::TripleO::Services::OctaviaHealthManager - OS::TripleO::Services::OctaviaHousekeeping - OS::TripleO::Services::OctaviaWorker - OS::TripleO::Services::OpenDaylightApi - OS::TripleO::Services::OpenDaylightOvs - OS::TripleO::Services::OVNDBs - OS::TripleO::Services::OVNController - OS::TripleO::Services::Pacemaker - OS::TripleO::Services::PankoApi - OS::TripleO::Services::Redis - OS::TripleO::Services::SaharaApi - OS::TripleO::Services::SaharaEngine - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Sshd - OS::TripleO::Services::SwiftProxy - OS::TripleO::Services::SwiftRingBuilder - OS::TripleO::Services::SwiftStorage - OS::TripleO::Services::Tacker - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned - OS::TripleO::Services::Vpp - OS::TripleO::Services::Zaqar ############################################################################### # Role: Database # ############################################################################### - name: Database description: | Standalone database role with the database being managed via Pacemaker networks: - InternalApi HostnameFormatDefault: '%stackname%-database-%index%' ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Clustercheck - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::MySQL - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::Pacemaker - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned ############################################################################### # Role: Messaging # ############################################################################### - name: Messaging description: | Standalone messaging role with RabbitMQ being managed via Pacemaker networks: - InternalApi HostnameFormatDefault: '%stackname%-messaging-%index%' ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::Pacemaker - OS::TripleO::Services::RabbitMQ - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned ############################################################################### # Role: Networker # ############################################################################### - name: Networker description: | Standalone networking role to run Neutron services their own. Includes Pacemaker integration via PacemakerRemote networks: - InternalApi - External HostnameFormatDefault: '%stackname%-networker-%index%' ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::NeutronApi - OS::TripleO::Services::NeutronBgpVpnApi - OS::TripleO::Services::NeutronCorePlugin - OS::TripleO::Services::NeutronDhcpAgent - OS::TripleO::Services::NeutronL2gwAgent - OS::TripleO::Services::NeutronL2gwApi - OS::TripleO::Services::NeutronL3Agent - OS::TripleO::Services::NeutronLbaasv2Agent - OS::TripleO::Services::NeutronMetadataAgent - OS::TripleO::Services::NeutronML2FujitsuCfab - OS::TripleO::Services::NeutronML2FujitsuFossw - OS::TripleO::Services::NeutronOvsAgent - OS::TripleO::Services::NeutronVppAgent - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::OpenDaylightOvs - OS::TripleO::Services::PacemakerRemote - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned ############################################################################### # Role: Compute # ############################################################################### - name: Compute description: | Basic Compute Node role CountDefault: 1 networks: - InternalApi - Tenant - Storage HostnameFormatDefault: '%stackname%-novacompute-%index%' # Deprecated & backward-compatible values (FIXME: Make parameters consistent) # Set uses_deprecated_params to True if any deprecated params are used. uses_deprecated_params: True deprecated_param_image: 'NovaImage' deprecated_param_extraconfig: 'NovaComputeExtraConfig' deprecated_param_metadata: 'NovaComputeServerMetadata' deprecated_param_scheduler_hints: 'NovaComputeSchedulerHints' deprecated_param_ips: 'NovaComputeIPs' deprecated_server_resource_name: 'NovaCompute' disable_upgrade_deployment: True ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CephClient - OS::TripleO::Services::CephExternal - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::ComputeCeilometerAgent - OS::TripleO::Services::ComputeNeutronCorePlugin - OS::TripleO::Services::ComputeNeutronL3Agent - OS::TripleO::Services::ComputeNeutronMetadataAgent - OS::TripleO::Services::ComputeNeutronOvsAgent - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Iscsid - OS::TripleO::Services::Kernel - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::NeutronLinuxbridgeAgent - OS::TripleO::Services::NeutronSriovAgent - OS::TripleO::Services::NeutronSriovHostConfig - OS::TripleO::Services::NeutronVppAgent - OS::TripleO::Services::NovaCompute - OS::TripleO::Services::NovaLibvirt - OS::TripleO::Services::NovaMigrationTarget - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::OpenDaylightOvs - OS::TripleO::Services::Securetty - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Sshd - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned - OS::TripleO::Services::Vpp - OS::TripleO::Services::OVNController ############################################################################### # Role: CephStorage # ############################################################################### - name: CephStorage description: | Ceph OSD Storage node role networks: - Storage - StorageMgmt ServicesDefault: - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CephOSD - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::MySQLClient - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::Securetty - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Sshd - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned Result: The deployment fails. Trying to debug it with heat, see this error: "status": "FAILED", "server_id": "d5512200-4c82-4a14-b744-8505bf29cc45", "config_id": "f6b4feb5-d729-4132-9b9c-985f02c81b11", "output_values": { "deploy_stdout": "\nPLAY [localhost] ***************************************************************\n\nTASK [Gathering Facts] *********************************************************\n ok: [localhost]\n\nTASK [Write the config_step hieradata] *****************************************\nchanged: [localhost]\n\nTASK [Run puppet host configuration for step 1] ***************** ***************\nok: [localhost]\n\nTASK [debug] *******************************************************************\nfatal: [localhost]: FAILED! => {\n \"(outputs.stderr|default('')).spl it('\\n')|union(outputs.stdout_lines|default([]))\": [\n \"Warning: Undefined variable 'deploy_config_name'; \", \n \" (file & line not available)\", \n \"Warning: Thi s method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Bool. There is further documentation for validate_legacy function in the README. at [\\\"/etc/pup pet/modules/ntp/manifests/init.pp\\\", 54]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\\\", 29]\", \n \" (at /etc/puppet/modules/stdlib/lib/puppet/functions/ deprecation.rb:25:in `deprecation')\", \n \"Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Absolute_Path. There is further do cumentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules/ntp/manifests/init.pp\\\", 55]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\\\", 29 ]\", \n \"Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules/ntp/manifests/init.pp\\\", 56]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\\\", 29]\", \n \"Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Array. There is further documentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules /ntp/manifests/init.pp\\\", 66]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\\\", 29]\", \n \"Warning: This method is deprecated, please use the stdlib validate _legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules/ntp/manifests/init.pp\\\", 68]:[\\\"/etc/puppet/modul es/tripleo/manifests/profile/base/time/ntp.pp\\\", 29]\", \n \"Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Numeric. There is further documentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules/ntp/manifests/init.pp\\\", 76]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/n tp.pp\\\", 29]\", \n \"Error: Evaluation Error: Illegal relationship operand, can not form a relationship with an Undef Value. A Catalog type is required. at /etc/puppet/modules/pacem aker/manifests/remote.pp:31:6 on node overcloud-networker-0.localdomain\", \n \"Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend\"\n ], \n \"changed\": false, \n \"failed\": true, \n \"failed_when_result\": true\n}\n\tto retry, use: --limit @/var/lib/heat-config/heat-config-ansible/f6b4feb5-d729-4132-9b9c-985f02c81b11_playbook.retry\n\nPLAY RECAP *********************************************************************\nlocalhost : ok=3 changed=1 unreachable=0 failed=1 \n\n", "deploy_stderr": "", "deploy_status_code": 2 Grepping /var/log/messages for errors on networker node: [root@overcloud-networker-0 ~]# grep -i error /var/log/messages Sep 20 14:14:09 localhost os-collect-config: [2017-09-20 14:14:09,486] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:14:46 localhost os-collect-config: [2017-09-20 14:14:46,470] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:15:26 localhost os-collect-config: [2017-09-20 14:15:26,291] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:16:04 localhost os-collect-config: [2017-09-20 14:16:04,318] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:20:12 localhost os-collect-config: [2017-09-20 14:20:12,360] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:21:20 localhost os-collect-config: [2017-09-20 14:21:20,719] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:21:59 localhost os-collect-config: [2017-09-20 14:21:59,020] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:22:37 localhost os-collect-config: [2017-09-20 14:22:37,066] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:23:22 localhost os-collect-config: [2017-09-20 14:23:22,647] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:23:58 localhost os-collect-config: [2017-09-20 14:23:58,840] (heat-config) [ERROR] Skipping group os-apply-config with no hook script None Sep 20 14:24:12 localhost puppet-user[15353]: Evaluation Error: Illegal relationship operand, can not form a relationship with an Undef Value. A Catalog type is required. at /etc/puppet/modules/pacemaker/manifests/remote.pp:31:6 on node overcloud-networker-0.localdomain Sep 20 14:24:12 localhost os-collect-config: documentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules/ntp/manifests/init.pp\\\", 66]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\\\", 29]\", \n \"Warning: This method is deprecated, please use the stdlib validate_legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules/ntp/manifests/init.pp\\\", 68]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\\\", 29]\", \n \"Warning: This method is deprecated, please use the stdlib validate_legacy function, with Stdlib::Compat::Numeric. There is further documentation for validate_legacy function in the README. at [\\\"/etc/puppet/modules/ntp/manifests/init.pp\\\", 76]:[\\\"/etc/puppet/modules/tripleo/manifests/profile/base/time/ntp.pp\\\", 29]\", \n \"Error: Evaluation Error: Illegal relationship operand, can not form a relationship with an Undef Value. A Catalog type is required. at /etc/puppet/modules/pacemaker/manifests/remote.pp:31:6 on node overcloud-networker-0.localdomain\", \n \"Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend\"\n ], \n \"changed\": false, \n \"failed\": true, \n \"failed_when_result\": true\n}\n\tto retry, use: --limit @/var/lib/heat-config/heat-config-ansible/f6b4feb5-d729-4132-9b9c-985f02c81b11_playbook.retry\n\nPLAY RECAP *********************************************************************\nlocalhost : ok=3 changed=1 unreachable=0 failed=1 \n\n", "deploy_stderr": "", "deploy_status_code": 2} Sep 20 14:24:12 localhost os-collect-config: "Error: Evaluation Error: Illegal relationship operand, can not form a relationship with an Undef Value. A Catalog type is required. at /etc/puppet/modules/pacemaker/manifests/remote.pp:31:6 on node overcloud-networker-0.localdomain", Sep 20 14:24:12 localhost os-collect-config: [2017-09-20 14:24:12,203] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-ansible/f6b4feb5-d729-4132-9b9c-985f02c81b11_playbook.yaml. [2]