Bug 1725146

Summary: [OVN] enable deployments with external loadbalancers
Product: Red Hat OpenStack Reporter: Luca Miccini <lmiccini>
Component: openstack-tripleo-heat-templatesAssignee: Kamil Sambor <ksambor>
Status: CLOSED ERRATA QA Contact: Roman Safronov <rsafrono>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 16.1 (Train)CC: dalvarez, ekuris, jiyan, jjoyce, ksambor, mburns, michele, tvignaud
Target Milestone: betaKeywords: Triaged, ZStream
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: All   
OS: Linux   
Whiteboard: Triaged,
Fixed In Version: openstack-tripleo-heat-templates-11.3.2-0.20200514233417.d185f39.el8ost puppet-tripleo-11.4.1-0.20200515103420.8cf3793.el8ost puppet-tripleo-11.4.1-0.20200523013426.658956d.el8ost.noarch.rpm Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-29 07:49:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Luca Miccini 2019-06-28 14:06:24 UTC
Description of problem:

Deployments using an external loadbalancer fail during ovn_dbs_init_bundle because of the following error:

        "stderr: + STEP=3",
        "+ TAGS=file,file_line,concat,augeas,pacemaker::resource::bundle,pacemaker::property,pacemaker::resource::ip,pacemaker::resource::ocf,pacemaker::constraint::order,pacemaker::constraint::colocation",
        "+ CONFIG='include ::tripleo::profile::base::pacemaker;include ::tripleo::profile::pacemaker::ovn_dbs_bundle'",
        "+ EXTRA_ARGS=--debug",
        "+ '[' -d /tmp/puppet-etc ']'",
        "+ cp -a /tmp/puppet-etc/auth.conf /tmp/puppet-etc/hiera.yaml /tmp/puppet-etc/hieradata /tmp/puppet-etc/modules /tmp/puppet-etc/puppet.conf /tmp/puppet-etc/ssl /etc/puppet",
        "+ echo '{\"step\": 3}'",
        "+ export FACTER_deployment_type=containers",
        "+ FACTER_deployment_type=containers",
        "+ set +e",
        "+ puppet apply --debug --verbose --detailed-exitcodes --summarize --color=false --modulepath /etc/puppet/modules:/opt/stack/puppet-modules:/usr/share/openstack-puppet/modules --tags file,file_line,concat,augeas,pacemaker::resource::bundle,pacemaker::property,pacemaker::resource::ip,pacemaker::resource::ocf,pacemaker::constraint::order,pacemaker::constraint::colocation -e 'noop_resource('\\''package'\\''); include ::tripleo::profile::base::pacemaker;include ::tripleo::profile::pacemaker::ovn_dbs_bundle'",
        "Warning: ModuleLoader: module 'tripleo' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules\\n   (file & line not available)",
        "Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5",
        "   (file: /etc/puppet/hiera.yaml)",
        "Warning: Undefined variable '::deploy_config_name'; \\n   (file & line not available)",
        "Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/5.5/deprecated_language.html\\n   (file & line not available)",
        "Warning: ModuleLoader: module 'openstacklib' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules\\n   (file & line not available)",
        "Error: pcs -f /var/lib/pacemaker/cib/puppet-cib-backup20190628-10-15y93xa create constraint failed: Error: Resource 'ip-172.17.1.250' does not exist",
        "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Ovn_dbs_bundle/Pacemaker::Constraint::Colocation[ip-172.17.1.250-with-ovndb_servers]/Pcmk_constraint[colo-ip-172.17.1.250-ovn-dbs-bundle]/ensure: change from 'absent' to 'present' failed: pcs -f /var/lib/pacemaker/cib/puppet-cib-backup20190628-10-15y93xa create constraint failed: Error: Resource 'ip-172.17.1.250' does not exist",
        "+ rc=6",


So pacemaker tries to set up a collocation constraint with a VIP that doesn't exist on any of the controllers.

I've been pointed to:

https://github.com/openvswitch/ovs/commit/33f9c9eafac30d49b4583e93f789a57dc3844523

where a "listen_on_master_ip_only" has been introduced - it would be great to have this exposed in THT/puppet so to unblock suck scenarios.

Version-Release number of selected component (if applicable):

openstack-tripleo-heat-templates-10.5.1-0.20190627050406.22ee26f.el8ost.noarch

Comment 1 Daniel Alvarez Sanchez 2019-06-28 14:23:24 UTC
I believe that the goal is to not create the IPAddr2 resource nor colocation constraint [0].
Instead, based on some parameter that we will drive from THT, we'd be doing what's described here [1].

[0] https://opendev.org/openstack/puppet-tripleo/src/tag/11.0.0/manifests/profile/pacemaker/ovn_dbs_bundle.pp#L155 
[1] https://github.com/openvswitch/ovs/commit/33f9c9eafac30d49b4583e93f789a57dc3844523#diff-fb92408f16ddd415d9906e3976e60f47R263

Comment 2 Michele Baldessari 2019-07-03 13:22:53 UTC
Likely incomplete and untested, but a potential start: https://review.opendev.org/668902

Comment 5 Michele Baldessari 2020-05-13 14:07:14 UTC
So we are missing backports for the THT bits for this to work. I just proposed them now:
https://review.opendev.org/727770 Enable external LB support with ovn   
https://review.opendev.org/727771 Fix listen_on_master_ip_only        

Am testing this, moving back to POST in the meantime

Comment 16 Roman Safronov 2020-06-08 14:15:30 UTC
Was verified on RHOS-16.1-RHEL-8-20200604.n.1 which uses the following packages:
openstack-tripleo-heat-templates-11.3.2-0.20200603183438.7b2c249.el8ost.noarch.rpm
puppet-tripleo-11.5.0-0.20200603223432.9aedd84.el8ost.noarch.rpm

Comment 18 Alex McLeod 2020-06-16 12:30:59 UTC
If this bug requires doc text for errata release, please set the 'Doc Type' and provide draft text according to the template in the 'Doc Text' field. The documentation team will review, edit, and approve the text.

If this bug does not require doc text, please set the 'requires_doc_text' flag to '-'.

Comment 20 errata-xmlrpc 2020-07-29 07:49:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3148