Bug 1679775
| Summary: | Unable to deploy with Octavia and Provider network - ERROR: The specified reference "subnet" (in api_lb.Properties.vip_subnet) is incorrect | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Chris Janiszewski <cjanisze> | ||||
| Component: | Installer | Assignee: | Tzu-Mainn Chen <tzumainn> | ||||
| Installer sub component: | openshift-ansible | QA Contact: | GenadiC <gcheresh> | ||||
| Status: | CLOSED ERRATA | Docs Contact: | |||||
| Severity: | unspecified | ||||||
| Priority: | unspecified | CC: | bleanhar, cjanisze, gpei, juriarte, ltomasbo, tzumainn | ||||
| Version: | 3.11.0 | Keywords: | UpcomingSprint | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 3.11.z | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2020-09-16 07:46:49 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Chris Janiszewski
2019-02-21 20:23:27 UTC
Hi! We haven't tested this specific configuration, but looking at the relevant heat template code - https://github.com/openshift/openshift-ansible/blob/master/roles/openshift_openstack/templates/heat_stack.yaml.j2#L179-L181 - could you try setting openshift_openstack_node_subnet_name? Small update: a similar configuration (which uses no floating IP) works for me with octavia,if I set openshift_openstack_node_subnet_name. I have added following to my all.yml inventory file:
openshift_openstack_use_lbaas_load_balancer: true
#openshift_openstack_use_vm_load_balancer: true
openshift_openstack_node_subnet_name: "provider1-sub1"
but still get the same error on deployment.
TASK [openshift_openstack : validate the Heat template] ******************************************************************************************************************************************************************************************
Tuesday 26 February 2019 13:18:58 -0500 (0:00:01.027) 0:00:06.672 ******
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["openstack", "orchestration", "template", "validate", "-t", "/tmp/openshift-ansiblekBfAvU/stack.yaml"], "delta": "0:00:04.296256", "end": "2019-02-26 13:19:02.892572", "msg": "non-zero return code", "rc": 1, "start": "2019-02-26 13:18:58.596316", "stderr": "ERROR: outputs.public_api_ip.value.get_attr: The specified reference \"api_lb_floating_ip\" (in unknown) is incorrect.", "stderr_lines": ["ERROR: outputs.public_api_ip.value.get_attr: The specified reference \"api_lb_floating_ip\" (in unknown) is incorrect."], "stdout": "", "stdout_lines": []}
My subnets are as follows:
(shiftstack) [stack@undercloud-osp14 ~]$ openstack subnet list
+--------------------------------------+-------------------+--------------------------------------+-----------------+
| ID | Name | Network | Subnet |
+--------------------------------------+-------------------+--------------------------------------+-----------------+
| 0b4bce88-2843-406b-a946-d6c67e5a14ef | openshift-dns | 69824114-6d71-49e8-8f30-8bc22d48e953 | 192.168.23.0/24 |
| 6d3ee7f1-275a-4a4f-86d2-d60f4f192717 | baremetal-sub | c219dd21-ad3d-41fc-a3f3-819dc656614a | 172.31.10.0/24 |
| 958aca3e-f128-47c1-ade5-36968a03f208 | public_subnet-int | eeebf1e6-2361-4575-86de-0a8a59ca2712 | 172.31.8.0/24 |
| aed181a9-95a0-4117-bf17-0d37d9e57bc2 | provider1-sub1 | 4a030c51-2a82-4e17-ab10-2464f6fa1a6d | 172.31.5.0/24 |
| b49a4559-8d35-4b85-94a9-55555e7e3643 | provider-sub1 | 961258b0-8510-4a36-9889-190d35ad5bd0 | 172.31.6.0/24 |
+--------------------------------------+-------------------+--------------------------------------+-----------------+
I just realized this looks like a different error Hi! Is openshift_openstack_load_balancer_floating_ip set to false? If I add the openshift_openstack_load_balancer_floating_ip false flag I get past the "validate the Heat template" step, but fail at:
openshift_openstack : Handle the Stack (create/delete) step
The stack creation fails with the following:
(shiftstack) [stack@undercloud-osp14 ~]$ openstack stack failures list chris-openshift
chris-openshift.infra_nodes.2:
resource_type: file:///tmp/openshift-ansible_jUc9t/server.yaml
physical_resource_id:
status: CREATE_FAILED
status_reason: |
resources[2]: Property error: resources.router_lb_pool_member_http.properties.subnet: The Parameter (subnet) was not provided.
Ahh - apologies, I missed the fact that you were using a provider network. Looking at the heat template - specifically https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_openstack/templates/heat_stack_server.yaml.j2#L64-L69 - it looks like this just might not work. Luis, could you comment on whether this configuration is supposed to work or not? I'm actually not sure here, probably we are missing the same if at the router loadbalancer than at the api lb one when using octavia. So something like this https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_openstack/templates/heat_stack_server.yaml.j2#L345-L347 at this: https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_openstack/templates/heat_stack_server.yaml.j2#L357 Hi! Thanks for the pointer. I think I may have misdirected you; the actual error is the one Chris reported in the first comment (everything after is confusion due to me not realizing that a provider network was involved). The error seems to take place here: https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_openstack/templates/heat_stack.yaml.j2#L167-L183 Looking at the conditional, it looks like the api_lb is created when a lbaas load balancer is specified; but it also looks like maybe it shouldn't be if a provider network is also specified? Do you know if that's supposed to work? I don't think a subnet will exist when using a provider network. Chris, I got the provisioning playbook to work with the following: a) set openshift_openstack_node_subnet_name b) set openshift_openstack_load_balancer_floating_ip to false c) apply https://github.com/openshift/openshift-ansible/pull/11285 I don't have a full setup to test for a provider network though; is this a fix you'd be able to test? Thanks. That worked for me as well. @Tzu-Mainn: If the setup was with octavia, I don't understand how PR 11285 helps here. It should only be needed in case lbaasv2 is used, right? And that is not the case. Did you try with only steps a) and b) Luis and I have resolved the somewhat confusing conditional in the PR and agreed that this solution can work! @Chris, could you confirm that the solution works for you in full, including the installation of openshift? If so, I'll update the PR to include docs changes and submit it. It works after performing all the steps from comment #10 .. I think there might be some issue on how octavia handles load balancing in this provider network setup, but this is outside the scope of this BZ. Thanks for providing the solutions to this BZ. Please let me know if I can help reviewing docs. OpenShift have no resources to verify the scenarios with provider_network, this need enable neurton internal dns. And also we have no octivia enabled OSP can do testing. Moving to verified as it was reported that the solution described in comment #10 worked. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 3.11.286 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3695 |