Description of problem: When creating an A/S LB then a UDP listener, the listener is not reachable. No Garp packet is sent by the ACTIVE amphora Version-Release number of selected component (if applicable): 16.1 How reproducible: 100% Steps to Reproduce: 1. Create a LB with A/S 2. Create a UDP listener and its components (pools, member) 3. the listener is not reachable
A workaround exists: if the user creates a TCP/HTTP listener, the UDP listener is "activated"
Reproducer and workaround: [stack@undercloud-0 ~]$ sudo dnf install python3-octavia-tests-tempest-golang.x86_64 [stack@undercloud-0 ~]$ sudo iptables -I INPUT -p udp --dport 8080 -j ACCEPT [stack@undercloud-0 ~]$ /usr/libexec/octavia-tempest-plugin-tests-httpd (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer create --vip-subnet private-subnet --name lb1 (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer listener create --protocol-port 1234 --protocol UDP --name listener1 lb1 (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer pool create --lb-algorithm LEAST_CONNECTIONS --listener listener1 --protocol UDP --name pool1 (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer member create --subnet-id external_subnet --address 10.0.0.77 --protocol-port 8080 pool1 (overcloud) [stack@undercloud-0 ~]$ fip_address=$(openstack floating ip create -f value -c floating_ip_address nova) (overcloud) [stack@undercloud-0 ~]$ openstack floating ip set --port $(openstack loadbalancer show -f value -c vip_port_id lb1) $fip_address (overcloud) [stack@undercloud-0 ~]$ nc -u $fip_address 1234 <PRESS ENTER> No data from the UDP server workaround: (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer listener create --protocol-port 1234 --protocol HTTP --name listener2 lb1 (overcloud) [stack@undercloud-0 ~]$ nc -u $fip_address 1234 <PRESS ENTER> 1 <PRESS ENTER> 1 We receive data from the UDP server
Hotfix verification process: On one terminal I opened, I ran: [stack@undercloud-0 ~]$ cat /etc/rhosp-release Red Hat OpenStack Platform release 16.1.8 (Train) [stack@undercloud-0 ~]$ sudo dnf install python3-octavia-tests-tempest-golang.x86_64 -y . . . Installed: python3-octavia-tests-tempest-golang-1.3.0-1.20210528004836.0ae7f10.el8ost.x86_64 Complete! [stack@undercloud-0 ~]$ sudo iptables -I INPUT -p udp --dport 8080 -j ACCEPT (overcloud) [stack@undercloud-0 ~]$ openstack server create --image cirros-0.3.4-x86_64 --security-group sec_group --flavor 1gb --network int_net udp_verification_member2 --key-name test-key1 (overcloud) [stack@undercloud-0 ~]$ openstack floating ip create nova (overcloud) [stack@undercloud-0 ~]$ openstack server add floating ip udp_verification_member2 10.0.0.162 (overcloud) [stack@undercloud-0 ~]$ scp /usr/libexec/octavia-tempest-plugin-tests-httpd cirros.0.162:/dev/shm/octavia-tempest-plugin-tests-httpd (overcloud) [stack@undercloud-0 ~]$ ssh cirros.0.162 $ /dev/shm/octavia-tempest-plugin-tests-httpd (no output should be seen) # I opened another host's instance on another terminal # I changed the Octavia topology to ACTIVE_STANDBY (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer create --vip-subnet int_sub --name lb1 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | admin_state_up | True | | created_at | 2022-04-27T11:53:08 | | description | | | flavor_id | None | | id | 2e21e82d-9ed6-4f80-8084-a13a91022f6f | | listeners | | | name | lb1 | | operating_status | OFFLINE | | pools | | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | provider | amphora | | provisioning_status | PENDING_CREATE | | updated_at | None | | vip_address | 192.168.1.10 | | vip_network_id | 19a8f0b9-41ea-42f9-9690-907c3f90829e | | vip_port_id | ad7b6b27-e465-4504-bef0-54ea49f4954d | | vip_qos_policy_id | None | | vip_subnet_id | e5e0dc35-fc23-47fb-a285-a79766e3dc3b | +---------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer listener create --protocol-port 1234 --protocol UDP --name listener1 lb1 +-----------------------------+--------------------------------------+ | Field | Value | +-----------------------------+--------------------------------------+ | admin_state_up | True | | connection_limit | -1 | | created_at | 2022-04-27T11:55:57 | | default_pool_id | None | | default_tls_container_ref | None | | description | | | id | 8730ed90-9b62-447e-8a1e-f04baa235e28 | | insert_headers | None | | l7policies | | | loadbalancers | 2e21e82d-9ed6-4f80-8084-a13a91022f6f | | name | listener1 | | operating_status | OFFLINE | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | protocol | UDP | | protocol_port | 1234 | | provisioning_status | PENDING_CREATE | | sni_container_refs | [] | | timeout_client_data | 50000 | | timeout_member_connect | 5000 | | timeout_member_data | 50000 | | timeout_tcp_inspect | 0 | | updated_at | None | | client_ca_tls_container_ref | None | | client_authentication | NONE | | client_crl_container_ref | None | | allowed_cidrs | None | +-----------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer pool create --lb-algorithm LEAST_CONNECTIONS --listener listener1 --protocol UDP --name pool1 +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | admin_state_up | True | | created_at | 2022-04-27T11:56:02 | | description | | | healthmonitor_id | | | id | a2680bff-6e7c-45d2-b18b-6c5180d76d08 | | lb_algorithm | LEAST_CONNECTIONS | | listeners | 8730ed90-9b62-447e-8a1e-f04baa235e28 | | loadbalancers | 2e21e82d-9ed6-4f80-8084-a13a91022f6f | | members | | | name | pool1 | | operating_status | OFFLINE | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | protocol | UDP | | provisioning_status | PENDING_CREATE | | session_persistence | None | | updated_at | None | | tls_container_ref | None | | ca_tls_container_ref | None | | crl_container_ref | None | | tls_enabled | False | +----------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer member create --subnet-id int_sub --address 192.168.1.75 --protocol-port 8080 pool1 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | address | 192.168.1.75 | | admin_state_up | True | | created_at | 2022-04-27T11:56:27 | | id | 2722ed20-076a-4228-8c77-683ebbcf04e8 | | name | | | operating_status | NO_MONITOR | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | protocol_port | 8080 | | provisioning_status | PENDING_CREATE | | subnet_id | e5e0dc35-fc23-47fb-a285-a79766e3dc3b | | updated_at | None | | weight | 1 | | monitor_port | None | | monitor_address | None | | backup | False | +---------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack floating ip list +--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+ | ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project | +--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+ | 40cad6c1-ec76-4564-81a1-008379c88b7b | 10.0.0.189 | None | None | 389308e5-31f6-4a00-bb8a-e543939245ab | ec3ebf1fd3144c80bfb12a884fdc7621 | | aa1e2710-ae31-4555-bdf2-f1f24819aabf | 10.0.0.224 | None | None | 389308e5-31f6-4a00-bb8a-e543939245ab | ec3ebf1fd3144c80bfb12a884fdc7621 | | f7c8f9ae-a2b5-43e9-8e12-521fabc695cd | 10.0.0.162 | 192.168.1.75 | 2ced98fd-9793-4d50-82f8-a45c205035cc | 389308e5-31f6-4a00-bb8a-e543939245ab | ec3ebf1fd3144c80bfb12a884fdc7621 | +--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack floating ip set --port $(openstack loadbalancer show -f value -c vip_port_id lb1) 10.0.0.189 (overcloud) [stack@undercloud-0 ~]$ nc -u 10.0.0.189 1234 <no response> ^C # # Creating an http listener - workaround # (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer listener create --protocol-port 1234 --protocol HTTP --name listener2 lb1 +-----------------------------+--------------------------------------+ | Field | Value | +-----------------------------+--------------------------------------+ | admin_state_up | True | | connection_limit | -1 | | created_at | 2022-04-27T11:57:15 | | default_pool_id | None | | default_tls_container_ref | None | | description | | | id | 50fd14a5-9b76-46fa-9131-b5cdfdbe8083 | | insert_headers | None | | l7policies | | | loadbalancers | 2e21e82d-9ed6-4f80-8084-a13a91022f6f | | name | listener2 | | operating_status | OFFLINE | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | protocol | HTTP | | protocol_port | 1234 | | provisioning_status | PENDING_CREATE | | sni_container_refs | [] | | timeout_client_data | 50000 | | timeout_member_connect | 5000 | | timeout_member_data | 50000 | | timeout_tcp_inspect | 0 | | updated_at | None | | client_ca_tls_container_ref | None | | client_authentication | NONE | | client_crl_container_ref | None | | allowed_cidrs | None | +-----------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ nc -u 10.0.0.189 1234 111 1 1^C # # Applying hotfix # (overcloud) [stack@undercloud-0 ~]$ octavia_version=5.0.3-1.20220427074533.8c32d2e (overcloud) [stack@undercloud-0 ~]$ rpm -qa | grep octavia-amphora-image octavia-amphora-image-x86_64-16.1-20220315.1.el8ost.noarch (overcloud) [stack@undercloud-0 ~]$ sudo dnf install libguestfs-tools -y . . . Complete! (overcloud) [stack@undercloud-0 ~]$ rpm -qa | grep octavia-amphora-image octavia-amphora-image-x86_64-16.1-20220315.1.el8ost.noarch (overcloud) [stack@undercloud-0 ~]$ amphora_image_version=16.1-20220315.1 (overcloud) [stack@undercloud-0 ~]$ rpm -ql octavia-amphora-image-x86_64-${amphora_image_version}.el8ost.noarch | grep qcow2 /usr/share/openstack-octavia-amphora-images/octavia-amphora-16.1-20220315.1.x86_64.qcow2 (overcloud) [stack@undercloud-0 ~]$ cp /usr/share/openstack-octavia-amphora-images/octavia-amphora-${amphora_image_version}.x86_64.qcow2 . (overcloud) [stack@undercloud-0 ~]$ virt-customize -a octavia-amphora-${amphora_image_version}.x86_64.qcow2 \ > --copy-in openstack-octavia-amphora-agent-${octavia_version}.el8ost.noarch.rpm:/tmp \ > --copy-in openstack-octavia-common-${octavia_version}.el8ost.noarch.rpm:/tmp \ > --copy-in python3-octavia-${octavia_version}.el8ost.noarch.rpm:/tmp \ > --install /tmp/openstack-octavia-amphora-agent-${octavia_version}.el8ost.noarch.rpm,/tmp/openstack-octavia-common-${octavia_version}.el8ost.noarch.rpm,/tmp/python3-octavia-${octavia_version}.el8ost.noarch.rpm \ > --delete /tmp/openstack-octavia-amphora-agent-${octavia_version}.el8ost.noarch.rpm \ > --delete /tmp/openstack-octavia-common-${octavia_version}.el8ost.noarch.rpm \ > --delete /tmp/python3-octavia-${octavia_version}.el8ost.noarch.rpm \ > --selinux-relabel [ 0.0] Examining the guest ... [ 14.4] Setting a random seed [ 14.4] Setting the machine ID in /etc/machine-id [ 14.4] Copying: openstack-octavia-amphora-agent-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm to /tmp [ 14.5] Copying: openstack-octavia-common-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm to /tmp [ 14.5] Copying: python3-octavia-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm to /tmp [ 14.5] Installing packages: /tmp/openstack-octavia-amphora-agent-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm /tmp/openstack-octavia-common-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm /tmp/python3-octavia-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm [ 23.8] Deleting: /tmp/openstack-octavia-amphora-agent-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm [ 23.8] Deleting: /tmp/openstack-octavia-common-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm [ 23.8] Deleting: /tmp/python3-octavia-5.0.3-1.20220427074533.8c32d2e.el8ost.noarch.rpm [ 23.8] SELinux relabelling [ 57.3] Finishing off (overcloud) [stack@undercloud-0 ~]$ openstack image create --disk-format qcow2 \ > --container-format bare \ > --project service \ > --tag amphora-image \ > --private \ > --file ./octavia-amphora-${amphora_image_version}.x86_64.qcow2 \ > octavia-amphora-${amphora_image_version}.x86_64.bz2078377 +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | 602bec78b5dddbabb24148ecb7e5f9b8 | | container_format | bare | | created_at | 2022-04-27T12:05:30Z | | disk_format | qcow2 | | file | /v2/images/66678368-ae02-4cd2-9607-4c84eea53315/file | | id | 66678368-ae02-4cd2-9607-4c84eea53315 | | min_disk | 0 | | min_ram | 0 | | name | octavia-amphora-16.1-20220315.1.x86_64.bz2078377 | | owner | 6476e6fad3a648a9847154211060da92 | | properties | direct_url='swift+config://ref1/glance/66678368-ae02-4cd2-9607-4c84eea53315', os_hash_algo='sha512', os_hash_value='e6493a4d337c90baf91ad80439aa506875dbd3b20fed611ef8a6655b1e5ed63be4a1c7c1116492d778b174a6d78ced525e5c858e6e1af7fdbd743b4006944467', os_hidden='False', stores='default_backend' | | protected | False | | schema | /v2/schemas/image | | size | 966131712 | | status | active | | tags | amphora-image | | updated_at | 2022-04-27T12:05:41Z | | virtual_size | None | | visibility | private | +------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack image unset --tag amphora-image octavia-amphora-${amphora_image_version}.x86_64 # # Testing hotfix # (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer delete --cascade lb1 (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer create --vip-subnet int_sub --name lb1 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | admin_state_up | True | | created_at | 2022-04-27T12:07:49 | | description | | | flavor_id | None | | id | 0d3a0310-8464-4abd-90aa-e74122f287ca | | listeners | | | name | lb1 | | operating_status | OFFLINE | | pools | | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | provider | amphora | | provisioning_status | PENDING_CREATE | | updated_at | None | | vip_address | 192.168.1.122 | | vip_network_id | 19a8f0b9-41ea-42f9-9690-907c3f90829e | | vip_port_id | 66c15306-e764-4c88-8f00-c5651b17e894 | | vip_qos_policy_id | None | | vip_subnet_id | e5e0dc35-fc23-47fb-a285-a79766e3dc3b | +---------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer listener create --protocol-port 1234 --protocol UDP --name listener1 lb1 +-----------------------------+--------------------------------------+ | Field | Value | +-----------------------------+--------------------------------------+ | admin_state_up | True | | connection_limit | -1 | | created_at | 2022-04-27T12:09:53 | | default_pool_id | None | | default_tls_container_ref | None | | description | | | id | 15414706-5b31-419d-bd24-1da8341be43b | | insert_headers | None | | l7policies | | | loadbalancers | 0d3a0310-8464-4abd-90aa-e74122f287ca | | name | listener1 | | operating_status | OFFLINE | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | protocol | UDP | | protocol_port | 1234 | | provisioning_status | PENDING_CREATE | | sni_container_refs | [] | | timeout_client_data | 50000 | | timeout_member_connect | 5000 | | timeout_member_data | 50000 | | timeout_tcp_inspect | 0 | | updated_at | None | | client_ca_tls_container_ref | None | | client_authentication | NONE | | client_crl_container_ref | None | | allowed_cidrs | None | +-----------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer pool create --lb-algorithm LEAST_CONNECTIONS --listener listener1 --protocol UDP --name pool1 +----------------------+--------------------------------------+ | Field | Value | +----------------------+--------------------------------------+ | admin_state_up | True | | created_at | 2022-04-27T12:09:59 | | description | | | healthmonitor_id | | | id | 929c52be-9259-4744-a08f-275261b22eb9 | | lb_algorithm | LEAST_CONNECTIONS | | listeners | 15414706-5b31-419d-bd24-1da8341be43b | | loadbalancers | 0d3a0310-8464-4abd-90aa-e74122f287ca | | members | | | name | pool1 | | operating_status | OFFLINE | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | protocol | UDP | | provisioning_status | PENDING_CREATE | | session_persistence | None | | updated_at | None | | tls_container_ref | None | | ca_tls_container_ref | None | | crl_container_ref | None | | tls_enabled | False | +----------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer member create --subnet-id int_sub --address 192.168.1.75 --protocol-port 8080 pool1 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | address | 192.168.1.75 | | admin_state_up | True | | created_at | 2022-04-27T12:10:06 | | id | 5f4a88aa-bdf4-440a-aca5-0f5e966f1a13 | | name | | | operating_status | NO_MONITOR | | project_id | ec3ebf1fd3144c80bfb12a884fdc7621 | | protocol_port | 8080 | | provisioning_status | PENDING_CREATE | | subnet_id | e5e0dc35-fc23-47fb-a285-a79766e3dc3b | | updated_at | None | | weight | 1 | | monitor_port | None | | monitor_address | None | | backup | False | +---------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack floating ip set --port $(openstack loadbalancer show -f value -c vip_port_id lb1) 10.0.0.189 (overcloud) [stack@undercloud-0 ~]$ nc -u 10.0.0.189 1234 1 1 1^C # hotfix verified
Verification process appears at comment #7 (https://bugzilla.redhat.com/show_bug.cgi?id=2078377#c7) I am moving the BZ status to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.9 bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:8795