Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2173776

Summary:	DNS packet corruption seen with Neutron internal DNS on 16.2 and 17.0
Product:	Red Hat OpenStack	Reporter:	Peter Legg <plegg>
Component:	openstack-neutron	Assignee:	Miguel Lavalle <mlavalle>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Eran Kuris <ekuris>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	17.0 (Wallaby)	CC:	averdagu, chrisw, scohen, vcojot
Target Milestone:	---	Flags:	plegg: needinfo- plegg: needinfo-
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2024-06-06 16:34:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Peter Legg 2023-02-27 23:44:47 UTC

Description of problem:

When doing a dns lookup (dig) on a fqdn in the NeutronDomain (neutron.conf = dns_domain=nekohouse.ca) the response from Neutron 

[root@idm-nekohouse-ca ~]# dig test1.nekohouse.ca
;; Warning: Message parser reports malformed message packet.

[.............] 
;; ANSWER SECTION:
.			0	CLASS4096 OPT	10 8 cClEBcwRGB4=
[.............] 

If you run the same request with the +noedns option, no corruption is present:

[root@idm-nekohouse-ca ~]# dig +noedns test1.nekohouse.ca
[.............]
;; ANSWER SECTION:
test1.nekohouse.ca.	3600	IN	A	10.10.30.84
[.............]

On other dns requests, where the fqdn doesn't match the NeutronDomain, resolves properly from my external DNS (pdns). 

[root@idm-nekohouse-ca ~]# hostname -A
idm-nekohouse-ca.nekohouse.ca 

[root@idm-nekohouse-ca ~]# dig idm.nekohouse.ca

[.............] 
;; ANSWER SECTION:
idm.nekohouse.ca.	3450	IN	A	10.10.30.68
[.............] 




Version-Release number of selected component (if applicable):
AIO 16.2 and 17.0

How reproducible:
100% of the time

Steps to Reproduce:
1. Stand up AIO 
2. Create an instance with a name that matches the fqdn of the neutron port
3. dig the fqdn 

Actual results:
Corrupted packet from Neutron in dns reply

Expected results:
No corruption

Additional info:

Comment 1 Peter Legg 2023-02-27 23:57:34 UTC

[peter@fedora ~]$ openstack port show 1f14d020-8fe4-48d3-ac83-6df71f43c68a
+-------------------------+-----------------------------------------------------------------------------+
| Field                   | Value                                                                       |
+-------------------------+-----------------------------------------------------------------------------+
| admin_state_up          | UP                                                                          |
| allowed_address_pairs   |                                                                             |
| binding_host_id         | aio.osp.nekohouse.ca                                                        |
| binding_profile         |                                                                             |
| binding_vif_details     | connectivity='l2', port_filter='True'                                       |
| binding_vif_type        | ovs                                                                         |
| binding_vnic_type       | normal                                                                      |
| created_at              | 2023-02-27T23:50:27Z                                                        |
| data_plane_status       | None                                                                        |
| description             |                                                                             |
| device_id               | 83e67612-ab34-440b-9d8b-7ceecef51b86                                        |
| device_owner            | compute:nova                                                                |
| device_profile          | None                                                                        |
| dns_assignment          | fqdn='test1.nekohouse.ca.', hostname='test1', ip_address='10.10.30.100'     | <========----- fqdn is being defined by Neutron
| dns_domain              |                                                                             |
| dns_name                | test1                                                                       |
| extra_dhcp_opts         |                                                                             |
| fixed_ips               | ip_address='10.10.30.100', subnet_id='cfc8f525-315b-4eb6-b1cd-0a469c6d8d9c' |
| id                      | 1f14d020-8fe4-48d3-ac83-6df71f43c68a                                        |
| ip_allocation           | immediate                                                                   |
| mac_address             | fa:16:3e:3d:64:9f                                                           |
| name                    |                                                                             |
| network_id              | 577d8daa-1a86-4380-9133-db584213b050                                        |
| numa_affinity_policy    | None                                                                        |
| port_security_enabled   | True                                                                        |
| project_id              | 89163b60e44444cdb15d038445fef253                                            |
| propagate_uplink_status | None                                                                        |
| qos_network_policy_id   | None                                                                        |
| qos_policy_id           | None                                                                        |
| resource_request        | None                                                                        |
| revision_number         | 4                                                                           |
| security_group_ids      | e8cde397-4518-4637-9767-ddc630a2d2e7                                        |
| status                  | ACTIVE                                                                      |
| tags                    |                                                                             |
| trunk_details           | None                                                                        |
| updated_at              | 2023-02-27T23:51:41Z                                                        |
+-------------------------+-----------------------------------------------------------------------------+

Comment 2 Miguel Lavalle 2023-03-18 00:01:58 UTC

Hi Peter,

I have some questions:

1) Is this happening with ML2 / OVS or ML2 / OVN?

2) What do you mean when you say "Create an instance with a name that matches the fqdn of the neutron port"? When you setup Neutron with a value in dns_domain other than its default (openstacklocal), the expected behavior is that the FQDN for the port will be the instance name + the value specified in the dns_domain Neutron config option, without having to match any fqdn. As an example, I just configured a test system with:

[heat-admin@controller-0 ~]$ sudo cat /var/lib/config-data/puppet-generated/neutron/etc/neutron/neutron.conf | grep dns_domain
dns_domain=nekohouse.ca

So when I create an instance named 'test1':

(overcloud) [stack@undercloud-0 ~]$ openstack server create --image 4d9872d5-5ff1-4cf9-a35c-0948c356de41 --flavor m1.tiny --network private1 --security-group sg1 test1
[.......]

(overcloud) [stack@undercloud-0 ~]$ openstack server list -c ID -c Name
+--------------------------------------+-------+
| ID                                   | Name  |
+--------------------------------------+-------+
| 2b7b74ec-df61-4bbe-bb37-7848bf0546f5 | test1 |
+--------------------------------------+-------+

Nova and Neutron cooperate to produce the values you see in the port's dns_assignment attribute:

(overcloud) [stack@undercloud-0 ~]$ openstack port list --server 2b7b74ec-df61-4bbe-bb37-7848bf0546f5
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+
| ID                                   | Name | MAC Address       | Fixed IP Addresses                                                           | Status |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+
| f3c0ec7b-f61b-4b5d-a95a-b6d92f80e546 |      | fa:16:3e:e4:ba:2f | ip_address='192.168.30.47', subnet_id='114807c7-d8f1-4062-9959-6117f2fa5eaf' | ACTIVE |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+

(overcloud) [stack@undercloud-0 ~]$ openstack port show f3c0ec7b-f61b-4b5d-a95a-b6d92f80e546 -c dns_assignment
+----------------+--------------------------------------------------------------------------+
| Field          | Value                                                                    |
+----------------+--------------------------------------------------------------------------+
| dns_assignment | fqdn='test1.nekohouse.ca.', hostname='test1', ip_address='192.168.30.47' |
+----------------+--------------------------------------------------------------------------+

This happened without any matching on my part. It is the expected behavior, as I said above

3) You execute your dig command from [root@idm-nekohouse-ca ~]#. Is this the system where you are running the AIO system? Are you inside a container?

@plegg