Bug 1759254

Summary:	Add parameter to control Octavia amphora connection_logging
Product:	Red Hat OpenStack	Reporter:	Carlos Goncalves <cgoncalves>
Component:	openstack-tripleo-heat-templates	Assignee:	Michael Johnson <michjohn>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Bruna Bonguardo <bbonguar>
Severity:	medium	Docs Contact:
Priority:	high
Version:	13.0 (Queens)	CC:	broose, dhill, dvd, mburns, mgarciac, michjohn, mircea.vutcovici
Target Milestone:	z12	Keywords:	TestOnly, Triaged, ZStream
Target Release:	13.0 (Queens)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-8.4.1-22.el7ost puppet-octavia-12.4.0-14.el7ost	Doc Type:	Enhancement
Doc Text:	This enhancement adds the `OctaviaConnectionLogging` parameter so that you can disable connection flow logging. The default setting is `true`, which means that connection flows are logged.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-06-02 10:44:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1800847, 1804412
Bug Blocks:	1732834, 1759724, 1873568, 1873569, 1873570

Description Carlos Goncalves 2019-10-07 17:20:41 UTC

Amphorae currently log connections. Logging the traffic flows impact performance. It would be nice if Director could expose a parameter to enable/disable connection flow logging in amphorae.

Comment 16 Lon Hohberger 2020-04-20 10:45:22 UTC

According to our records, this should be resolved by openstack-tripleo-heat-templates-8.4.1-51.el7ost.  This build is available now.

Comment 17 Lon Hohberger 2020-04-20 10:45:26 UTC

According to our records, this should be resolved by puppet-octavia-12.4.0-15.el7ost.  This build is available now.

Comment 18 Bruna Bonguardo 2020-05-29 15:06:50 UTC

(overcloud) [stack@undercloud-0 ~]$ rpm -qa | grep openstack-tripleo-heat-templates
openstack-tripleo-heat-templates-8.4.1-58.el7ost.noarch
(overcloud) [stack@undercloud-0 ~]$ rpm -qa | grep puppet-octavia-
puppet-octavia-12.4.0-16.el7ost.noarch
(overcloud) [stack@undercloud-0 ~]$ cat /var/lib/rhos-release/latest-installed
13  -p 2020-05-19.2


# "connection_logging" is True in the octavia.conf file (inside the controllers):

[root@controller-0 ~]# cat /var/lib/config-data/puppet-generated/octavia/etc/octavia/octavia.conf | grep connection_logging
# connection_logging = True
connection_logging=True

# In undercloud, traffic was sent to the loadbalancer's VIP:

[stack@undercloud-0 ~]$ req='curl 10.0.0.235'; for i in {1..10}; do $req; echo; done
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn


# Inside the amphora, the connections run in the previous step are logged:

[cloud-user@amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 ~]$ sudo journalctl | grep haproxy

May 29 10:10:46 amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 haproxy[1817]: 10.0.0.78:48158 [29/May/2020:10:10:46.319] e2cf723c-220f-477d-81c5-3a0c1c993d3d a6c8663f-cca9-461e-a79a-14c6947d3601/06d8d50e-c42e-4912-b9fe-ed963561be78 0/0/3/15/18 200 73 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"
May 29 10:10:46 amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 haproxy[1817]: 10.0.0.78:48160 [29/May/2020:10:10:46.349] e2cf723c-220f-477d-81c5-3a0c1c993d3d a6c8663f-cca9-461e-a79a-14c6947d3601/533ad006-d430-41ab-9d8d-91081dcc930a 0/0/0/17/17 200 73 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"
May 29 10:10:46 amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 haproxy[1817]: 10.0.0.78:48162 [29/May/2020:10:10:46.375] e2cf723c-220f-477d-81c5-3a0c1c993d3d a6c8663f-cca9-461e-a79a-14c6947d3601/06d8d50e-c42e-4912-b9fe-ed963561be78 0/0/0/16/16 200 73 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"
May 29 10:10:46 amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 haproxy[1817]: 10.0.0.78:48164 [29/May/2020:10:10:46.402] e2cf723c-220f-477d-81c5-3a0c1c993d3d a6c8663f-cca9-461e-a79a-14c6947d3601/533ad006-d430-41ab-9d8d-91081dcc930a 0/0/1/21/22 200 73 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"
May 29 10:10:46 amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 haproxy[1817]: 10.0.0.78:48166 [29/May/2020:10:10:46.434] e2cf723c-220f-477d-81c5-3a0c1c993d3d a6c8663f-cca9-461e-a79a-14c6947d3601/06d8d50e-c42e-4912-b9fe-ed963561be78 0/0/0/17/17 200 73 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"
May 29 10:10:46 amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 haproxy[1817]: 10.0.0.78:48168 [29/May/2020:10:10:46.462] e2cf723c-220f-477d-81c5-3a0c1c993d3d a6c8663f-cca9-461e-a79a-14c6947d3601/533ad006-d430-41ab-9d8d-91081dcc930a 0/0/1/17/18 200 73 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"
May 29 10:10:46 amphora-1ece9c77-eea5-47b6-a6c8-777ada305cc7 haproxy[1817]: 10.0.0.78:48170 [29/May/2020:10:10:46.490] e2cf723c-220f-477d-81c5-3a0c1c993d3d a6c8663f-cca9-461e-a79a-14c6947d3601/06d8d50e-c42e-4912-b9fe-ed963561be78 0/0/1/17/18 200 73 - - ---- 1/1/0/1/0 0/0 "GET / HTTP/1.1"


# "connection_logging" was changed to False in the octavia.conf file (inside the controllers):

[root@controller-0 ~]# cat /var/lib/config-data/puppet-generated/octavia/etc/octavia/octavia.conf | grep connection_logging
# connection_logging = True
connection_logging=False

# A failover was triggered to the load balancer, as the setting of connection_logging is passed on to the amphora on amphora creation:
(overcloud) [stack@undercloud-0 ~]$ openstack loadbalancer failover lbtreevmshttp-loadbalancer-wvdesgex2x7o
[2020-05-29 10:51:19] (overcloud) [stack@undercloud-0 ~]$ 


# In undercloud, traffic was sent to the loadbalancer's VIP:

[stack@undercloud-0 ~]$ req='curl 10.0.0.235'; for i in {1..10}; do $req; echo; done
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn
lbtreevmshttp-server1-3pajopi2af5i
lbtreevmshttp-server2-dhjb3zqoakrn


# Inside the amphora, no more connection logging:
[cloud-user@amphora-c9329006-56f4-4381-87bd-b916c02a745d ~]$ sudo journalctl | grep haproxy
May 29 10:52:27 amphora-c9329006-56f4-4381-87bd-b916c02a745d amphora-agent[814]: 2020-05-29 10:52:27.697 1201 WARNING octavia.amphorae.backends.agent.api_server.haproxy_compatibility [-] Found 1.5 version of haproxy. Disabling external checks. Health monitor of type PING will revert to TCP.: error: [Errno 11] Resource temporarily unavailable
May 29 10:52:27 amphora-c9329006-56f4-4381-87bd-b916c02a745d systemd[1]: Starting Configure amphora-haproxy network namespace...
May 29 10:52:28 amphora-c9329006-56f4-4381-87bd-b916c02a745d systemd[1]: Started Configure amphora-haproxy network namespace.
May 29 10:52:28 amphora-c9329006-56f4-4381-87bd-b916c02a745d ip[1271]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /var/lib/octavia/e2cf723c-220f-477d-81c5-3a0c1c993d3d/haproxy.cfg -f /var/lib/octavia/haproxy-default-user-group.conf -p /var/lib/octavia/e2cf723c-220f-477d-81c5-3a0c1c993d3d/e2cf723c-220f-477d-81c5-3a0c1c993d3d.pid -L BKNsJFgU-04GMTUxYv55oBEV34g -Ds
[cloud-user@amphora-c9329006-56f4-4381-87bd-b916c02a745d ~]$ 
[cloud-user@amphora-c9329006-56f4-4381-87bd-b916c02a745d ~]$ 

Moving the bug to VERIFIED.