Bug 2213742 - TCP/HTTP health-monitors in UDP pools work only on specific ports [NEEDINFO]
Summary: TCP/HTTP health-monitors in UDP pools work only on specific ports
Keywords:
Status: MODIFIED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-octavia
Version: 17.1 (Wallaby)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z2
: 17.1
Assignee: Gregory Thiemonge
QA Contact: Bruna Bonguardo
Greg Rakauskas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-09 06:08 UTC by Gregory Thiemonge
Modified: 2023-08-03 15:46 UTC (History)
3 users (show)

Fixed In Version: openstack-octavia-12.0.1-18.0.20230713233746.3ac52a0.el9ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:
ifrangs: needinfo? (gthiemon)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 886046 0 None MERGED Fix TCP HMs on UDP pools with SELinux 2023-06-26 06:16:13 UTC
OpenStack gerrit 887482 0 None NEW Fix TCP HMs on UDP pools with SELinux 2023-07-12 13:17:23 UTC
Red Hat Issue Tracker OSP-25711 0 None None None 2023-06-09 06:27:01 UTC

Description Gregory Thiemonge 2023-06-09 06:08:24 UTC
Description of problem:
With OSP17, users can add TCP/HTTP health-monitors to UDP pools, but some ports are denied by SELinux

Version-Release number of selected component (if applicable):
17.1

How reproducible:
100%

Steps to Reproduce:
1. Deploy OSP+Octavia
2. Create a LB+UDP Listener+UDP Pool+1 member (port 1234 or monitor_port 1234)+TCP or HTTP HM
3. 

Actual results:
No TCP/HTTP packets are sent by keepalived (and there are SELinux denials in the logs)
HM doesn't work

Expected results:
HM packets should not be blocked

Additional info:

Reproducer:

openstack loadbalancer create --vip-subnet external_subnet --name lb1
openstack loadbalancer listener create --protocol UDP --protocol-port 1234 --name listener1 lb1
openstack loadbalancer pool create --protocol UDP --listener listener1 --lb-algorithm ROUND_ROBIN --name pool1
openstack loadbalancer member create --address 10.0.0.72 --protocol-port 1234 --name member1 pool1
openstack loadbalancer healthmonitor create --name hm1 --type HTTP --delay 5 --timeout 4 --max-retries 3 pool1


Log into the amphora, check the audit log:

# tail -f /var/log/audit/audit.log
type=AVC msg=audit(1686240221.266:838): avc:  denied  { name_connect } for  pid=15945 comm="keepalived" dest=1234 scontext=system_u:system_r:keepalived_t:s0 tcontext=system_u:object_r:monopd_port_t:s0 tclass=tcp_socket permissive=0
type=SYSCALL msg=audit(1686240221.266:838): arch=c000003e syscall=42 success=no exit=-13 a0=8 a1=555c80b97600 a2=80 a3=7ffcdc746b50 items=0 ppid=15944 pid=15945 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="keepalived" exe="/usr/sbin/keepalived" subj=system_u:system_r:keepalived_t:s0 key=(null)ARCH=x86_64 SYSCALL=connect AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

Check tcpdump output:

# ip netns exec amphora-haproxy tcpdump -nn -i eth1
[..] no TCP packets


Try another port number:

openstack loadbalancer member set --monitor-port 1235 pool1 member1

Check the logs and tcpdump output:

# tail -f /var/log/audit/audit.log
[..] no denied for name_connect

# ip netns exec amphora-haproxy tcpdump -nn -i eth1
[..]
12:06:22.175945 IP 10.0.0.233.18583 > 10.0.0.72.1235: Flags [S], seq 1796162379, win 29200, options [mss 1460,sackOK,TS val 3659440612 ecr 0,nop,wscale 11], length 0
[..]


It seems that keepalived allows only TCP connections to a specific list of port numbers:

There's a related BZ for RHEL7 https://bugzilla.redhat.com/show_bug.cgi?id=1443473
Related KCS https://access.redhat.com/solutions/2119681


It can be mitigated with a SELinux boolean:

# semanage boolean -l | grep keepal
keepalived_connect_any         (off  ,  off)  Allow keepalived to connect any

From the keepalived_selinux manpage:

| If you want to determine whether keepalived can connect to all TCP ports, you must turn on the keepalived_connect_any boolean. Disabled by default.
| setsebool -P keepalived_connect_any 1

Comment 6 Gregory Thiemonge 2023-06-14 08:42:49 UTC
Fix proposed to octavia master

Comment 8 Gregory Thiemonge 2023-07-03 08:28:51 UTC
Backport proposed to stable/wallaby


Note You need to log in before you can comment on or make changes to this bug.