When you upgrade from RHOSP 16.2 to 17.1.4, Ansible `iptables` modules are automatically migrated to `nftables` modules. Puppet tripleo firewall options also change to a new format. For more information about firewall options, see link:https://docs.redhat.com/en/documentation/red_hat_openstack_platform/17.1/html-single/hardening_red_hat_openstack_platform/index#proc_adding-services-to-the-overcloud-firewall_security-enhancements[Adding services to the overcloud firewall] in _Hardening Red Hat OpenStack Platform_.
Description of problem:
During an overcloud FFU from 16.2 to 17.1.3, nftables are flushed and not reloaded
Version-Release number of selected component (if applicable):
17.1.3 with hotfixes provided to migrate from iptables to nftables [0]
How reproducible:
Always reproducible
Steps to Reproduce:
1. Install hotfixes
2. Perform undercloud and overcloud upgrade as documented
3.
Actual results:
Upgraded compute node shows no nft rules loaded
Expected results:
nft rules are loaded after overcloud upgrade process is successfully completed
Additional info:
Based on the investigation on the attached case, we can see that the rules are in place on disk and were apparently created on July 2nd at 9:53:
~~~
# ls -lahrt /etc/nftables/
total 52K
-rw-------. 1 root root 407 Feb 21 2023 router.nft
drwx------. 2 root root 19 Feb 21 2023 osf
-rw-------. 1 root root 1.1K Feb 21 2023 nat.nft
-rw-------. 1 root root 1.7K Feb 21 2023 main.nft
-rw-r--r--. 1 root root 3.0K Jul 2 09:53 iptables.nft
-rw-r--r--. 1 root root 233 Jul 2 09:53 tripleo-jumps.nft
-rw-r--r--. 1 root root 233 Jul 2 09:53 tripleo-update-jumps.nft
-rw-r--r--. 1 root root 161 Jul 2 09:53 tripleo-flushes.nft
-rw-r--r--. 1 root root 290 Jul 2 09:53 tripleo-chains.nft
-rw-r--r--. 1 root root 6.5K Jul 2 09:53 tripleo-rules.nft
drwx------. 3 root root 221 Jul 3 13:38 .
drwxr-xr-x. 139 root root 8.0K Jul 4 07:36 ..
~~~
The host was LEAPP'd at 13:27 and came back up 14:01 on July 3rd:
~~~
2024-07-03 13:27:08.246468 | 525400e7-eddc-6d46-f2b1-0000000004d6 | TASK | reboot to perform the upgrade
2024-07-03 14:01:02.286954 | 525400e7-eddc-6d46-f2b1-0000000004d6 | CHANGED | reboot to perform the upgrade | ********
~~~
The nftables service was started at 13:59 on July 3rd, matching the LEAPP upgrade:
~~~
# systemctl status nftables.service
● nftables.service - Netfilter Tables
Loaded: loaded (/usr/lib/systemd/system/nftables.service; enabled; preset: disabled)
Active: active (exited) since Wed 2024-07-03 13:59:34 UTC; 18h ago
Docs: man:nft(8)
Main PID: 6889 (code=exited, status=0/SUCCESS)
CPU: 15ms
Jul 03 13:59:32 ******** systemd[1]: Starting Netfilter Tables...
Jul 03 13:59:34 ******** systemd[1]: Finished Netfilter Tables.
~~~
But the rules were flushed 4 minutes later and the service not restarted/reloaded afterwards:
~~~
2024-07-03 14:04:58.066994 | 525400e7-eddc-efee-6b5e-000000001275 | TASK | Empty nftables from anything that may lay around
2024-07-03 14:04:58.264654 | 525400e7-eddc-efee-6b5e-000000001275 | CHANGED | Empty nftables from anything that may lay around | ********
~~~
[0] https://bugzilla.redhat.com/show_bug.cgi?id=2243267#c13
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (RHOSP 17.1.4 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2024:9974
Description of problem: During an overcloud FFU from 16.2 to 17.1.3, nftables are flushed and not reloaded Version-Release number of selected component (if applicable): 17.1.3 with hotfixes provided to migrate from iptables to nftables [0] How reproducible: Always reproducible Steps to Reproduce: 1. Install hotfixes 2. Perform undercloud and overcloud upgrade as documented 3. Actual results: Upgraded compute node shows no nft rules loaded Expected results: nft rules are loaded after overcloud upgrade process is successfully completed Additional info: Based on the investigation on the attached case, we can see that the rules are in place on disk and were apparently created on July 2nd at 9:53: ~~~ # ls -lahrt /etc/nftables/ total 52K -rw-------. 1 root root 407 Feb 21 2023 router.nft drwx------. 2 root root 19 Feb 21 2023 osf -rw-------. 1 root root 1.1K Feb 21 2023 nat.nft -rw-------. 1 root root 1.7K Feb 21 2023 main.nft -rw-r--r--. 1 root root 3.0K Jul 2 09:53 iptables.nft -rw-r--r--. 1 root root 233 Jul 2 09:53 tripleo-jumps.nft -rw-r--r--. 1 root root 233 Jul 2 09:53 tripleo-update-jumps.nft -rw-r--r--. 1 root root 161 Jul 2 09:53 tripleo-flushes.nft -rw-r--r--. 1 root root 290 Jul 2 09:53 tripleo-chains.nft -rw-r--r--. 1 root root 6.5K Jul 2 09:53 tripleo-rules.nft drwx------. 3 root root 221 Jul 3 13:38 . drwxr-xr-x. 139 root root 8.0K Jul 4 07:36 .. ~~~ The host was LEAPP'd at 13:27 and came back up 14:01 on July 3rd: ~~~ 2024-07-03 13:27:08.246468 | 525400e7-eddc-6d46-f2b1-0000000004d6 | TASK | reboot to perform the upgrade 2024-07-03 14:01:02.286954 | 525400e7-eddc-6d46-f2b1-0000000004d6 | CHANGED | reboot to perform the upgrade | ******** ~~~ The nftables service was started at 13:59 on July 3rd, matching the LEAPP upgrade: ~~~ # systemctl status nftables.service ● nftables.service - Netfilter Tables Loaded: loaded (/usr/lib/systemd/system/nftables.service; enabled; preset: disabled) Active: active (exited) since Wed 2024-07-03 13:59:34 UTC; 18h ago Docs: man:nft(8) Main PID: 6889 (code=exited, status=0/SUCCESS) CPU: 15ms Jul 03 13:59:32 ******** systemd[1]: Starting Netfilter Tables... Jul 03 13:59:34 ******** systemd[1]: Finished Netfilter Tables. ~~~ But the rules were flushed 4 minutes later and the service not restarted/reloaded afterwards: ~~~ 2024-07-03 14:04:58.066994 | 525400e7-eddc-efee-6b5e-000000001275 | TASK | Empty nftables from anything that may lay around 2024-07-03 14:04:58.264654 | 525400e7-eddc-efee-6b5e-000000001275 | CHANGED | Empty nftables from anything that may lay around | ******** ~~~ [0] https://bugzilla.redhat.com/show_bug.cgi?id=2243267#c13