Bug 1473307 - after RHN-H upgrade the firewalld.service is enabled and started
after RHN-H upgrade the firewalld.service is enabled and started
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: rhev-hypervisor-ng (Show other bugs)
4.1.0
x86_64 Linux
high Severity high
: ovirt-4.1.5
: ---
Assigned To: Ryan Barry
Huijuan Zhao
: Rebase, ZStream
: 1481624 1481924 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-20 08:51 EDT by Olimp Bockowski
Modified: 2017-08-22 13:44 EDT (History)
19 users (show)

See Also:
Fixed In Version: imgbased-0.9.42-0.1.el7ev
Doc Type: Bug Fix
Doc Text:
In order to enable services added in new layers, RHVH synchronizes systemd levels. Previously, if a default service was disabled by users, it was re-enabled when RHVH was upgraded. Now, systemd service status has improved synchronization logic when upgrades are performed, so disabled services stay disabled.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-08-22 13:44:45 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Node
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rbarry: needinfo-


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3153191 None None None 2017-08-16 08:42 EDT
oVirt gerrit 80034 master MERGED osupdater: sync systemd levels after remediation 2017-08-09 14:14 EDT
oVirt gerrit 80486 ovirt-4.1 MERGED osupdater: sync systemd levels after remediation 2017-08-09 14:50 EDT

  None (edit)
Description Olimp Bockowski 2017-07-20 08:51:16 EDT
Description of problem:
After RHN-H upgrade firewalld is enabled + started, what could lead to other issues (like for example LM due to not enabled libvirt-tls service)

Version-Release number of selected component (if applicable):
Upgrade from RHV-h 4.0* to RHV-H 4.1

How reproducible:
Not always, depends on the exact version of RHV-H-4.0.*

Steps to Reproduce:
1. Upgrade RHV-H

Actual results:

Sometimes firewalld is enabled and started, sometimes iptabales.service


Expected results:
iptables.service is present everytime


Additional info:

during the upgrade:

Here we just rsync the latest LiveOS to the latest layer
[DEBUG] Calling binary: (['mount', u'/tmp/mnt.lLwys/LiveOS/rootfs.img', u'/tmp/mnt.XLnpf'],) {}
[DEBUG] Calling binary: (['mount', u'/dev/rhvh/rhvh-4.1-0.20170616.0', u'/tmp/mnt.XzRlE'],) {}
[DEBUG] Running: ['ionice', 'rsync', '-pogAXlHrx', '-Sc', '--no-i-r', '-t', '--exclude', 'mnt.*/*', u'/tmp/mnt.XLnpf/', u'/tmp/mnt.XzRlE/']

Than we rsync /etc from 20170104 to 20170201
[DEBUG] Calling binary: (['mount', u'/dev/rhvh/rhvh-4.0-0.20170104.0+1', u'/tmp/mnt.YxXth'],) {}
[DEBUG] Calling binary: (['mount', u'/dev/rhvh/rhvh-4.0-0.20170201.0+1', u'/tmp/mnt.fmubK'],) {}
[DEBUG] Running: ['ionice', 'rsync', '-pogAXlHrx', '-Sc', '--no-i-r', '--checksum', '--update', '--exclude', 'mnt.*/*', u'/tmp/mnt.YxXth//etc/', u'/tmp/mnt.fmubK//etc']

Finally we rsync 20170201 to the latest 20170616
[DEBUG] Calling binary: (['mount', u'/dev/rhvh/rhvh-4.0-0.20170201.0+1', u'/tmp/mnt.eEJH0'],) {}
[DEBUG] Calling binary: (['mount', u'/dev/rhvh/rhvh-4.1-0.20170616.0+1', u'/tmp/mnt.WcAYi'],) {}
[DEBUG] Running: ['ionice', 'rsync', '-pogAXlHrx', '-Sc', '--no-i-r', '--checksum', '--update', '--exclude', 'mnt.*/*', u'/tmp/mnt.eEJH0//etc/', u'/tmp/mnt.WcAYi//etc']

That means that one of the layers contains the firewalld.service in the /etc The problem here is that we actually do the rsync from the previous layers. That is a nonsense we should copy just the last one.
There was a bug for this and it was fixed in next image (not reported one). The next image does not have firewalld enabled, BUT as it had been enabled in the past we always enable it even if the new image is fixed (because of rsync for all layers).
Comment 1 Ryan Barry 2017-07-21 08:13:17 EDT
We cannot simply copy /etc from the most recent layer (or even do a naive rsync), since there was a problem in the upgrade logic of RHVH 4.0 which would cause new versions of unmodified configuration files to be overwritten, which differs from the expected behavior of yum/etc.

We have a method method which explicitly checks whether systemd services are being accidentally re-enabled, but wasn't added to the code to remediate this bad update logic (RHVH 4.1 will go through all of the old layers and retroactively do the 'right thing').

I'll set up a reproducer to fix the systemd services when we do this also.
Comment 4 Huijuan Zhao 2017-07-27 02:53:33 EDT
QE can reproduce this issue.

Test version:
From: rhvh-4.0-0.20170201.0
To:   rhvh-4.1-0.20170616.0

# imgbase layout
rhvh-4.0-0.20170201.0
 +- rhvh-4.0-0.20170201.0+1
rhvh-4.1-0.20170616.0
 +- rhvh-4.1-0.20170616.0+1


Test steps:
1. Install rhvh-4.0-0.20170201.0
2. Login rhvh-4.0, check the iptables.service and firewalld.service
   # systemctl status iptables.service
   # systemctl status firewalld.service
3. Download upgrade rpm, and upgrade to rhvh-4.1-0.20170616.0
   # yum install redhat-virtualization-host-image-update-4.1-20170616.0.el7_3.noarch.rpm
4. Reboot rhvh and login rhvh-4.1, check iptables.service and firewalld.service
   # systemctl status iptables.service
   # systemctl status firewalld.service


Actual results:
1. After step 2,  firewalld.service is enabled and started, iptables.service is inactive.
# systemctl status iptables.service
● iptables.service - IPv4 firewall with iptables
   Loaded: loaded (/usr/lib/systemd/system/iptables.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

# systemctl status firewalld.service
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2017-07-27 05:34:13 GMT; 3min 12s ago
     Docs: man:firewalld(1)
 Main PID: 1218 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─1218 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

Jul 27 05:34:09 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Jul 27 05:34:13 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.

2. After step 4, check results are same as step 2, firewalld.service is enabled and started, iptables.service is inactive.


Expected results:
After step 4, iptables.service should be enabled and started, firewalld.service is inactive.


Additional info:
Same results as above when upgrade from rhvh-4.0-0.20170307.0 to rhvh-4.1-0.20170616.0, NetworkManager is also enabled.
Comment 6 Ryan Barry 2017-08-15 08:33:05 EDT
*** Bug 1481624 has been marked as a duplicate of this bug. ***
Comment 7 Ryan Barry 2017-08-16 08:42:21 EDT
*** Bug 1481924 has been marked as a duplicate of this bug. ***
Comment 9 Huijuan Zhao 2017-08-20 23:12:13 EDT
Test version:
From: rhvh-4.0-0.20170307.1
To:   rhvh-4.1-0.20170816.2
      imgbased-0.9.47-0.1.el7ev.noarch

# imgbase layout
rhvh-4.0-0.20170307.0
 +- rhvh-4.0-0.20170307.0+1
rhvh-4.1-0.20170817.0
 +- rhvh-4.1-0.20170817.0+1


Test steps:
1. Install rhvh-4.0-0.20170307.1
2. Register rhvh-4.0 to rhvm
3. Check iptables and firewalld in rhvh-4.0, iptables is active, firewalld is inactive.
   # systemctl status iptables
● iptables.service - IPv4 firewall with iptables
   Loaded: loaded (/usr/lib/systemd/system/iptables.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2017-08-21 01:55:30 GMT; 1min 11s ago
  Process: 1185 ExecStart=/usr/libexec/iptables/iptables.init start (code=exited, status=0/SUCCESS)
 Main PID: 1185 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/iptables.service

   # systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)
4. Setup local repos and update rhvh to rhvh-4.1-0.20170816.2
   # yum update
5. Reboot and login new layer, check iptables and firewalld in rhvh-4.1


Test results:
After step 5, iptables is active, firewalld is inactive.
# systemctl status iptables
● iptables.service - IPv4 firewall with iptables
   Loaded: loaded (/usr/lib/systemd/system/iptables.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2017-08-21 02:53:19 GMT; 2min 16s ago
  Process: 1217 ExecStart=/usr/libexec/iptables/iptables.init start (code=exited, status=0/SUCCESS)
 Main PID: 1217 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/iptables.service

Aug 21 02:53:17 localhost.localdomain systemd[1]: Starting IPv4 firewall with iptables...
Aug 21 02:53:19 localhost.localdomain iptables.init[1217]: iptables: Applying firewall rules: [  OK  ]
Aug 21 02:53:19 localhost.localdomain systemd[1]: Started IPv4 firewall with iptables.

# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)


Note: If rhvh NOT register to rhvm, the firewalld is active(iptables is inactive) in rhvh old layer, then after upgrade, the firewalld is still active(iptables is inactive) in new layer.


QE think this is reasonable, so change the status to VERIFIED.
Comment 12 errata-xmlrpc 2017-08-22 13:44:45 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2529

Note You need to log in before you can comment on or make changes to this bug.