Bug 1418179 - [z-stream clone - 4.0.7] unmodified configuration files should be updated during update.
Summary: [z-stream clone - 4.0.7] unmodified configuration files should be updated dur...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-node-ng
Version: 4.0.6
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ovirt-4.0.7
: ---
Assignee: Ryan Barry
QA Contact: Huijuan Zhao
URL:
Whiteboard:
Depends On: 1417534
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-01 07:29 UTC by rhev-integ
Modified: 2017-03-16 15:39 UTC (History)
16 users (show)

Fixed In Version: imgbased-0.8.12-0.1.el7ev
Doc Type: Bug Fix
Doc Text:
Previously, imgbased blindly copied /etc from old layers into new layers in order to keep configuration changes between upgrades. This meant that imgbased's behavior differed from RPM, in that unmodified configuration files would be preserved across imgbased upgrades whereas 'yum upgrade' of the same packages would have replaced them. Now, imgbased compares the sums of files to the originals kept per-layer in /usr/share/factory/etc so that unmodified configuration files are now handled appropriately.
Clone Of: 1417534
Environment:
Last Closed: 2017-03-16 15:39:32 UTC
oVirt Team: Node
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0549 normal SHIPPED_LIVE Moderate: redhat-virtualization-host security and bug fix update 2017-03-16 19:26:25 UTC
oVirt gerrit 71403 None None None 2017-02-01 07:30:12 UTC

Description rhev-integ 2017-02-01 07:29:31 UTC
+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1417534 +++
======================================================================

Description of problem:

As we have seen on bugzilla #1416278, many configurations files are not updated when a new image is installed.

How reproducible:
100%

Steps to Reproduce:
1. Install RHVH 4.0.4, check timestamps/contents of configuration files
   * /etc/vdsm for example
2. Attach entitlements, update to 4.0-20170104.1 and reboot
3. Compare timestamps/contents of configuration files

A notable file between these two versions is logger.conf, 4.0.6 and 4.0.4 contents are different, but the upgraded node doesn't have the newer configurations.

Pretty much every config file is with the older image timestamp after upgrade.

# ll /etc/vdsm/
total 32
-rw-r--r--. 1 root root  1747 Sep  7 21:14 logger.conf
drwxr-xr-x. 2 root root    17 Oct 13 00:02 logrotate
-rw-r--r--. 1 root root  3161 Sep  7 21:14 mom.conf
drwxr-xr-x. 2 root root  4096 Oct 13 00:02 mom.d
-rw-r--r--. 1 root root   619 Sep  7 21:14 svdsm.logger.conf
-rw-r--r--. 1 root root 12725 Sep  7 21:14 vdsm.conf
drwxr-xr-x. 2 root root     6 Sep  7 21:14 vdsm.conf.d

Whereas in a fresh 4.0-20170104.1 install:

# ll /etc/vdsm/
total 40
-rw-r--r--. 1 root root  1825 Dec 14 20:52 logger.conf
drwxr-xr-x. 2 root root  4096 Jan  5 01:33 logrotate
-rw-r--r--. 1 root root  3166 Dec 14 20:52 mom.conf
drwxr-xr-x. 2 root root  4096 Jan  5 01:33 mom.d
-rw-r--r--. 1 root root   619 Dec 14 20:52 svdsm.logger.conf
-rw-r--r--. 1 root root 12655 Dec 14 20:52 vdsm.conf
drwxr-xr-x. 2 root root  4096 Dec 14 20:52 vdsm.conf.d

Version-Release number of selected component (if applicable):
4.0-20170104.1

Expected results:
Files should be rolled forward to the latest version.

Additional info:
Files were not manually modified

(Originally by Germano Veit Michel)

Comment 1 rhev-integ 2017-02-01 07:29:41 UTC
I just want to point out a potential risk here --

Systems upgraded from:

4.0.4 -> 4.0.6 -> some other version

Will not carry this change across. In a manner similar to the way RPM treats .rpmnew files, we can compare against a "good" file (in /usr/share/factory, which is put aside when the image is built). However, a 4.0.6 system will see logger.conf as already modified, and any upgrades from that image will keep holding onto the old logger.conf.

We can also check "rpm -V", but will see the same result -- logger.conf already differs.

I'll try to think of a way to work around this, but we'll definitely need a release note.

(Originally by Ryan Barry)

Comment 5 rhev-integ 2017-02-01 07:29:55 UTC
Example:

* User installs RHV 4.0.4
* User upgrades to 4.0.6 (with a new logger.conf)
* The upgrade copies all of /etc
* logger.conf now differs from what is in the rpmdb and /usr/share/factory/etc
* User upgrades to 4.0.6-1 or 4.0.7
* logger.conf (and potentially other files) differ from what's in the rpmdb because they were blindly copied, even though they were never modified, and will not be copied by the new patch.
* In this case, relying on the rpmdb is not reliable, because the file is modified.

The difficulty is that it was modified by imgbased copying the file, but differentiating this from an intentional change programatically is almost impossible. We have the option of going back through the rpmdb on old imgbased layers to see if the files match the rpmdb on any of those, but this seems risky, and it adds a lot of runtime to the upgrade.

I'm still investigating to see whether the tradeoff of going through old layers is worth the additional runtime/complexity.

(Originally by Ryan Barry)

Comment 7 Huijuan Zhao 2017-02-09 02:44:23 UTC
I can reproduce this issue.

Test version:
1. Before upgrade:
redhat-virtualization-host-4.0-20160817.0
2. After upgrade:
redhat-virtualization-host-4.0-20170201.0
imgbased-0.8.11-0.1.el7ev.noarch
kernel-3.10.0-514.2.2.el7.x86_64

Test steps:
1. Install RHVH-4.0-20160817.0, check timestamps/contents of configuration files
   * /etc/vdsm for example
2. Login RHVH and setup local repos, update to RHVH-4.0-20170201.0 and reboot
3. Compare timestamps/contents of configuration files
   # ll /etc/vdsm
4. Clean install RHVH-4.0-20170201.0, check timestamps/contents of configuration files
   # ll /etc/vdsm

Actual results:
1. After step3,
# ll /etc/vdsm/
total 32
-rw-r--r--. 1 root root  1747 Aug 10 17:48 logger.conf
drwxr-xr-x. 2 root root    17 Aug 17 20:16 logrotate
-rw-r--r--. 1 root root  3161 Aug 10 17:48 mom.conf
drwxr-xr-x. 2 root root  4096 Aug 17 20:16 mom.d
-rw-r--r--. 1 root root   619 Aug 10 17:48 svdsm.logger.conf
-rw-r--r--. 1 root root 12622 Aug 10 17:48 vdsm.conf
drwxr-xr-x. 2 root root     6 Aug 10 17:48 vdsm.conf.d

2. After step4,
# ll /etc/vdsm
total 40
-rw-r--r--. 1 root root  1825 Jan 31 14:28 logger.conf
drwxr-xr-x. 2 root root  4096 Feb  1 22:18 logrotate
-rw-r--r--. 1 root root  3166 Jan 31 14:28 mom.conf
drwxr-xr-x. 2 root root  4096 Feb  1 22:18 mom.d
-rw-r--r--. 1 root root   619 Jan 31 14:28 svdsm.logger.conf
-rw-r--r--. 1 root root 12655 Jan 31 14:28 vdsm.conf
drwxr-xr-x. 2 root root  4096 Jan 31 14:28 vdsm.conf.d

Expected results:
The timestamps/contents of configuration files should be same between step3 and step4.

Comment 11 jianwu 2017-02-23 09:47:39 UTC
Hi 
I can still reproduce this issue.

Test version:
Before upgrade:
redhat-virtualization-host-4.0-20161116.1.x86_64(el7.3)
After upgrade:
redhat-virtualization-host-4.0-20170222.0.x86_64
kernel-3.10.0-514.6.2.el7.x86_64
imgbased-0.8.13-0.1.el7ev.noarch

Test steps:
1. Install RHVH-4.0-20161116.1, check timestamps/contents of configuration files
   * /etc/vdsm for example
2. Login RHVH and setup local repos, update to redhat-virtualization-host-4.0-20170222.0 and reboot
3. Compare timestamps/contents of configuration files
   # ll /etc/vdsm
4. Clean install redhat-virtualization-host-4.0-20170222.0, check timestamps/contents of configuration files
   # ll /etc/vdsm

Actual results:
After step 3:
#ls -al /etc/vdsm
total 48
total 40
drwxr-xr-x.   5 root root   132 Nov 17 02:41 .
drwxr-xr-x. 111 root root  8192 Feb 23  2017 ..
-rw-r--r--.   1 root root  1825 Nov 11 19:25 logger.conf
drwxr-xr-x.   2 root root    18 Nov 17 02:41 logrotate
-rw-r--r--.   1 root root  3166 Feb 16 00:09 mom.conf
drwxr-xr-x.   2 root root   154 Nov 17 02:41 mom.d
-rw-r--r--.   1 root root   619 Nov 11 19:25 svdsm.logger.conf
-rw-r--r--.   1 root root 12655 Feb 16 00:09 vdsm.conf
drwxr-xr-x.   2 root root     6 Nov 11 19:25 vdsm.conf.d

After step 4:
#ls -al /etc/vdsm
total 56
drwxr-xr-x.   5 root root  4096 Feb 23 06:39 .
drwxr-xr-x. 111 root root 12288 Feb 23 13:53 ..
-rw-r--r--.   1 root root  1825 Feb 16 00:09 logger.conf
drwxr-xr-x.   2 root root  4096 Feb 23 06:39 logrotate
-rw-r--r--.   1 root root  3166 Feb 16 00:09 mom.conf
drwxr-xr-x.   2 root root  4096 Feb 23 06:39 mom.d
-rw-r--r--.   1 root root   619 Feb 16 00:09 svdsm.logger.conf
-rw-r--r--.   1 root root 12655 Feb 16 00:09 vdsm.conf
drwxr-xr-x.   2 root root  4096 Feb 16 00:09 vdsm.conf.d

Expected results:
The timestamps/contents of configuration files should be same between step3 and step4.

So I think this bug is not fixed on redhat-virtualization-host-4.0-20170222.0.x86_64, I will change status to ASSIGNED.

Jiawu

Comment 12 Ryan Barry 2017-02-23 18:03:54 UTC
The contents of the actual files are the same.

The difference in the sizes of the folder is due to XFS block directories:
http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Block_Directories.html

Directory sizing is tricky, and we can't guarantee that XFS will treat this exactly the same on a copy, since XFS has at least 3 different on-disk representations of directories (file, block, inode), depending on what the filesystem determines is most efficient when allocation happens.

Please use a different bug to track the timestamps (one which will not get a z-stream).

The md5sums of 

redhat-virtualization-host-4.0-20161116.1+1:/etc/vdsm/logger.conf 
redhat-virtualization-host-4.0-20170222.0+1:/etc/vdsm/logger.conf 

Are identical. The comparison in imgbased is using sums, and not also checking modification times. This is something which we can do, but it's a very minor issue compared to the files themselves.

diff --brief -r /etc /usr/share/factory/etc 2>/dev/null | grep -v Only

Should show appropriate results (only files which were actually modified, either as part of install or by you) should differ.

Comment 13 Huijuan Zhao 2017-02-24 11:11:19 UTC
According to Comment 12, I tested again with the below version, the contents of the actual files in /etc/vdsm/ are the same.

Test version:
From:
RHVH-4.0-20160919.1-RHVH-x86_64-dvd1.iso
To:
redhat-virtualization-host-4.0-20170222.0.x86_64
imgbased-0.8.13-0.1.el7ev.noarch

Test steps:
1. Install RHVH-4.0-20160919.1-RHVH-x86_64-dvd1.iso, check timestamps/contents of configuration files
   * /etc/vdsm for example
2. Login RHVH and setup local repos, update to redhat-virtualization-host-4.0-20170222.0 and reboot
3. Check timestamps/contents of configuration files
   # ll /etc/vdsm
4. Clean install redhat-virtualization-host-4.0-20170222.0, check timestamps/contents of configuration files
   # ll /etc/vdsm

Test results:
diff the files in /etc/vdsm/, step 3 and step4 are the same, step3 and step1 are different. So the unmodified content is updated during upgrade.

But timestamps are not same between step 3 and step4:
After step 1:
# ls -lh /etc/vdsm
total 32K
-rw-r--r--. 1 root root 1.8K Sep  7 11:14 logger.conf
drwxr-xr-x. 2 root root   17 Sep 19 20:16 logrotate
-rw-r--r--. 1 root root 3.1K Sep  7 11:14 mom.conf
drwxr-xr-x. 2 root root 4.0K Sep 19 20:16 mom.d
-rw-r--r--. 1 root root  619 Sep  7 11:14 svdsm.logger.conf
-rw-r--r--. 1 root root  13K Sep  7 11:14 vdsm.conf
drwxr-xr-x. 2 root root    6 Sep  7 11:14 vdsm.conf.d

After step3:
# ls -lh /etc/vdsm/
total 32K
-rw-r--r--. 1 root root 1.8K Feb 15 16:09 logger.conf
drwxr-xr-x. 2 root root   17 Sep 19 20:16 logrotate
-rw-r--r--. 1 root root 3.1K Feb 15 16:09 mom.conf
drwxr-xr-x. 2 root root 4.0K Sep 19 20:16 mom.d
-rw-r--r--. 1 root root  619 Sep  7 11:14 svdsm.logger.conf
-rw-r--r--. 1 root root  13K Feb 15 16:09 vdsm.conf
drwxr-xr-x. 2 root root    6 Sep  7 11:14 vdsm.conf.d

After step4:
# ls -lh /etc/vdsm
total 40K
-rw-r--r--. 1 root root 1.8K Feb 15 16:09 logger.conf
drwxr-xr-x. 2 root root 4.0K Feb 22 22:39 logrotate
-rw-r--r--. 1 root root 3.1K Feb 15 16:09 mom.conf
drwxr-xr-x. 2 root root 4.0K Feb 22 22:39 mom.d
-rw-r--r--. 1 root root  619 Feb 15 16:09 svdsm.logger.conf
-rw-r--r--. 1 root root  13K Feb 15 16:09 vdsm.conf
drwxr-xr-x. 2 root root 4.0K Feb 15 16:09 vdsm.conf.d

According to comment 12, timestamps issue will be tracked by Bug 1417534.

So I will verify this bug.

Comment 15 Yaniv Kaul 2017-03-16 13:17:12 UTC
Is it in the errata?

Comment 16 Ryan Barry 2017-03-16 13:55:45 UTC
Yep

Comment 18 errata-xmlrpc 2017-03-16 15:39:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0549.html


Note You need to log in before you can comment on or make changes to this bug.