Bug 1649658 - RHV-H upgrade from 4.1 to 4.2 will fail with error "not writing through dangling symlink" if server is registered to insight
Summary: RHV-H upgrade from 4.1 to 4.2 will fail with error "not writing through dangl...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-node-ng
Version: 4.2.7
Hardware: All
OS: Linux
urgent
high
Target Milestone: ovirt-4.2.8
: ---
Assignee: Yuval Turgeman
QA Contact: Huijuan Zhao
Rolfe Dlugy-Hegwer
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-11-14 07:35 UTC by nijin ashok
Modified: 2021-12-10 18:27 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: imgbased tries to override dangling symlinks to redhat-access-insights. Consequence: RHVH upgrade fails when trying to copy over the dangling symlinks. Fix: Copy /etc/redhat-access-insights files from previous layer to /etc/insights-client. Result: The files are placed in the correct location, and the symlinks point to the updated files correctly.
Clone Of:
Environment:
Last Closed: 2019-01-22 12:44:15 UTC
oVirt Team: Node
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-44341 0 None None None 2021-12-10 18:27:05 UTC
Red Hat Product Errata RHBA-2019:0116 0 None None None 2019-01-22 12:44:20 UTC
oVirt gerrit 95731 0 master MERGED osupdater: copy insights files to new location 2020-08-11 09:49:42 UTC
oVirt gerrit 95782 0 ovirt-4.2 MERGED osupdater: copy insights files to new location 2020-08-11 09:49:42 UTC

Description nijin ashok 2018-11-14 07:35:04 UTC
Description of problem:

The rootfs of 4.2 ISO is having a dangling symlink on the "/etc/redhat-access-insights/machine-id" file.

===
mount usr/share/redhat-virtualization-host/image/redhat-virtualization-host-4.2-20181026.0.el7_6.squashfs.img squashfs

mount squashfs/LiveOS/rootfs.img rootfs

ls -l rootfs/etc/redhat-access-insights/machine-id 
lrwxrwxrwx. 1 root root 31 Oct 26 12:15 rootfs/etc/redhat-access-insights/machine-id -> /etc/insights-client/machine-id

ls -l rootfs/etc/insights-client/machine-id
ls: cannot access 'rootfs/etc/insights-client/machine-id': No such file or directory
===

From 7.5, redhat-access-insights is being deprecated and has been replaced with insights-client. Refer https://access.redhat.com/articles/3405441. So the 4.2 ISO will be having insights-client and 4.1 is having redhat-access-insights.  

The 4.1 is not having this symlink because it's having redhat-access-insights package.

===
ls -l /etc/redhat-access-insights/machine-id
-rw-r--r--. 1 root root 36 Dec  7  2017 /etc/redhat-access-insights/machine-id
===

Now when we do the upgrade, the imgbased will try to copy the /etc/redhat-access-insights/machine-id from 4.1 to 4.2 and it will fail with the error below since destination file is a dangling symlink.

====
2018-11-14 06:56:45,467 [DEBUG] (migrate_etc) Calling binary: (['mount', u'/dev/rhvh/rhvh-4.1-0.20170417.0+1', u'/tmp/mnt.uinB1'],) {}
2018-11-14 06:56:45,310 [DEBUG] (migrate_etc) Calling binary: (['mount', u'/dev/rhvh/rhvh-4.2.7.3-0.20181026.0+1', u'/tmp/mnt.avsbD'],) {}


2018-11-14 06:57:12,730 [DEBUG] (migrate_etc) Calling: (['cp', '-a', '-r', u'/tmp/mnt.uinB1///etc/redhat-access-insights/machine-id', u'/tmp/mnt.avsbD///etc/redhat-access-insights/machine-id'],) {'close_fds': True, 'stderr': -2}

2018-11-14 06:57:12,734 [DEBUG] (migrate_etc) Exception! cp: not writing through dangling symlink '/tmp/mnt.avsbD///etc/redhat-access-insights/machine-id'

===


Version-Release number of selected component (if applicable):

imgbased-1.0.29


How reproducible:

100%

Steps to Reproduce:

1. In a 4.1 RHV-H, register to insights using command "redhat-access-insights --register". This will generate file "/etc/redhat-access-insights/machine-id" file.

2. Change the date of machine-id file.

touch -d 20171207 /etc/redhat-access-insights/machine-id

This is required to reproduce the issue or else rsync will correct the symlink when it does copying from 4.1 layer to 4.2 layer as we do rsync with --update and the destination machine-id will be having an older timestamp compared to the file in source 4.1 layer.   

I set the same date as customer machine-id.

3. Do the update. It will fail with the error below.

===
2018-11-14 06:57:13,722 [DEBUG] (MainThread) Traceback (most recent call last):
  File "/tmp/tmp.lwoc2tT3YW/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1521, in thread_group_handler
    t.join_with_exceptions()
  File "/tmp/tmp.lwoc2tT3YW/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1513, in join_with_exceptions
    raise exc[1]
CalledProcessError: Command '['cp', '-a', '-r', u'/tmp/mnt.uinB1///etc/redhat-access-insights/machine-id', u'/tmp/mnt.avsbD///etc/redhat-access-insights/machine-id']' returned non-zero exit status 1

2018-11-14 06:57:13,722 [ERROR] (MainThread) Failed to migrate etc
Traceback (most recent call last):
  File "/tmp/tmp.lwoc2tT3YW/usr/lib/python2.7/site-packages/imgbased/plugins/osupdater.py", line 130, in on_new_layer
    thread_group_handler(threads)
  File "/tmp/tmp.lwoc2tT3YW/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1524, in thread_group_handler
    sys.exit(1)
SystemExit: 1
===


Actual results:

RHV-H upgrade from 4.1 to 4.2 is failing with error "not writing through dangling symlink" if the server is registered to insight

Expected results:

Upgrade should work.

Additional info:

Comment 3 Yuval Turgeman 2018-11-15 15:03:53 UTC
Ok, I didn't read this correctly, we could just ship the image without the dangling symlinks, and rerun the post and posttrans scripts for insights-client on update

Comment 4 Huijuan Zhao 2018-11-16 05:14:14 UTC
QE can reproduce this issue.

Test version:
Build 1: rhvh-4.1-0.20170417.0
Build 2: rhvh-4.2.7.3-0.20181026.0

Test steps:
1. Install rhvh-4.1-0.20170417.0
2. Check redhat-access-insights rpm installed on rhvh
3. Register rhvh to RHSM
4. Register rhvh to insights
# redhat-access-insights --register
5. Change the date of machine-id file
touch -d 20171207 /etc/redhat-access-insights/machine-id
6. Upgrade rhvh to rhvh-4.2.7.3-0.20181026.0

Actual results:
After step 6, upgrade failed.

Expected results:
After step 6, upgrade should be successful.

Comment 5 nijin ashok 2018-11-21 07:23:43 UTC
(In reply to Yuval Turgeman from comment #3)
> Ok, I didn't read this correctly, we could just ship the image without the
> dangling symlinks, and rerun the post and posttrans scripts for
> insights-client on update

I didn't have much knowledge about insights-client. I think ISO without symlinks would be fine. Also, note that it's not only "machine-id" and there are other dangling links as well.

ls -laR rootfs/etc/redhat-access-insights/
rootfs/etc/redhat-access-insights/:
total 16
drwxr-xr-x.   2 root root  4096 Oct 26 12:15 .
drwxr-xr-x. 129 root root 12288 Oct 26 12:16 ..
lrwxrwxrwx.   1 root root    32 Oct 26 12:15 .lastupload -> /etc/insights-client/.lastupload
lrwxrwxrwx.   1 root root    31 Oct 26 12:15 machine-id -> /etc/insights-client/machine-id
lrwxrwxrwx.   1 root root    41 Oct 26 12:15 redhat-access-insights.conf -> /etc/insights-client/insights-client.conf
lrwxrwxrwx.   1 root root    41 Oct 26 12:15 redhat-access-insights.cron -> /etc/insights-client/insights-client.cron
lrwxrwxrwx.   1 root root    32 Oct 26 12:15 .registered -> /etc/insights-client/.registered
lrwxrwxrwx.   1 root root    34 Oct 26 12:15 .unregistered -> /etc/insights-client/.unregistered

Comment 6 Huijuan Zhao 2018-11-29 08:42:41 UTC
This bug is fixed in redhat-virtualization-host-4.2-20181128.0

Test version:
# imgbase layout
rhvh-4.1-0.20180425.0
 +- rhvh-4.1-0.20180425.0+1
rhvh-4.2.8.0-0.20181127.0
 +- rhvh-4.2.8.0-0.20181127.0+1

Test steps:
Same as comment 4

Test results:
After step6, upgrade successful

I will verify this bug once the status move to ON_QA

Comment 7 Huijuan Zhao 2018-12-04 02:25:34 UTC
According to comment 6, move to VERIFIED

Comment 9 errata-xmlrpc 2019-01-22 12:44:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0116


Note You need to log in before you can comment on or make changes to this bug.