Bug 1614971 - Upgrading RHV-H from 4.0.X to 4.2 is failing during migrate_var
Summary: Upgrading RHV-H from 4.0.X to 4.2 is failing during migrate_var
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: imgbased
Version: 4.2.5
Hardware: All
OS: Linux
medium
medium
Target Milestone: ovirt-4.2.7
: ---
Assignee: Yuval Turgeman
QA Contact: Yaning Wang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-08-10 21:36 UTC by nijin ashok
Modified: 2022-07-09 10:08 UTC (History)
15 users (show)

Fixed In Version: imgbased-1.0.25
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-05 15:00:05 UTC
oVirt Team: Node
Target Upstream Version:
Embargoed:
peyu: testing_plan_complete+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-38462 0 None None None 2021-09-09 15:22:43 UTC
Red Hat Knowledge Base (Solution) 3588701 0 None None None 2018-08-28 10:19:03 UTC
Red Hat Product Errata RHSA-2018:3470 0 None None None 2018-11-05 15:02:34 UTC
oVirt gerrit 93745 0 master MERGED osupdater: check if src file exists before copy 2020-08-13 07:01:43 UTC
oVirt gerrit 94046 0 ovirt-4.2 MERGED osupdater: check if src file exists before copy 2020-08-13 07:01:42 UTC

Description nijin ashok 2018-08-10 21:36:31 UTC
Description of problem:

The 4.0 RHV-H is not having glusterfs-server installed. So the file "/usr/libexec/glusterfs/glusterfind/S57glusterfind-delete-post.py" doesn't exist in 4.0 which come from this package.

While upgrading from 4.0 to 4.2, we will sync items present in the new 4.2 /var/ to the existing /var/ filesystem. Since "/var/lib/glustersd" directory is not present in the existing var filesystem, it will be synced.

===
2018-08-10 19:53:18,391 [DEBUG] (migrate_var) Copying /tmp/mnt.6fSmc//var/lib/glusterd to /var/lib/glusterd
===

We have symlink "/var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post" which points to "/usr/libexec/glusterfs/glusterfind/S57glusterfind-delete-post.py" under this directory. Since file S57glusterfind-delete-post.py doesn't exist in the 4.0 server, this symlink will be a often/broken.

===
[root@ ~]# imgbase w
[INFO] You are on rhvh-4.0-0.20170104.0+1

[root@ ~]# mount /dev/rhvh/rhvh-4.2.5.1-0.20180801.0+1 /tmp/rhvh-4.2
[root@ ~]# cd /tmp/rhvh-4.2
[root@ rhvh-4.2]# ls -l var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post
lrwxrwxrwx. 1 root root 64 Aug  1 14:05 var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post -> /usr/libexec/glusterfs/glusterfind/S57glusterfind-delete-post.py

rhvh-4.2]# python
Python 2.7.5 (default, Aug  2 2016, 04:20:16) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.exists("/tmp/rhvh-4.2/var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post")
False
====

As per https://gerrit.ovirt.org/#/c/92376/, we are syncing files as well after syncing the directory. Although the link 
"/var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post" is already copied to existing /var/ as a part of "directory" sync, the imgbased will try to sync it again since it will fail the "os.path.exists" check as it's a broken link. This copy will fail with "No such file or directory:" since source file doesn't exist.


===
2018-08-10 19:53:18,601 [DEBUG] (migrate_var) Copying /tmp/mnt.6fSmc//var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post to /var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post

2018-08-10 19:53:29,742 [DEBUG] (MainThread) Traceback (most recent call last):
  File "/tmp/tmp.ybG3syJOJ9/usr/lib/python2.7/site-packages/imgbased/utils.py", line 1491, in run
    self._function(*self._args, **self._kwargs)
  File "/tmp/tmp.ybG3syJOJ9/usr/lib/python2.7/site-packages/imgbased/plugins/osupdater.py", line 245, in migrate_var
    shutil.copy2(newlv_path, realpath)
  File "/usr/lib64/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib64/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: u'/tmp/mnt.6fSmc//var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post'
====

Hence the upgrade will fail.


Version-Release number of selected component (if applicable):

imgbased-1.0.22

How reproducible:

100%

Steps to Reproduce:

Upgrade RHV-H 4.0 directly to 4.2.

Actual results:

The upgrade is failing during "var" migration

Expected results:

The upgrade should work fine.

Additional info:

Comment 2 Yuval Turgeman 2018-08-15 08:32:56 UTC
Thanks for the detailed description, Nijin !

Comment 3 cshao 2018-09-13 12:11:04 UTC
Test version:
rhvh-4.0-0.20170104.0+1
rhvh-4.2.5.1-0.20180801.0

Test steps:
1. Install rhvh-4.0-0.20170104.0+1
2. Upgrade to rhvh-4.2.5.1-0.20180801.0
3. Reboot and login.

Test result:
The upgrade is failing, only can login to RHVH 4.0.


# imgbase  w 
[INFO] You are on rhvh-4.0-0.20170104.0+1
# mount /dev/rhvh/rhvh-4.2.5.1-0.20180801.0+1 /tmp/rhvh-4.2
# cd /tmp/rhvh-4.2
# ls -l var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post
lrwxrwxrwx. 1 root root 64 Aug  1 04:35 var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post -> /usr/libexec/glusterfs/glusterfind/S57glusterfind-delete-post.py



# python
Python 2.7.5 (default, Aug  2 2016, 04:20:16) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.exists("/tmp/rhvh-4.2/var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post")
False
>>>

Comment 5 Yaning Wang 2018-09-25 08:57:11 UTC
Test version:

rhvh-4.0-0.20170104.0+1
rhvh-4.2.7.0-0.20180918.0+1

Test steps:
1. Install rhvh-4.0-0.20170104.0+1
2. Upgrade to rhvh-4.2.5.1-0.20180801.0
3. Reboot and login.

Test result:
upgrade success


[root@dhcp-8-129 ~]# imgbase w
You are on rhvh-4.2.7.0-0.20180918.0+1
[root@dhcp-8-129 ~]# imgbase layout
rhvh-4.0-0.20170104.0
 +- rhvh-4.0-0.20170104.0+1
rhvh-4.2.7.0-0.20180918.0
 +- rhvh-4.2.7.0-0.20180918.0+1

Comment 8 errata-xmlrpc 2018-11-05 15:00:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3470

Comment 9 Daniel Gur 2019-08-28 13:13:00 UTC
sync2jira

Comment 10 Daniel Gur 2019-08-28 13:17:12 UTC
sync2jira


Note You need to log in before you can comment on or make changes to this bug.