Created attachment 1073968 [details] upgrade error screenshot Description of problem: Exception occurs when upgrade from 7.1 GA to 7.2 Version-Release number of selected component (if applicable): rhev-hypervisor-7-7.2-20150913.0 ovirt-node-3.2.3-20.el7.noarch How reproducible: 100% QA Whiteboard: node Regression bug Steps to Reproduce: 1. Clean install rhev-hypervisor-7-7.1-20150911.0 2. TUI Upgrade to rhev-hypervisor-7-7.2-20150913.0 step by step Actual results: after step2, there is error report: ------------------ Exception: System error("Error replacing '/etc/passwd': Device or resource busy",) ------------------ Expected results: after step2, upgrade should successful Additional info:
Created attachment 1073970 [details] upgrade error log
After setting new password during upgrade, reboot rhevh, the password did not change to new one.
*** This bug has been marked as a duplicate of bug 1246117 ***
Update test steps for more clearly. Test steps: 1. Clean install rhev-hypervisor-7-7.1-20150911.0 2. TUI Upgrade to rhev-hypervisor-7-7.2-20150913.0 3. Setting new password during upgrade, reboot. 4. Login RHEV-H with new password. Actual results: 1. after step2, there is error report: Exception: System error("Error replacing '/etc/passwd': Device or resource busy",) 2. After step 4, failed to login with new password. But old password still can work fine.
Likely a dupe of bug 1269840
Still encounter this issue on rhev-hypervisor-7-7.2-20151029.0. Test steps: 1. Clean install rhev-hypervisor-7-7.1-20151015.0 2. Deploy HE on rhevh 3. TUI Upgrade to rhev-hypervisor-7-7.2-20151029.0 4. Setting new password during upgrade. Actual results: 1. after step4, there is error report: Exception: System error("Error replacing '/etc/passwd': Device or resource busy",) Please refer to attachment: "upgrade failed log", "upgrade failed screenshot". So I will change the status to assigned.
Created attachment 1088542 [details] upgrade failed log
Created attachment 1088543 [details] upgrade failed screenshot
Taken from the ovirt-node.log file: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt/node/installer/core/progress_page.py", line 126, in __run File "/usr/lib/python2.7/site-packages/ovirt/node/plugins.py", line 188, in dry_or File "/usr/lib/python2.7/site-packages/ovirt/node/installer/core/progress_page.py", line 120, in do_commit File "/usr/lib/python2.7/site-packages/ovirt/node/installer/core/progress_page.py", line 266, in commit File "/usr/lib/python2.7/site-packages/ovirtnode/password.py", line 45, in set_password SystemError: Error replacing `/etc/passwd': Device or resource busy
Okay, I see that ovirt-node-3.2.3-20.el7.noarch is used in the tested build. This build is known to be broken wrt to persistence. Please try a 3.5 build with at least ovirt-node-3.2.3-24 (we need the "ntp" patch a380c6be248c88e68c8b278bcbe278c5bceb7abc), the latest 3.5.6 build should have that ovirt-node build.
Tested build rhev-hypervisor-7-7.2-20151029.0, have the ovirt-node-3.2.3-26.el7.noarch
(In reply to Huijuan Zhao from comment #13) > Tested build rhev-hypervisor-7-7.2-20151029.0, have the > ovirt-node-3.2.3-26.el7.noarch Hi Huijuan, Could you please clarify the last comment? Still happening to you?
Yes, it was still happening with the new build as well, see comment 7.
I have reproduced the report: Installed rhev-hypervisor-7-7.1-20151015.0 and upgraded to rhev-hypervisor-7-7.2-20151029.0 via Text User Interface. However, I would like to share the workaround for now, upgrade via RHEV-M or ovirt-node-upgrade tool. Upgraded from rhev-hypervisor7-7.1-20150911.0.iso to rhev-hypervisor-7-7.2-20150913.0.iso worked without any issue. or Upgraded from rhev-hypervisor-7-7.1-20151015.0.iso to rhev-hypervisor-7-7.2-20151029.0.iso worked without any issue. Huijuan, Could you please confirm the workaround also worked for you? Thanks!
Hi, Douglas The workaround you shared: upgrade via RHEV-M or ovirt-node-upgrade tool also worked without any issue in my test. Just upgrade via TUI failed with error report: Upgraded from rhev-hypervisor7-7.1-20150911.0.iso to rhev-hypervisor-7-7.2-20150913.0.iso via TUI failed. or Upgraded from rhev-hypervisor-7-7.1-20151015.0.iso to rhev-hypervisor-7-7.2-20151029.0.iso via TUI failed. The error report "System error("Error replacing '/etc/passwd': Device or resource busy",)" likely caused by setup adminpw during TUI upgrade. Upgrade via RHEV-M or ovirt-node-upgrade tool will skip setup adminpw, so that worked well.
Reducing the priority for now as we have workaround. The only affects cases where users want to upgrade *and* also change the admin password.
Ying, I tested the following cases each for 5 times, detailed info please refer to following: 1. TUI Upgrade AND change password during upgrade Failed with error report. How reproducible: 100% 2. Automatic Upgrade AND change password during upgrade Without any issue, upgrade successful. 3. TUI Upgrade AND _AFTERWARDS_ change password in TUI Failed with error report during TUI upgrade. After TUI upgrade, reboot, change password in TUI successful without any issue. 4. Automatic Upgrade AND _AFTERWARDS_ change password in TUI Without any issue, upgrade successful. Change password in TUI successful.
The problem seems to be that udevd on RHEL 7.2 is using a slave subtree. That,s fine. But all mounts within that subtree should become slaves - this is especially important for our config bind-mounts in /etc. This patch should solve it: https://github.com/systemd/systemd/pull/2050
Still encounter this issue in rhev-hypervisor-7-7.2-20151210.1 Test version: rhev-hypervisor-7-7.2-20151210.1.e17ev ovirt-node-3.6.0-0.24.20151209gitc0fa931.el7ev.noarch Test steps: 1. TUI install rhev-hypervisor-7-7.1-20151015.0 2. TUI Upgrade to rhev-hypervisor-7-7.2-20151210.1 3. Setting new password during upgrade, reboot. 4. Login RHEV-H with new password. Actual results: 1. after step2, there is error report: Exception: System error("Error replacing '/etc/passwd': Device or resource busy",) 2. After step 4, failed to login with new password. But old password still can work fine. So this bug is not fix in rhev-hypervisor-7-7.2-20151210.1.e17ev, change the status to assigned.
(In reply to Huijuan Zhao from comment #29) > > 2. After step 4, failed to login with new password. But old password still > can work fine. This reminds me of one Bug 1251867 - Failed to reset the initial password after RHEV-H automatic installation. The bug 1251867 was closed dup of bug 1246117, but the bug 1246117 was verified. Huijuan, please check the bug 1251867, if the bug is you mentioned in comment 29, we have to reopen the bug 1251867, too.
Ying, I checked the bug 1251867, no issue in rhev-hypervisor-7-7.2-20151210.1.e17ev. Test Steps: 1. PXE install rhev-hypervisor-7-7.2-20151210.1.e17ev with arguments: management_server=10.66.108.1 BOOTIF=em1 adminpw=OKr05SbCu3D3g storage_init=/dev/sda reinstall ssh_pwauth=1 2. After the system install successful, approve the system on RHEV-M and the system status is up. 3. Input the initial UNIX password and set the new password successful, and login the TUI successful, reboot the system. 4. In the login session, input the new password, successful to login the TUI. Test results: In step4, can login with the new password So bug 1251867 is fixed in rhev-hypervisor-7-7.2-20151210.1.e17ev.
The remaining problem seemed to be that the mount propagation of /etc got broken. Huijuan, can you please try if the following steps are fixing the issues for you: 1. Boot ISO 2. Do the first install 3. Reboot 4. Boot from ISO again 5. In installer, hit F2, dro pto shell 6. Run: mount --make-rshared /etc 7. Run: exit - to get back into installer 8. Run update/installation/reinstallation and set a new password After step 8: No error is shown
Fabian, I tried the steps as Comment 32, after step 8: No error is shown. Test version: rhev-hypervisor-7-7.2-20151210.1.e17ev ovirt-node-3.6.0-0.24.20151209gitc0fa931.el7ev.noarch Test steps: 1. Boot ISO (rhev-hypervisor-7-7.2-20151129.1) 2. Do the first install 3. Reboot 4. Boot from ISO again (rhev-hypervisor-7-7.2-20151210.1) 5. In installer, hit F2, drop to shell 6. Run: mount --make-rshared /etc 7. Run: exit - to get back into installer 8. Run update and set a new password Test results: After step 8: No error is shown
Great, thanks for testing, we'll merge the patch shortly.
Test Version: rhev-hypervisor-7-7.2-20151229.0 ovirt-node-3.6.1-1.0.el7.noarch Steps to Reproduce: 1. Clean install rhev-hypervisor-7-7.2-20151129.1 2. TUI Upgrade to rhev-hypervisor-7-7.2-20151229.0 3. Setting new password during upgrade, reboot. 4. Login RHEV-H with new password. Test results: Upgrade successful, login RHEV-H with new password successful. So this bug is fixed on rhev-hypervisor-7-7.2-20151229.0, change the status to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0378.html