Bug 1557200
Summary: | Need add a check for swap configuration for upgrade | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Weihua Meng <wmeng> |
Component: | Cluster Version Operator | Assignee: | Russell Teague <rteague> |
Status: | CLOSED ERRATA | QA Contact: | Weihua Meng <wmeng> |
Severity: | medium | Docs Contact: | |
Priority: | medium | ||
Version: | 3.9.0 | CC: | aos-bugs, jiajliu, jialiu, jokerman, mmccomas, rteague, sdodson |
Target Milestone: | --- | Keywords: | Triaged |
Target Release: | 3.9.z | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: |
The ability to skip disabling swap by use of openshift_disable_swap=False has been removed from 3.9. This feature was undocumented and should not be used.
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2018-12-13 19:26:59 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Weihua Meng
2018-03-16 07:42:06 UTC
After did some investigation, seem like this is related to kube "fail-swap-on" default setting. In 3.9.9, when swap is on, node service is started successfully. But in 3.8.34, when swap is on, node service fail to be started, just like the initial report. Seem like in 3.9.9, fail-swap-on is set to false by default, while in 3.8.34, fail-swap-on is set to true by default, then hit this bug. Because here is doing 3.7 -> 3.8 -> 3.9 upgrade. In 3.9 doc, there are several doc is asking user to disable swap, but did not mentioned that in upgrade section, so maybe we could fix this bug in 3.9.z to do a pre-check to ask user disable swap before upgrade. Based on this, I would set the target release to 3.9.z. @wmeng, pls make sure "disable swap" as a must in upgrade doc. @scott, set target release to 3.9.z is okay for you? doc issue is tracking here: https://bugzilla.redhat.com/show_bug.cgi?id=1557218 (In reply to Johnny Liu from comment #2) > @scott, set target release to 3.9.z is okay for you? Yes, as long as this doesn't disrupt the upgrade path from 3.7 to 3.9. The upgrade should be disabling swap while the node is drained, we need to figure out why this is not happening. (In reply to Scott Dodson from comment #4) > The upgrade should be disabling swap while the node is drained, we need to > figure out why this is not happening. After talking about the initial reporter, when he was installing 3.7 env with openshift_disable_swap=false in inventory file, then trigger upgrade to 3.9 with the same openshift_disable_swap=false setting, that is why swap is not disabled by openshift-ansible. then the issue is hit. I think we should remove this ability in 3.9.z. The ability to override disabling swap has been removed in 3.9. Swap will be disabled during upgrade while the node is drained. https://github.com/openshift/openshift-ansible/pull/10607 Fixed in openshift-ansible-3.9.51-1 Fixed. openshift-ansible-3.9.54-1.git.0.8a67eb1.el7.noarch before upgrade: # free -h total used free shared buff/cache available Mem: 15G 1.1G 8.9G 1.4M 5.5G 14G Swap: 2.0G 0B 2.0G # cat /etc/fstab # # /etc/fstab # Created by anaconda on Sun Nov 25 17:28:52 2018 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/rhel-root / xfs defaults 0 0 UUID=cd1d5cbd-93f3-4222-9596-b4f7f22e52d1 /boot xfs defaults 0 0 /var/swapfile swap swap defaults 0 0 upgrade success. after upgrade: # free -h total used free shared buff/cache available Mem: 15G 1.6G 3.8G 2.7M 10G 13G Swap: 0B 0B 0B # cat /etc/fstab # # /etc/fstab # Created by anaconda on Sun Nov 25 17:28:52 2018 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/rhel-root / xfs defaults 0 0 UUID=cd1d5cbd-93f3-4222-9596-b4f7f22e52d1 /boot xfs defaults 0 0 #/var/swapfile swap swap defaults 0 0 Kernel Version: 3.10.0-957.el7.x86_64 Operating System: Red Hat Enterprise Linux Server 7.6 (Maipo) Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:3748 |