Bug 2278831 - No disk space check causing unbootable node during leapp upgrade
Summary: No disk space check causing unbootable node during leapp upgrade
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: documentation
Version: 16.2 (Train)
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: async
: 17.1
Assignee: kgilliga
QA Contact: Archana Singh
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-05-03 09:18 UTC by Kenny Tordeurs
Modified: 2024-12-04 18:21 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2024-12-04 18:21:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker OSP-31997 0 None None None 2024-05-03 09:22:31 UTC

Description Kenny Tordeurs 2024-05-03 09:18:34 UTC
Description of problem:
There should be a check to ensure there is enough disk space available.

Version-Release number of selected component (if applicable):
OSP 16.2

How reproducible:
100%

Steps to Reproduce:
This is the original disk layout, unfortunately the / partition (3.4G out of 4G was in use) did not have enough space available but leapp did not complain about this causing the node to be rebooted at the end and going into the emergency shell at which point we can see that there was not enough disk space available for dnf to continue:

[root@openstack ~]# lsblk

NAME              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                 8:0    0 446.6G  0 disk
__sda1              8:1    0   200M  0 part /boot/efi
__sda2              8:2    0     1M  0 part
__sda3              8:3    0  40.3G  0 part
_ __vg-lv_root    253:0    0     4G  0 lvm  /
_ __vg-lv_tmp     253:1    0     2G  0 lvm  /tmp
_ __vg-lv_var     253:2    0  28.2G  0 lvm  /var
_ __vg-lv_home    253:3    0     2G  0 lvm  /home
_ __vg-lv_varnova 253:4    0     4G  0 lvm  /var/lib/nova
__sda4              8:4    0    65M  0 part
 

They applied correct partitioning with enough space available in the / partition and they had to restore and redo the leapp upgrade after changing the disk layout like below

[root@openstack ~]# lsblk

NAME              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                 8:0    0 446.6G  0 disk
__sda1              8:1    0   200M  0 part /boot/efi
__sda2              8:2    0     1M  0 part
__sda3              8:3    0 446.4G  0 part
_ __vg-lv_root    253:0    0    20G  0 lvm  /
_ __vg-lv_tmp     253:1    0     5G  0 lvm  /tmp
_ __vg-lv_var     253:2    0    50G  0 lvm  /var
_ __vg-lv_home    253:3    0     2G  0 lvm  /home
_ __vg-lv_varnova 253:4    0   100G  0 lvm  /var/lib/nova
__sda4              8:4    0    65M  0 part
 
Here is the disk space usage (after the new partitioning/ extension but we can see the used sizes)

[root@openstack ~]# df -hP
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   746G     0  746G   0% /dev
tmpfs                      756G   88K  756G   1% /dev/shm
tmpfs                      756G  9.8M  756G   1% /run
tmpfs                      756G     0  756G   0% /sys/fs/cgroup
/dev/mapper/vg-lv_root      20G  3.4G   17G  17% /
/dev/sda1                  200M  5.9M  194M   3% /boot/efi
/dev/mapper/vg-lv_var       50G  8.8G   42G  18% /var
/dev/mapper/vg-lv_tmp      5.0G   69M  5.0G   2% /tmp
/dev/mapper/vg-lv_home     2.1G   47M  2.0G   3% /home
/dev/mapper/vg-lv_varnova  100G  753M  100G   1% /var/lib/nova

Actual results:
node is not able to boot

Expected results:
Disk space should be properly checked before upgrade


Additional info:

Comment 3 Sergii Golovatiuk 2024-11-29 20:24:04 UTC
Leapp version that upgrades RHEL 8.4 to 9.2 doesn't have enough checks to verify wether all partitions have enough space before creating overlays and make non destructible operations. That functionality was implemented in leapp for 8.9 [1]. Taking into account that only critical security issues are backported only to 16.2>17.1 upgrade the documentation will be enhanced to verify disk spaces manually before upgrades [2]

[1] https://github.com/oamg/leapp-repository/pull/1097
[2] https://docs.redhat.com/en/documentation/red_hat_openstack_platform/17.1/html-single/framework_for_upgrades_16.2_to_17.1/index#known-issues-that-might-block-an-upgrade


Note You need to log in before you can comment on or make changes to this bug.