Bug 1710895

Summary: openstack overcloud image build config-file RHEL8 bits are not available for building whole disk image
Product: Red Hat OpenStack Reporter: mlammon
Component: openstack-tripleo-commonAssignee: Bob Fournier <bfournie>
Status: CLOSED ERRATA QA Contact: mlammon
Severity: medium Docs Contact:
Priority: urgent    
Version: 15.0 (Stein)CC: aschultz, bfournie, derekh, mburns, racedoro, slinaber, yroblamo
Target Milestone: rcKeywords: AutomationBlocker, Regression, Triaged
Target Release: 15.0 (Stein)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-common-10.8.1-0.20190821190506.42d9fdb.el8ost openstack-tripleo-puppet-elements-10.3.2-0.20190820220452.5453b89.el8ost Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-09-21 11:22:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1711083    
Bug Blocks:    

Description mlammon 2019-05-16 14:10:55 UTC
Description of problem:
openstack overcloud image build config-file /usr/share/openstack-tripleo-common/image-yaml/ RHEL8 bits are not available for building whole disk image



Example steps for building with RHEL7.5 whole disk

1. Deploy openstack undercloud
2. Install director to /home/stack/images directory
3. Download rhel 7.5 qcow image (rhel-guest-image-7.5-146.x86_64.qcow2)
3. Prepare whole disk secure hardened image

The following were the commands used to create the disk image.
export DIB_LOCAL_IMAGE=/home/stack/rhel-guest-image-7.5-146.x86_64.qcow2
export DIB_YUM_REPO_CONF="/etc/yum.repos.d/rhos-release-13.repo /etc/yum.repos.d/rhos-release-rhel-7.5.repo"
openstack overcloud image build --image-name overcloud-hardened-full --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel7.yaml --verbose



(undercloud) [stack@undercloud-0 ~]$ ls -ltr /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images
ls: cannot access '/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images': No such file or directory
(undercloud) [stack@undercloud-0 ~]$ ls -ltr /usr/share/openstack-tripleo-common/image-yaml/
total 60
-rw-r--r--. 1 root root  927 May  7 23:02 overcloud-realtime-compute.yaml
-rw-r--r--. 1 root root   93 May  7 23:02 overcloud-realtime-compute-rhel7.yaml
-rw-r--r--. 1 root root  180 May  7 23:02 overcloud-realtime-compute-centos7.yaml
-rw-r--r--. 1 root root  556 May  7 23:02 overcloud-odl-rhel7.yaml
-rw-r--r--. 1 root root 1419 May  7 23:02 overcloud-images.yaml
-rw-r--r--. 1 root root  243 May  7 23:02 overcloud-images-rhel7.yaml
-rw-r--r--. 1 root root 1419 May  7 23:02 overcloud-images-python3.yaml
-rw-r--r--. 1 root root  328 May  7 23:02 overcloud-images-fedora.yaml
-rw-r--r--. 1 root root  418 May  7 23:02 overcloud-images-centos7.yaml
-rw-r--r--. 1 root root 1195 May  7 23:02 overcloud-hardened-images.yaml
-rw-r--r--. 1 root root 1230 May  7 23:02 overcloud-hardened-images-uefi.yaml
-rw-r--r--. 1 root root   95 May  7 23:02 overcloud-hardened-images-uefi-rhel7.yaml
-rw-r--r--. 1 root root  182 May  7 23:02 overcloud-hardened-images-uefi-centos7.yaml
-rw-r--r--. 1 root root   90 May  7 23:02 overcloud-hardened-images-rhel7.yaml
-rw-r--r--. 1 root root  177 May  7 23:02 overcloud-hardened-images-centos7.yaml
(undercloud) [stack@undercloud-0 ~]$ cat /etc/rhosp-release
Red Hat OpenStack Platform release 15.0.0 Beta (Rocky)
(undercloud) [stack@undercloud-0 ~]$ cat core_puddle_version

We also need support with UEFI

At this time we cannot build regression jobs which would utilize RHEL8 whole disk

Comment 2 Derek Higgins 2019-05-16 16:17:45 UTC
The file for the hardened image appears to be provided by "openstack-tripleo-common-", do you have this package installed?

[root@f28f86c7a272 /]# rpm -qf /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml
openstack-tripleo-common-10.7.1-0.20190510090422.a9e05d4.el8ost.noarch

Comment 3 mlammon 2019-05-16 16:39:31 UTC
I should been more specific

Env:
openstack-tripleo-common-10.7.1-0.20190509140420.e46da94.el8ost.noarch


cat /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel7.yaml
disk_images:
  -
    imagename: overcloud-hardened-full
    type: qcow2
    distro: rhel7

cat /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-uefi-rhel7.yaml
disk_images:
  -
    imagename: overcloud-hardened-uefi-full
    type: qcow2
    distro: rhel7


I guess we could try and add files with distro: rhel8 and test?     I think this is the specifics. 
As for the "/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml", I assume 
this one *should* work.  WDYT?

Comment 4 Alex Schultz 2019-05-16 19:46:54 UTC
https://review.opendev.org/#/c/659136/ is that what you're looking for?

Comment 5 mlammon 2019-05-16 21:58:28 UTC
Hi Alex,

Thanks for checking.  These are for the secure hardened images (whole disk) but I think its also
going to involve many updates to the diskimage-builder as it looks like it only supports rhel7
We probably should raise a bug on this as well and make it a dependency

I see rhel7 support but no rhel8
https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder/elements

Then openstack needs additional configuration files to support RHEL8 
/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel7.yaml
/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-uefi-rhel7.yaml
and if any changes are needed to support RHEL8
/usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml

Comment 6 Derek Higgins 2019-05-17 09:11:38 UTC
(In reply to mlammon from comment #3)
...
> I guess we could try and add files with distro: rhel8 and test?     I think
> this is the specifics. 

Ahh Sorry, I see what you were talking about now


(In reply to mlammon from comment #5)

> I see rhel7 support but no rhel8
> https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder/
> elements

Same here, but this confuses me, without it I'm wondering how we are currently building the overcloud-full image...

Comment 9 Bob Fournier 2019-05-22 13:36:09 UTC
Removing patches from this BZ as there is a separate BZ for diskimage-builder with these patches -  https://bugzilla.redhat.com/show_bug.cgi?id=1711083.

Comment 11 Bob Fournier 2019-05-31 14:45:20 UTC
Fixes for diskimage-builder for rhel-8 have merged for https://bugzilla.redhat.com/show_bug.cgi?id=1711083.

Yolanda - do you know what changes now are needed to tripleo-image-elements?

Comment 12 Bob Fournier 2019-06-03 16:11:15 UTC
Yolanda - I added your patch https://review.opendev.org/#/c/662693.  Thanks.

Comment 18 Bob Fournier 2019-08-19 17:30:45 UTC
There are three upstream patches listed above that need to merge, they are currently blocked from merging by an unrelated tripleo-ci fix - https://review.opendev.org/#/c/677063.

We've verified that we were able to build a hardened whole disk image with these patches using the command:

$ openstack overcloud image build --image-name overcloud-hardened-full --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-python3.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-rhel8.yaml --verbose

We verified that the resulting overcloud-hardened-full.qcow2 image got deployed on titan58 (deployment on sealusa6 is still pending).

I'd recommend removing the blocker flag and picking up these patches in the first osp-15 async release.

Comment 19 Bob Fournier 2019-08-19 20:18:21 UTC
If we keep as blocker the ETA is 2 days (8/21) to merge patches, generate compose, and retest.

Comment 22 mlammon 2019-08-26 14:37:12 UTC
Env:
openstack-tripleo-common-10.8.1-0.20190821190506.42d9fdb.el8ost.noarch
openstack-tripleo-common-containers-10.8.1-0.20190821190506.42d9fdb.el8ost.noarch
openstack-tripleo-puppet-elements-10.3.2-0.20190820220452.5453b89.el8ost.noarch

Our whole disk regression job passed now for building whole disk RHEL8 image. We can mark it verified.
All the whole disk files are complete with exception of overcloud-hardened-images-uefi-python3.yaml
being addressed now by https://bugzilla.redhat.com/show_bug.cgi?id=1745189

is_whole_disk_image is true on all nodes deployed
(undercloud) [stack@undercloud-0 ~]$ openstack baremetal node show -f json -c driver_internal_info  controller-0 | jq '.driver_internal_info.is_whole_disk_image'
true
(undercloud) [stack@undercloud-0 ~]$ openstack baremetal node show -f json -c driver_internal_info  controller-1 | jq '.driver_internal_info.is_whole_disk_image'
true
(undercloud) [stack@undercloud-0 ~]$ openstack baremetal node show -f json -c driver_internal_info  controller-2 | jq '.driver_internal_info.is_whole_disk_image'
true
(undercloud) [stack@undercloud-0 ~]$ openstack baremetal node show -f json -c driver_internal_info  compute-0 | jq '.driver_internal_info.is_whole_disk_image'
true
(undercloud) [stack@undercloud-0 ~]$ openstack baremetal node show -f json -c driver_internal_info  compute-1 | jq '.driver_internal_info.is_whole_disk_image'
true


[root@controller-0 ~]# parted -l
Model: Virtio Block Device (virtblk)
Disk /dev/vda: 42.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  40.0GB  40.0GB  primary               boot
 2      42.9GB  42.9GB  67.1MB  primary               lba


[root@controller-0 ~]# df
Filesystem              1K-blocks    Used Available Use% Mounted on
devtmpfs                 16367104       0  16367104   0% /dev
tmpfs                    16385892   54624  16331268   1% /dev/shm
tmpfs                    16385892    8124  16377768   1% /run
tmpfs                    16385892       0  16385892   0% /sys/fs/cgroup
/dev/mapper/vg-lv_root    8972288 2296768   6675520  26% /
/dev/mapper/vg-lv_home     383648   22560    361088   6% /home
/dev/mapper/vg-lv_tmp     1550336   44020   1506316   3% /tmp
/dev/mapper/vg-lv_var    17565696 8815996   8749700  51% /var
/dev/mapper/vg-lv_log     8972288 1308516   7663772  15% /var/log
tmpfs                     3277176       0   3277176   0% /run/user/1000

Comment 26 errata-xmlrpc 2019-09-21 11:22:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811