Bug 1933751 - Customize overcloud_containers.yaml.j2 for new openstack tripleo container image prepare workflow
Summary: Customize overcloud_containers.yaml.j2 for new openstack tripleo container im...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z4
: 16.2 (Train on RHEL 8.4)
Assignee: Cédric Jeanneret
QA Contact: David Rosenfeld
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-01 15:55 UTC by Uemit Seren
Modified: 2022-12-07 19:22 UTC (History)
8 users (show)

Fixed In Version: openstack-tripleo-common-11.7.1-2.20220810165241.dfc51dd.el8ost
Doc Type: Enhancement
Doc Text:
This enhancement adds a method for pulling down third-party containers by introducing a Jinja template processing approach and adding a `template basedir` parameter, which is required by the Jinja loader, to the BaseImageManager. With this update, pulling down the Ceph-related containers is now optional. You can avoid pulling down the Ceph-related containers by setting the `ceph_images` Boolean value to `False`.
Clone Of:
Environment:
Last Closed: 2022-12-07 19:21:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 824431 0 None MERGED Split off Ceph related container images 2022-07-25 07:29:25 UTC
OpenStack gerrit 846356 0 None MERGED Split off Ceph related container images 2022-07-25 07:29:24 UTC
OpenStack gerrit 846421 0 None MERGED Split off Ceph related container images 2022-07-25 12:46:16 UTC
Red Hat Issue Tracker OSP-5736 0 None None None 2022-01-12 17:11:31 UTC
Red Hat Product Errata RHBA-2022:8794 0 None None None 2022-12-07 19:22:17 UTC

Description Uemit Seren 2021-03-01 15:55:24 UTC
Description of problem:

We have been using the a custom overcloud_containers.yaml.j2 in OSP13 to map additional services to containers for the Cisco ACI integration (specifically the AIM and OpFlex agent containers) by running: 

openstack tripleo container image prepare ... -r roles_data.yaml --template overcloud_containers.yaml.j2

This command will properly create the overcloud_images.yaml that contains the service mappings for those additional containers and it works fine for the deployment after uploading/storing all containers in the undercloud.

In OSP16 this command is deprecated in favor of the openstack tripleo contaimer image prepare workflow. However it is not clear how to provide a customized overcloud_containers.yaml.j2 file. As far as I can tell, the workflow will always use the default /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 file.

Is there a way to provide a customized overcloud_containers.yaml.j2 file or do we have to modify the package upstream usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 file or is the file not used at all when using the new workflow ? 


Version-Release number of selected component (if applicable):

OSP 16.1




Additional info:

Comment 1 Alex Schultz 2021-03-01 16:58:22 UTC
I don't think we've ever supported this. We'll have to see if there's a way to handle this.

Comment 2 Uemit Seren 2021-03-01 18:12:52 UTC
FYI: Modifying the upstream /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 and adding the additonal service/container mappings for the Cisco containers and running openstack tripleo container image prepare works similar how it was done in OSP13 with openstack overcloud container image prepare and passing in a custom overcloud_containers.yaml.j2 file.

This might be a workaround for us but if possible we would like to avoid modifying the upstream file.

Comment 3 Uemit Seren 2021-03-02 09:43:48 UTC
The openstack tripleo container image prepare works, however deploying the overcloud without the generated overcloud_images.yaml and just passing in the ContainerImagePrepare environment paraemeter will not include those additional containers from the modified /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 file in the overcloud plan (specifically the environments/containers-default-parameters.yaml does not contain them although according to the contents it clearly took the ContainerImagePrepare settings). 

How it the environments/containers-default-parameters.yaml generated in the overcloud plan ? There is no jinja2 template in /usr/share/openstack-tripleo-heat-templates/ and I also couldn't find anything in tripleo-common

Comment 4 Uemit Seren 2021-03-03 09:41:03 UTC
We are running into 2 issues right now which makes it a bit hard to use the container prepare workflow to deploy the additional Cisco ACI containers: 

1.) Similar to our OSP13 workflow we tried to first generate the overcloud container yaml file by running: openstack tripleo container image prepare -e container-image-prepare.yaml -r roles_data.yaml --output-env-file overcloud_images.yaml. As the new triploe container image prepare command is missing the --template-file to pass in a custom overcloud_images.yaml.j2 we had to modified the /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 and add the additional services/container definitions. 
Running the above comman will properly generate the additional cisco containers, however it will also generate entries for the OVN neutron container and not the OVS neutron containers even tough neutron_driver in container-image-prepare.yaml is set to "null". According to this bug report this seems to be expected: https://bugzilla.redhat.com/show_bug.cgi?id=1842976 and one should use the regular continer prepare workflow as part of the deployment workflow by just passing in the container-image-prepare.yaml file to the deploy command (see step 2 below)

2.) So instead of running the openstack tripleo container image prepare command beforehand manually we just modify the /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 file and pass in the container-image-prepare.yaml file with neutron_driver set to "null" to the deploy command. 
Checking the environments/containers-default-parameters.yaml in the generated overcloud plan shows that the OVS neutron containers are used which is what we expect. However the additional Cisco containers are missing despite the modified /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 file on the undercloud. I believe the reason is that the environments/containers-default-parameters.yaml is generated by the mistrail container on the undercloud which contains it's own copy of the /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 which is of course not modified. 

So as it stands I see 3 workarounds:

a.) Somehow get the manual openstack tripleo container image prepare to take the neutron_driver into account (maybe it's a mistake on our part but whatever I tried I could not get it to generate the OVS neutron containers).

b.) Modify the /usr/share/tripleo-common/container-images/overcloud_containers.yaml.j2 file inside the mistrail container and then just rely on the container image prepare workflow of the overcloud deployment.

c.) Manually add the cisco containers to the overcloud deployment with an environment file.

Comment 5 Uemit Seren 2021-07-02 13:45:37 UTC
Any update on this ? 
We run into the same issue for the minor update workflow. 
The openstack overcloud external-update run -y  --tags container_image_prepare won't pull the latest 3rd party containers because of the same reasons.

Comment 24 David Rosenfeld 2022-11-01 18:46:16 UTC
Included: ceph_images: false in containers-prepare-parameter.yaml and executed: sudo openstack tripleo container image prepare -e containers-prepare-parameter.yaml --output-env-file prepare_output_cephfalse.yaml --log-file /var/log/tripleo-container-image-prepare_cephfalse.log --debug

Included: ceph_images: true in containers-prepare-parameter.yaml and executed: sudo openstack tripleo container image prepare -e containers-prepare-parameter.yaml --output-env-file prepare_output_cephtrue.yaml --log-file /var/log/tripleo-container-image-prepare_cephtrue.log --debug

diff of the two output-env-files shows that ceph containers were pulled in the case when ceph_images was true, but not when ceph_images was false:

diff prepare_output_cephfalse.yaml prepare_output_cephtrue.yaml
1c1
< # Generated with the following on 2022-11-01T18:32:01.770576
---
> # Generated with the following on 2022-11-01T18:38:26.810219
3c3
< #   openstack tripleo container image prepare -e containers-prepare-parameter.yaml --output-env-file prepare_output_cephfalse.yaml --log-file /var/log/tripleo-container-image-prepare_cephfalse.log --debug
---
> #   openstack tripleo container image prepare -e containers-prepare-parameter.yaml --output-env-file prepare_output_cephtrue.yaml --log-file /var/log/tripleo-container-image-prepare_cephtrue.log --debug
6a7
>   AlertManagerContainerImage: undercloud-0.ctlplane.redhat.local:8787/rh-osbs/openshift-ose-prometheus-alertmanager:v4.6
21a23
>   ContainerCephDaemonImage: undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhceph:4-195
148a151,153
>   GrafanaContainerImage: undercloud-0.ctlplane.redhat.local:8787/rhceph/rhceph-3-dashboard-rhel7:3
>   NodeExporterContainerImage: undercloud-0.ctlplane.redhat.local:8787/rh-osbs/openshift-ose-prometheus-node-exporter:v4.6
>   PrometheusContainerImage: undercloud-0.ctlplane.redhat.local:8787/rh-osbs/openshift-ose-prometheus:v4.6

Comment 36 errata-xmlrpc 2022-12-07 19:21:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.4), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8794


Note You need to log in before you can comment on or make changes to this bug.