Bug 2215053

Summary: [OSP17.1][FRR]Frr container is deployed before the container image prepare task is done
Product: Red Hat OpenStack Reporter: Candido Campos <ccamposr>
Component: ovn-bgp-agentAssignee: Luca Miccini <lmiccini>
Status: ASSIGNED --- QA Contact: Candido Campos <ccamposr>
Severity: medium Docs Contact:
Priority: medium    
Version: 17.1 (Wallaby)CC: dalvarez, gregraka, jelynch, lmartins, lmiccini, ltomasbo
Target Milestone: gaKeywords: Triaged
Target Release: 17.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
In RHOSP 17.1 environments that use Border Gateway Protocol (BGP) dynamic routing, there is currently a known issue where the FRRouting (FRR) container fails to deploy. This failure occurs because the RHOSP director deploys the FRR container before the container image prepare task finishes. Workaround: In your heat templates, ensure that the `ContainerImagePrepare` precedes the `overcloud deploy` command.
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Candido Campos 2023-06-14 15:05:20 UTC
Description of problem:

Frr container is deployed before the container image prepare task is done then if the container image for the overcloud deploy is different than the frr image loaded in the undercloud deploy the deployment go to fail.

As simple W/A the container image prepare command can be exectuted before the overcloud deploy


frr deploy task example:

[root@undercloudbgp-0 stack]# cat /home/stack/overcloud-deploy/overcloud/config-download/overcloud/ControllerRack2/pre_deploy_step_tasks.yaml
- import_role:
    name: tripleo_frr
  name: Configure FRR
  vars:
    tripleo_frr_bfd: true
    tripleo_frr_bgp: true
    tripleo_frr_bgp_asn: 64999
    tripleo_frr_bgp_ipv4: true
    tripleo_frr_bgp_ipv4_allowas_in: true
    tripleo_frr_bgp_ipv4_src_network: main_network
    tripleo_frr_bgp_ipv6: true
    tripleo_frr_bgp_ipv6_allowas_in: true
    tripleo_frr_bgp_ipv6_src_network: main_network_ipv6
    tripleo_frr_bgp_l2vpn: true
    tripleo_frr_bgp_l2vpn_ebgp_multihop: 10
    tripleo_frr_bgp_l2vpn_peers:
    - 5.1.1.1
    tripleo_frr_bgp_l2vpn_peers_scope: external
    tripleo_frr_bgp_l2vpn_uplink_activate: true
    tripleo_frr_bgp_neighbor_ttl_security_hops: 1
    tripleo_frr_bgp_uplinks:
    - nic2
    - nic3
    tripleo_frr_bgp_uplinks_scope: internal
    tripleo_frr_config_basedir: /var/lib/config-data/ansible-generated/frr
    tripleo_frr_ipv4_src_ip: ''
    tripleo_frr_ipv6_src_ip: ''
    tripleo_frr_log_level: debugging
    tripleo_frr_router_id: ''
    tripleo_frr_zebra: true
- include_role:
    name: tripleo_container_manage
  name: Start FRR
  vars:
    tripleo_container_manage_clean_orphans: false
    tripleo_container_manage_config: /var/lib/tripleo-config/container-startup-config/step_0
    tripleo_container_manage_config_id: frr
    tripleo_container_manage_config_patterns: frr.json
    tripleo_container_manage_systemd_order: true
- import_role:
    name: tripleo_frr
  name: Configure Ovn Bgp Agent
  vars:
    tripleo_frr_ovn_bgp_agent_bgp_as: 64999
    tripleo_frr_ovn_bgp_agent_ca_cert: /etc/ipa/ca.crt
    tripleo_frr_ovn_bgp_agent_driver: ovn_bgp_driver
    tripleo_frr_ovn_bgp_agent_enable: true
    tripleo_frr_ovn_bgp_agent_expose_tenant_networks: true
    tripleo_frr_ovn_bgp_agent_internal_tls_enable: true
    tripleo_frr_ovn_bgp_agent_ovsdb_connection: tcp:127.0.0.1:6640
    tripleo_frr_ovn_bgp_agent_reconcile_interval: 300
[root@undercloudbgp-0 stack]# 


executed in:

....
2023-06-14 13:23:57.976608 | 52540059-0aef-43d7-e2f6-00000000dc98 |    SKIPPED | restart frr container | cmp-1-1
2023-06-14 13:23:57.981039 | 52540059-0aef-43d7-e2f6-00000000dc98 |     TIMING | tripleo_frr : restart frr container | cmp-1-1 | 0:11:14.285342 | 0.05s
2023-06-14 13:23:57.981548 | 52540059-0aef-43d7-e2f6-00000000dc97 |    CHANGED | check if tripleo_frr systemd service is active | cmp-2-0
2023-06-14 13:23:57.982103 | 52540059-0aef-43d7-e2f6-00000000dc97 |     TIMING | tripleo_frr : check if tripleo_frr systemd service is active | cmp-2-0 | 0:11:14.286408 | 0.28s
2023-06-14 13:23:58.004905 | 1dc61823-39ce-40ff-bfbd-6db5499f18cc |   INCLUDED | /usr/share/ansible/roles/tripleo_frr/tasks/ovn_bgp_agent.yml | cmp-1-0
2023-06-14 13:23:58.030195 | 52540059-0aef-43d7-e2f6-00000000df72 |       TASK | create OVN BGP agent directories
2023-06-14 13:23:58.050432 | 52540059-0aef-43d7-e2f6-00000000dc9a |       TASK | include OVN BGP agent
2023-06-14 13:23:58.072395 | 52540059-0aef-43d7-e2f6-00000000dc98 |       TASK | restart frr container
2023-06-14 13:23:58.099901 | 52540059-0aef-43d7-e2f6-00000000dc97 |       TASK | check if tripleo_frr systemd service is active
2023-06-14 13:23:58.110886 | 52540059-0aef-43d7-e2f6-00000000dc97 |    CHANGED | check if tripleo_frr systemd service is active | cmp-2-1
2023-06-14 13:23:58.118114 | 52540059-0aef-43d7-e2f6-00000000dc97 |     TIMING | tripleo_frr : check if tripleo_frr systemd service is active | cmp-2-1 | 0:11:14.422395 | 0.31s
2023-06-14 13:23:58.118792 | 52540059-0aef-43d7-e2f6-00000000dc9a |     TIMING | tripleo_frr : include OVN BGP agent | cmp-1-1 | 0:11:14.423099 | 0.07s
2023-06-14 13:23:58.119226 | 52540059-0aef-43d7-e2f6-00000000dc98 |    SKIPPED | restart frr container | cmp-2-0
..........


 and container image prepare executed in :

vi /home/stack/overcloud-deploy/overcloud/config-download/overcloud/external_deploy_steps_tasks_step1.yaml

.....

PLAY [External deployment step 1] **********************************************
2023-06-14 13:24:31.823791 | 52540059-0aef-43d7-e2f6-0000000000b4 |       TASK | External deployment step 1
2023-06-14 13:24:31.865666 | 52540059-0aef-43d7-e2f6-0000000000b4 |         OK | External deployment step 1 | undercloud -> localhost | result={^M
    "changed": false,^M
    "msg": "Use --start-at-task 'External deployment step 1' to resume from this task"^M
}
[WARNING]: ('undercloud -> localhost', '52540059-0aef-43d7-e2f6-0000000000b4')^M
missing from stats
2023-06-14 13:24:31.930562 | 52540059-0aef-43d7-e2f6-0000000000b5 |     TIMING | include_tasks | undercloud | 0:11:48.234843 | 0.04s
2023-06-14 13:24:31.987371 | 9551f768-c644-44e1-a7d8-ae3003322513 |   INCLUDED | /home/stack/overcloud-deploy/overcloud/config-download/overcloud/external_deploy_steps_tasks_step1.yaml | undercloud
2023-06-14 13:24:32.008933 | 52540059-0aef-43d7-e2f6-00000000f6f7 |       TASK | Container image prepare
2023-06-14 13:25:58.348646 | 52540059-0aef-43d7-e2f6-00000000f6f7 |    CHANGED | Container image prepare | undercloud
2023-06-14 13:25:58.353181 | 52540059-0aef-43d7-e2f6-00000000f6f7 |     TIMING | tripleo_container_image_prepare : Container image prepare | undercloud | 0:13:14.657472 | 86.34s


...............