Bug 2166224 - Update/upgrade run doesn't honour blacklist node
Summary: Update/upgrade run doesn't honour blacklist node
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-tripleoclient
Version: 17.0 (Wallaby)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: beta
: 17.1
Assignee: Mikolaj Ciecierski
QA Contact: Khomesh Thakre
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-02-01 08:19 UTC by Khomesh Thakre
Modified: 2023-10-03 21:21 UTC (History)
5 users (show)

Fixed In Version: python-tripleoclient-16.5.1-1.20230407001101.210ed7c.el9ost
Doc Type: If docs needed, set a value
Doc Text:
In Red Hat OpenStack Platform 17.1, overcloud nodes on the skiplist are excluded when you run the `openstack overcloud update run` command. Previously, nodes on the skiplist were unintentionally added to the `openstack overcloud update run` command. The - -limit flag now includes the `excluded_overcloud` group to exclude all nodes in the skiplist. The excluded_overcloud` group is only added to the - -limit flag when skiplist contains any host.
Clone Of:
: 2242016 (view as bug list)
Environment:
Last Closed: 2023-08-16 01:13:42 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 878021 0 None MERGED Fix update run to not run update on excluded nodes 2023-04-05 05:58:22 UTC
Red Hat Issue Tracker OSP-21949 0 None None None 2023-02-01 08:20:03 UTC
Red Hat Product Errata RHEA-2023:4577 0 None None None 2023-08-16 01:14:10 UTC

Description Khomesh Thakre 2023-02-01 08:19:26 UTC
Description of problem:

Update run doesnt honour blacklist node and still try to run on blacklisted node.

We have a similar bug https://bugzilla.redhat.com/show_bug.cgi?id=2151846.

Above bug fixed the blacklist in external-update task but for update run it not working.


Version-Release number of selected component (if applicable):
Rhosp version 17.0.1 wallaby

openstack-tripleo-common-containers-15.4.1-0.20221208160327.51f6577.el9ost.noarch
openstack-tripleo-common-15.4.1-0.20221208160327.51f6577.el9ost.noarch
openstack-tripleo-heat-templates-14.3.1-0.20221208160327.feca772.el9ost.noarch
openstack-tripleo-validations-14.2.2-0.20221208090406.37bfae3.el9ost.noarch


How reproducible:


Steps to Reproduce:
1. Deploy 17.0 with GA build.

2. Blacklisted am overcloud node
  passed the below file and ran overcloud deploy again to update the overcloud plan.
  ~~~
  (undercloud) [stack@undercloud-0 qe-Cloud-0]$ cat ~/server-blacklist.yaml
  parameter_defaults:
    DeploymentServerBlacklist:
      - compute-0
  ~~~

3. Undercloud upgrade to latest passed_phase2 

4. Ran overcloud update prepare with the same server-blacklist.yaml file

5. blacklist node is add in excluded_overcloud grp

~~~
(undercloud) [stack@undercloud-0 ~]$ ansible -i config-download/qe-Cloud-0/tripleo-ansible-inventory.yaml --list-host excluded_overcloud
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details
[WARNING]: Skipping key (deprecated) in group (overcloud) as it is not a mapping, it is a <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>
  hosts (1):
    compute-0
~~~

6. Ran Compute update and it failed as compute-0 is unreachable

~~~
(undercloud) [stack@undercloud-0 ~]$ openstack overcloud update run --stack qe-Cloud-0 --limit Compute

... 

2023-01-31 09:45:42.897514 | 5254000d-ee37-d3e9-44af-0000000025cb |       TASK | Clean podman volumes
2023-01-31 09:45:43.245025 | 5254000d-ee37-d3e9-44af-0000000025cb |    CHANGED | Clean podman volumes | compute-1
2023-01-31 09:45:43.246333 | 5254000d-ee37-d3e9-44af-0000000025cb |     TIMING | tripleo_podman : Clean podman volumes | compute-1 | 0:05:15.125310 | 0.35s

PLAY RECAP *********************************************************************
compute-0                  : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0
compute-1                  : ok=351  changed=94   unreachable=0    failed=0    skipped=175  rescued=0    ignored=1   
2023-01-31 09:45:43.285455 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-01-31 09:45:43.285859 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 449        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-01-31 09:45:43.286204 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:05:15.165190 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-01-31 09:45:43.286589 |                                 UUID |       Info |       Host |   Task Name |   Run Time
2023-01-31 09:45:43.286939 | 5254000d-ee37-d3e9-44af-000000000688 |    SUMMARY |  compute-0 | Gathering Facts | 25.04s
2023-01-31 09:45:43.287322 | 5254000d-ee37-d3e9-44af-0000000024f0 |    SUMMARY |  compute-1 | tripleo_container_manage : Create containers managed by Podman for {{ tripleo_container_manage_config }} | 11.30s
2023-01-31 09:45:43.287682 | 5254000d-ee37-d3e9-44af-000000000627 |    SUMMARY |  compute-1 | Wait for puppet host configuration to finish | 10.77s
2023-01-31 09:45:43.288001 | 5254000d-ee37-d3e9-44af-000000000645 |    SUMMARY |  compute-1 | Wait for puppet host configuration to finish | 10.59s
2023-01-31 09:45:43.288377 | 5254000d-ee37-d3e9-44af-000000000663 |    SUMMARY |  compute-1 | Wait for puppet host configuration to finish | 10.58s
2023-01-31 09:45:43.288709 | 5254000d-ee37-d3e9-44af-000000000654 |    SUMMARY |  compute-1 | Wait for puppet host configuration to finish | 10.58s
2023-01-31 09:45:43.289040 | 5254000d-ee37-d3e9-44af-000000000636 |    SUMMARY |  compute-1 | Wait for puppet host configuration to finish | 10.55s
2023-01-31 09:45:43.289419 | 5254000d-ee37-d3e9-44af-000000002061 |    SUMMARY |  compute-1 | tripleo_container_manage : Manage container systemd services and cleanup old systemd healthchecks for {{ tripleo_container_manage_config }} | 10.34s
2023-01-31 09:45:43.289776 | 5254000d-ee37-d3e9-44af-000000000924 |    SUMMARY |  compute-1 | tripleo_firewall : Manage firewall rules | 9.01s
2023-01-31 09:45:43.290142 | 5254000d-ee37-d3e9-44af-0000000022a9 |    SUMMARY |  compute-1 | tripleo_container_manage : Manage container systemd services and cleanup old systemd healthchecks for {{ tripleo_container_manage_config }} | 6.65s          
2023-01-31 09:45:43.290537 | 5254000d-ee37-d3e9-44af-000000000787 |    SUMMARY |  compute-1 | tripleo_hieradata : Render hieradata from template | 6.55s
2023-01-31 09:45:43.290898 | 5254000d-ee37-d3e9-44af-000000002060 |    SUMMARY |  compute-1 | tripleo_container_manage : Create containers managed by Podman for {{ tripleo_container_manage_config }} | 5.91s
2023-01-31 09:45:43.291243 | 5254000d-ee37-d3e9-44af-000000000af0 |    SUMMARY |  compute-1 | tripleo_kernel : Set extra sysctl options | 5.49s
2023-01-31 09:45:43.291594 | 5254000d-ee37-d3e9-44af-00000000059e |    SUMMARY |  compute-1 | Pre-fetch all the containers | 3.84s
2023-01-31 09:45:43.291947 | 5254000d-ee37-d3e9-44af-0000000005c6 |    SUMMARY |  compute-1 | create libvirt persistent data directories | 2.62s
2023-01-31 09:45:43.292322 | 5254000d-ee37-d3e9-44af-000000000591 |    SUMMARY |  compute-1 | Special treatment for OpenvSwitch | 2.25s
2023-01-31 09:45:43.292681 | 5254000d-ee37-d3e9-44af-0000000022a8 |    SUMMARY |  compute-1 | tripleo_container_manage : Create containers managed by Podman for {{ tripleo_container_manage_config }} | 2.09s
2023-01-31 09:45:43.293013 | 5254000d-ee37-d3e9-44af-000000001b40 |    SUMMARY |  compute-1 | tripleo_container_manage : Create containers managed by Podman for {{ tripleo_container_manage_config }} | 2.07s
2023-01-31 09:45:43.293413 | 5254000d-ee37-d3e9-44af-000000000590 |    SUMMARY |  compute-1 | Remove openstack-nova-compute and python-nova package during upgrade | 2.05s
2023-01-31 09:45:43.293873 | 5254000d-ee37-d3e9-44af-0000000024f1 |    SUMMARY |  compute-1 | tripleo_container_manage : Manage container systemd services and cleanup old systemd healthchecks for {{ tripleo_container_manage_config }} | 1.91s          
2023-01-31 09:45:43.294289 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-01-31 09:45:43.294679 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-01-31 09:45:43.295020 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~
2023-01-31 09:45:43.295396 |  The following node(s) had failures: compute-0
2023-01-31 09:45:43.295804 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~

Comment 17 errata-xmlrpc 2023-08-16 01:13:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4577


Note You need to log in before you can comment on or make changes to this bug.