Bug 1566424 - [RFE] Scale out Overcloud VM-based OCP on OSP with Ansible
Summary: [RFE] Scale out Overcloud VM-based OCP on OSP with Ansible
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: ---
: 3.11.z
Assignee: Tomas Sedovic
QA Contact: Jon Uriarte
URL:
Whiteboard:
Depends On:
Blocks: 1566010 1566927
TreeView+ depends on / blocked
 
Reported: 2018-04-12 09:52 UTC by Ramon Acedo
Modified: 2019-01-10 09:04 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Feature: Add scale out Ansible playbooks for the OpenStack deployed clusters. Reason: When installing OpenShift on top of OpenStack with the OpenStack provisioning playbooks (`playbooks/openstack/openshift-cluster/provision_install.yml`), scaling the cluster out required several manual steps such as writing the inventory by hand and running two extra playbooks. This was more brittle, required more complex documentation and did not match the initial deployment experience. Result: To scale out OpenShift on OpenStack, the user can now simply change the desired number of nodes and run one of the following playbooks (depending on whether they want to scale the worker or master nodes): playbooks/openstack/openshift-cluster/node-scaleup.yml playbooks/openstack/openshift-cluster/master-scaleup.yml
Clone Of:
Environment:
Last Closed: 2019-01-10 09:03:57 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift openshift-ansible pull 9008 None closed Add scaleup playbook and docs for OpenStack 2020-05-25 10:21:02 UTC
Github openshift openshift-ansible pull 9243 None closed Add OpenStack master and node scaleup 2020-05-25 10:21:02 UTC
Red Hat Product Errata RHBA-2019:0024 None None None 2019-01-10 09:04:04 UTC

Description Ramon Acedo 2018-04-12 09:52:31 UTC
Write a playbook that takes the number of new VM OpenShift app nodes. It will provision the new nodes without touching any of the existing ones and perform the necessary steps to add the new nodes to the OCP cluster.

Comment 1 Tomas Sedovic 2018-06-28 10:56:33 UTC
This is a first step:

https://github.com/openshift/openshift-ansible/pull/9008

It's not fully automated -- the user must explicitly mark which nodes are the newly-added ones -- but it works, it's simple to understand and will serve as a basis for a fully-automated scaleup solution.

Comment 2 Tomas Sedovic 2018-07-18 11:42:01 UTC
This PR adds scaling for all node types (master, infra, compute):

https://github.com/openshift/openshift-ansible/pull/9243

Comment 3 Tomas Sedovic 2018-07-30 10:25:20 UTC
The pull requests have now been merged.

Comment 4 Scott Dodson 2018-08-02 12:41:22 UTC
In openshift-ansible-3.11.0-0.10.0

Comment 5 N. Harrison Ripps 2018-09-21 20:15:51 UTC
Per OCP program call on 21-SEP-2018 we are deferring Kuryr-related bugs to 3.11.z

Comment 6 weiwei jiang 2018-12-18 14:09:51 UTC
Checked with openshift-ansible-3.11.57-1 and both master and node scaleup work well.

master(from 2 to 3): 
ansible-playbook --private-key ~/.ssh/libra-new.pem --user openshift   -i openshift-ansible/playbooks/openstack/scaleup_inventory.py   -i inventory   openshift-ansible/playbooks/openstack/openshift-cluster/master-scaleup.yml -vv

node(infra from 1 to 2 and compute from 2 to 3):
ansible-playbook --private-key ~/.ssh/libra-new.pem --user openshift   -i openshift-ansible/playbooks/openstack/scaleup_inventory.py   -i inventory   openshift-ansible/playbooks/openstack/openshift-cluster/node-scaleup.yml -vv


[openshift@master-0 ~]$ oc get pods -o wide -l run=h
NAME        READY     STATUS    RESTARTS   AGE       IP           NODE                                  NOMINATED NODE
h-1-4s5w5   1/1       Running   0          2m        10.129.2.3   infra-node-1.wjiang-ocp.example.com   <none>
h-1-5kt5z   1/1       Running   0          2m        10.131.2.5   master-2.wjiang-ocp.example.com       <none>
h-1-7d9qn   1/1       Running   0          1m        10.131.0.8   infra-node-0.wjiang-ocp.example.com   <none>
h-1-7mnlk   1/1       Running   0          1m        10.130.2.4   app-node-2.wjiang-ocp.example.com     <none>
h-1-cxsrr   1/1       Running   0          2m        10.128.2.3   app-node-1.wjiang-ocp.example.com     <none>
h-1-dfzhx   1/1       Running   0          1m        10.128.2.4   app-node-1.wjiang-ocp.example.com     <none>
h-1-fmzcb   1/1       Running   0          2m        10.130.2.3   app-node-2.wjiang-ocp.example.com     <none>
h-1-gp8x9   1/1       Running   0          2m        10.128.2.2   app-node-1.wjiang-ocp.example.com     <none>
h-1-k45zg   1/1       Running   0          2m        10.130.2.2   app-node-2.wjiang-ocp.example.com     <none>
h-1-kl575   1/1       Running   0          2m        10.131.0.7   infra-node-0.wjiang-ocp.example.com   <none>
h-1-ktfgh   1/1       Running   0          1m        10.129.2.5   infra-node-1.wjiang-ocp.example.com   <none>
h-1-l6vp8   1/1       Running   0          2m        10.130.0.2   app-node-0.wjiang-ocp.example.com     <none>
h-1-rlrzj   1/1       Running   0          2m        10.129.2.4   infra-node-1.wjiang-ocp.example.com   <none>
h-1-wmrsp   1/1       Running   0          2m        10.130.0.3   app-node-0.wjiang-ocp.example.com     <none>
[openshift@master-0 ~]$ oc get nodes  -o wide 
NAME                                  STATUS    ROLES     AGE       VERSION           INTERNAL-IP     EXTERNAL-IP    OS-IMAGE                                      KERNEL-VERSION              CONTAINER-RUNTIME
app-node-0.wjiang-ocp.example.com     Ready     compute   3h        v1.11.0+d4cacc0   192.168.99.12   10.8.248.117   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1
app-node-1.wjiang-ocp.example.com     Ready     compute   1h        v1.11.0+d4cacc0   192.168.99.16   10.8.252.236   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1
app-node-2.wjiang-ocp.example.com     Ready     compute   42m       v1.11.0+d4cacc0   192.168.99.10   10.8.252.159   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1
infra-node-0.wjiang-ocp.example.com   Ready     infra     3h        v1.11.0+d4cacc0   192.168.99.15   10.8.249.246   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1
infra-node-1.wjiang-ocp.example.com   Ready     infra     42m       v1.11.0+d4cacc0   192.168.99.13   10.8.245.246   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1
master-0.wjiang-ocp.example.com       Ready     master    3h        v1.11.0+d4cacc0   192.168.99.7    10.8.250.138   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1
master-1.wjiang-ocp.example.com       Ready     master    3h        v1.11.0+d4cacc0   192.168.99.8    10.8.244.173   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1
master-2.wjiang-ocp.example.com       Ready     master    20m       v1.11.0+d4cacc0   192.168.99.18   10.8.245.157   Red Hat Enterprise Linux Server 7.6 (Maipo)   3.10.0-957.1.3.el7.x86_64   docker://1.13.1

Comment 8 errata-xmlrpc 2019-01-10 09:03:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0024


Note You need to log in before you can comment on or make changes to this bug.