Bug 1797687 - Can't apply Machine Configs to Masters with roles different from "master" and machineconfigpools different from "master"
Summary: Can't apply Machine Configs to Masters with roles different from "master" and...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Machine Config Operator
Version: 4.2.z
Hardware: x86_64
OS: All
Target Milestone: ---
: 4.5.0
Assignee: Sinny Kumari
QA Contact: Michael Nguyen
Depends On:
TreeView+ depends on / blocked
Reported: 2020-02-03 16:29 UTC by Jose Kaz C R
Modified: 2020-07-13 17:14 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Cause: Custom pool is not supported on master node. Consequence: Custom pool on a node having master role is not supported. `oc label node` will apply the new custom role to the target master node but MCO will not apply changes specific to the custom pool. Error can be seen in the Machine Config Controller pod logs. Workaround (if any): User is advised to not apply multiple roles on master. This behavior is to make sure that control plane nodes remain stable.
Clone Of:
Last Closed: 2020-07-13 17:13:54 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Github openshift machine-config-operator pull 1463 None closed Bug 1797687: docs: Document that MCO doesn't apply custom pool on master node 2020-07-01 08:45:06 UTC
Red Hat Product Errata RHBA-2020:2409 None None None 2020-07-13 17:14:09 UTC

Description Jose Kaz C R 2020-02-03 16:29:40 UTC
Description of problem:

Version-Release number of selected component (if applicable):

How reproducible:

Create a MachineConfig for the Masters that require a different node role, e.g. master-0 and try to apply it to the masters.

Steps to Reproduce:
1. Create a Machine Config and Machine Config Pools for the master that are different from "master". Possible Use Case: Add a fixed IP to a new network card to one of the masters. E.g. MachineConfigPool: 01-Master-mcp, MachineConfig: 01-Master-mc

oc create -f 01-Master-mcp.yaml
oc create -f 01-Master-mc.yaml

2. Label the MachineConfigPool with the respective new label.

oc label machineconfigpool master 0-master=""  

3. Add the tag to the master node with a new role that belongs to the new Machine Config Pools. E.g. 0-master

oc label node <node_name> node-role.kubernetes.io/0-master=""

Actual results:
The configuration is never applied to the masters.

Expected results:
The node receives the configuration from the MachineConfig

Additional info:
One possible use case is to have network cards that have fixed IPs that are added post install to the masters, for example, when you are using a Storage Solution like Solidfire and the trident driver requires to reach it from the masters too.

Comment 2 Jose Kaz C R 2020-02-05 16:05:23 UTC
Yes, this does not work with the masters for some unknown reason. It works flawlessly with the workers, but not the masters.

Comment 3 Antonio Murdaca 2020-02-05 16:11:10 UTC
(In reply to Jose Kaz C R from comment #2)
> Yes, this does not work with the masters for some unknown reason. It works
> flawlessly with the workers, but not the masters.

uhm, we can take a look at it I guess and see what's happening - and asses if masters can be assigned a custom pool at all (I see the use case tho)

Comment 4 Sinny Kumari 2020-02-07 08:36:55 UTC
When a node has both master and custom role (say 0-master), MCC would be logging error message: 

E0207 08:02:20.931403       1 node_controller.go:415] error finding pool for node: node <node> has both master role and custom role 01-master

By looking at https://github.com/cgwalters/machine-config-operator/blob/b9d6c62de34d1fe01aa3b98a3559339c73c09d57/pkg/controller/node/node_controller.go#L301 it seems MCO currently doesn't support custom role being applied on master.

@Colin @Antonio Any idea if there is a reason behind not supporting custom pool for master node?

Comment 5 Colin Walters 2020-02-11 23:07:29 UTC
I think what you really want for this is machine-specific machineconfigs (which makes sense when written that way ;) )

This would be an RFE for the MCO, and actually the CNTO https://github.com/openshift/cluster-node-tuning-operator needs it too.

Comment 8 Michael Nguyen 2020-02-25 00:03:45 UTC
Verified documentation in github now has warnings about custom pools with master nodes.

Comment 10 errata-xmlrpc 2020-07-13 17:13:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.