Bug 1259913

Summary: oadm create-node-config fails with error.
Product: OpenShift Container Platform Reporter: Ryan Howe <rhowe>
Component: ocAssignee: David Eads <deads>
Status: CLOSED EOL QA Contact: Yanping Zhang <yanpzhan>
Severity: low Docs Contact:
Priority: medium    
Version: 3.0.0CC: aos-bugs, danielepasquini, dmcphers, ffranz, jokerman, maszulik, mmasters, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-02-26 19:04:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ryan Howe 2015-09-03 19:01:48 UTC
Description of problem:oadm create-node-config does fails with error. It does not receive the certificate-authority location

Version-Release number of selected component (if applicable): OSE v3.0.1


How reproducible:
100%

Steps to Reproduce:
From the master 

[root@master ~]# oadm create-node-config --node-dir=/etc/openshift/node/node-node3.example.com --node=node3.lizard.com --hostnames=node3.example.com,172.17.28.10

Generating node credentials ...
error: open openshift.local.config/master/ca.crt: no such file or directory

[root@master ~]# oadm create-node-config --node-dir=/etc/openshift/node/node-node3.example.com --node=node3.lizard.com --hostnames=node3.example.com,172.17.28.10 --certificate-authority=/etc/openshift/master/ca.crt 

Generating node credentials ...
error: open openshift.local.config/master/ca.crt: no such file or directory


Actual results:
Error


Expected results:
Should create a config 

Additional info:

https://github.com/openshift/ose/blob/f828b28595179bfa71af2e57f79c6d6bd66fc54a/pkg/cmd/server/admin/create_nodeconfig.go#L118-L119

Comment 2 David Eads 2015-09-04 12:56:05 UTC
See pull https://github.com/openshift/origin/pull/4527.

You're getting the error because there are some required certificates in order to build the node config `oadm create-node-config --help` will provide some details, but the reader digest version is that you need:
 1. certificate-authority
 2. signer-certificate
 3. signer-key
 4. signer-serial

There are default values that are often, but not always correct.  Right now, the message is really hard to understand.  The pull I've referenced improves the error messages.

Comment 3 Ryan Howe 2015-09-04 17:31:44 UTC
This error happen even when you define all the  certificates. 

On this test environment all required certs are in the default location and have not been changed since a successful install. 

`oadm create-node-config`  should auto fill certs with the openshift.local.config variable. 


The install does not use this command to create the certs for the nodes.

https://github.com/openshift/openshift-ansible/blob/795a1d2c6a34d1ca3eb8e84325d9109d134e0a33/roles/openshift_node_certificates/tasks/main.yml#L14



We may need to update the documentation as it the commands shown do not work. to create new configuration files. 
 
https://docs.openshift.com/enterprise/3.0/admin_guide/master_node_configuration.html#creating-new-configuration-files



[root@master ~]# oadm create-node-config --node-dir=/etc/openshift/node/node-node3.example.com --node=node3.example.com --hostnames=node3.example.com,172.17.28.10 --certificate-authority=/etc/openshift/master/ca.crt --signer-cert=/etc/openshift/master/ca.crt --signer-key=/etc/openshift/master/ca.key --signer-serial=/etc/openshift/master/ca.serial.txt --server-certificate=/etc/openshift/master/master.server.crt --server-key=/etc/openshift/master/master.server.key
Generating node credentials ...
error: open openshift.local.config/master/ca.crt: no such file or directory


[root@master ~]# oadm create-node-config --node-dir=/etc/openshift/node/node-node3.example.com --node=node3.example.com --hostnames=node3.example.com,172.17.28.10 --certificate-authority=/etc/openshift/master/ca.crt --signer-cert=/etc/openshift/master/ca.crt --signer-key=/etc/openshift/master/ca.key --signer-serial=/etc/openshift/master/ca.serial.txt 
Generating node credentials ...
error: open openshift.local.config/master/ca.crt: no such file or directory



[root@master ~]# oadm create-node-config --node-dir=/etc/openshift/node/node-node3.example.com --node=node3.example.com --hostnames=node3.example.com,172.17.28.10 --certificate-authority="/etc/openshift/master/ca.crt" --signer-cert="/etc/openshift/master/ca.crt" --signer-key="/etc/openshift/master/ca.key" --signer-serial="/etc/openshift/master/ca.serial.txt"
Generating node credentials ...
error: open openshift.local.config/master/ca.crt: no such file or directory

Comment 4 David Eads 2015-09-04 17:59:29 UTC
Looking at the help, I'm guessing `--node-client-certificate-authority` ?  

Which variable are you referring to here: "openshift.local.config variable".  I'd expect variable expansion to happen before the command is executed, so if you want to use an env-var you can.

Would this command be easier to use if we eliminated all the default values so you know exactly what you're typing?  I don't have any objection to that, but it means that "stand up an env in the local directory" case will be harder.

Comment 5 Ryan Howe 2015-09-04 21:56:04 UTC
The value is empty so  "all requests will be allowed" according to the help

oadm create-node-config -h

--node-client-certificate-authority='openshift.local.config/master/ca.crt': The file containing signing authorities to use to verify requests to the node. If empty, all requests will be allowed.




Running the following ended up working and generated everything with out and error. 

[root@master node]# oadm create-node-config --node-dir=/etc/openshift/node/node-node3.example.com --node=node3.example.com --hostnames=node3.example.com,172.17.28.10 --certificate-authority="/etc/openshift/master/ca.crt" --signer-cert="/etc/openshift/master/ca.crt" --signer-key="/etc/openshift/master/ca.key" --signer-serial="/etc/openshift/master/ca.serial.txt" --node-client-certificate-authority="/etc/openshift/master/ca.crt"


To correct the above running:

[root@master ~]# oadm create-node-config --node-dir=/etc/openshift/node/node-node3.example.com --node=node3.example.com --hostnames=node3.example.com,172.17.28.10 --certificate-authority="/etc/openshift/master/ca.crt" --signer-cert="/etc/openshift/master/ca.crt" --signer-key="/etc/openshift/master/ca.key" --signer-serial="/etc/openshift/master/ca.serial.txt"
Generating node credentials ...
error: open openshift.local.config/master/ca.crt: no such file or directory

-This generates and error that is misleading 

[root@master node-node3.example.com]# ll
total 16
-rw-r--r--. 1 root root 1123 Sep  4 17:47 master-client.crt
-rw-------. 1 root root 1675 Sep  4 17:47 master-client.key
-rw-r--r--. 1 root root 2205 Sep  4 17:47 server.crt
-rw-------. 1 root root 1679 Sep  4 17:47 server.key

4 - Configurations are created. 


This is seems to also be an issue with the docs as the docs have you running the following which create nothing. 

$ oadm create-node-config --node-dir=/openshift.local.config/node-<node_hostname> --node=<node_hostname> --hostnames=<hostname>,<ip_address>

Lastly shouldn't "openshift.local.config" look in the default location. Or should it be better to just document what is required to be passed with the  oadm create-node-config

Comment 6 Miciah Dashiel Butler Masters 2016-01-19 16:30:31 UTC
Following is the experience I had today in an Origin devenv (I know I should provide IP address and public hostname to `--hostnames`, but I was being lazy):

    # oadm create-node-config --node-dir=/openshift.local.config/node-`hostname` --node=`hostname` --hostnames=`hostname`
    error: --certificate-authority, "/root/openshift.local.config/master/ca.crt" must be a valid certificate file
    See 'oadm create-node-config -h' for help and examples.

That's clear enough (I have `/openshift.local.config/`, but `oadm create-node-config` is looking under `/root/openshift.local.config/`), and I'm too lazy to look at `oadm create-node-config -h` when the error output already tells me which flag I need to add, so let's just add --certificate-authority and try again:

    # oadm create-node-config --node-dir=/openshift.local.config/node-`hostname` --node=`hostname` --hostnames=`hostname` --certificate-authority=/openshift.local.config/master/ca.crt
    Generating node credentials ...
    error: --signer-cert, "/root/openshift.local.config/master/ca.crt" must be a valid certificate file

Oh, I'm missing another flag.  No worries, I'll add --signer-cert...

    # oadm create-node-config --node-dir=/openshift.local.config/node-`hostname` --node=`hostname` --hostnames=`hostname` --certificate-authority=/openshift.local.config/master/ca.crt --signer-cert=/openshift.local.config/master/ca.crt                                                               
    Generating node credentials ...
    error: --signer-key, "/root/openshift.local.config/master/ca.key" must be a valid key file

This is getting tedious.  I'm wishing I could just provide a flag that told `oadm create-node-config` the location of my master configuration directory and it would just read files from there.  Anyway, let's add --signer-key and hope I'm near the end of the road...

    # oadm create-node-config --node-dir=/openshift.local.config/node-`hostname` --node=`hostname` --hostnames=`hostname` --certificate-authority=/openshift.local.config/master/ca.crt --signer-cert=/openshift.local.config/master/ca.crt --signer-key=/openshift.local.config/master/ca.key
    Generating node credentials ...
    error: --signer-serial, "/root/openshift.local.config/master/ca.serial.txt" must be a valid file

Oh, there's a serial file? Well, I'll add a flag for that, and then I will have one flag (or more) for each file ca.*, so that's gotta be enough, right?

    # oadm create-node-config --node-dir=/openshift.local.config/node-`hostname` --node=`hostname` --hostnames=`hostname` --certificate-authority=/openshift.local.config/master/ca.crt --signer-cert=/openshift.local.config/master/ca.crt --signer-key=/openshift.local.config/master/ca.key  --signer-serial=/openshift.local.config/master/ca.serial.txt 
    Generating node credentials ...
    error: open openshift.local.config/master/ca.crt: no such file or directory

Still not enough flags? Bah, which one do I need to add now? It doesn't say—is it a different problem? I finally look at `oadm create-node-config --help`, and I see that the default value for `--certificate-authority`, `--node-client-certificate-authority`, and `--signer-cert` is 'openshift.local.config/master/ca.crt', so I realise that `oadm create-node-config` is intended to be run from the directory containing `openshift.local.config/`, but never mind that, I think I'm almost there; let's try adding `--node-client-certificate-authority`:

    oadm create-node-config --node-dir=/openshift.local.config/node-`hostname` --node=`hostname` --hostnames=`hostname` --certificate-authority=/openshift.local.config/master/ca.crt --signer-cert=/openshift.local.config/master/ca.crt --signer-key=/openshift.local.config/master/ca.key  --signer-serial=/openshift.local.config/master/ca.serial.txt --node-client-certificate-authority=/openshift.local.config/master/ca.crt
    Generating node credentials ...
    Created node config for [redacted] in /openshift.local.config/[redacted]

Yay! But here are a couple ideas for usability:

1. If several flags are missing, list off *all* the missing flags, not just one.

2. If the file is reading from relative paths (as per the default option values), include a note along the lines of, "oadm create-node-config is meant to be run from the directory containing openshift.local.conf/".

3. If the node client certificate authority file cannot be found, tell the user to specify `--node-client-certificate-authority`.

4. As an alternative to requiring to the user to change the CWD or give absolute pathnames or pathnames relative to the CWD, provide a flag through which the user can specify a pathname for a directory in which `ca.crt`, `ca.key`, and `ca.serial.txt` can be found.

On the other hand, I may have been a bit thick to take so long to try `oadm create-node-config --help`, so feel free to take the above suggestions in that light, but I hope my experience is helpful.

Comment 7 Daniele Pasquini 2017-11-27 10:14:24 UTC
Provide IP address and public hostname depend on your private/public cloud hostname resolution.

On your master server create master and node yml file:

Generate master
#openshift start master --write-config=/openshift.local.config/master

Generate node1
# oadm create-node-config --node-dir=/openshift.local.config/node-node1 --node=node1 --hostnames=node1 --certificate-authority=/openshift.local.config/master/ca.crt --signer-cert=/openshift.local.config/master/ca.crt --signer-key=/openshift.local.config/master/ca.key --signer-serial=/openshift.local.config/master/ca.serial.txt --node-client-certificate-authority=/openshift.local.config/master/ca.crt
Generate node2
# oadm create-node-config --node-dir=/openshift.local.config/node-node2 --node=node2 --hostnames=node2 --certificate-authority=/openshift.local.config/master/ca.crt --signer-cert=/openshift.local.config/master/ca.crt --signer-key=/openshift.local.config/master/ca.key --signer-serial=/openshift.local.config/master/ca.serial.txt --node-client-certificate-authority=/openshift.local.config/master/ca.crt
Start
#openshift start --master-config=/openshift.local.config/master/master-config.yaml --node-config=/openshift.local.config/node-node1/node-config.yaml --node-config=/openshift.local.config/node-node2/node-config.yaml