This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1293563 - Can not successfully configure cinder volume plugin in containerized openshift node
Can not successfully configure cinder volume plugin in containerized openshif...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Container Platform
Classification: Red Hat
Component: Documentation (Show other bugs)
3.1.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Vikram Goyal
Vikram Goyal
Vikram Goyal
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-22 02:16 EST by Jianwei Hou
Modified: 2016-02-15 22:17 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-15 22:17:06 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jianwei Hou 2015-12-22 02:16:58 EST
Description of problem:
Can not successfully configure cinder volume plugin in containerized openshift node.

Version-Release number of selected component (if applicable):
openshift v3.1.1.0
kubernetes v1.1.0-origin-1107-g4c8e6f4
etcd 2.1.2

How reproducible:
Always

Steps to Reproduce:
1. Create a valid cloud.conf in /etc/origin/node
2. On node, edit /etc/origin/node/node-config.yaml, add following options:
kubeletArguments:
  cloud-provider:
    - "openstack"
  cloud-config:
    - "/etc/origin/node/cloud.conf"
3. systemctl restart atomic-openshift-node
4. journalctl -u atomic-openshift-node -f

Actual results:
After step 4:
Dec 22 14:44:53 atomic.nay.redhat.com systemd[1]: Started atomic-openshift-node.service.
Dec 22 14:44:55 atomic.nay.redhat.com docker[112547]: I1222 14:44:55.666096  112593 openstack.go:289] Claiming to support Instances
Dec 22 14:44:55 atomic.nay.redhat.com docker[112547]: E1222 14:44:55.723496  112593 kubelet.go:925] Unable to construct api.Node object for kubelet: failed to get external ID from cloud provider: Failed to find object
Dec 22 14:45:02 atomic.nay.redhat.com docker[112547]: I1222 14:45:02.147093  112593 openstack.go:289] Claiming to support Instances
Dec 22 14:45:02 atomic.nay.redhat.com docker[112547]: E1222 14:45:02.203757  112593 kubelet.go:925] Unable to construct api.Node object for kubelet: failed to get external ID from cloud provider: Failed to find object

Expected results:
Should not see such error from node log

Additional info:
Comment 1 Jan Safranek 2016-01-05 06:21:32 EST
This is typically caused by difference in hostname and OpenStack InstanceID. They do not need to be the same, but then please edit 'nodeName' item in node-config.yaml and set it to the OpenStack InstanceID.

If it does not help, please give me access to the machine under test.
Comment 2 Jianwei Hou 2016-01-07 03:37:39 EST
I've set the instance name same with the node hostname(in my case, it is openshift-145.lab.eng.nay.redhat.com), and it worked! Thank you!
Comment 3 Jan Safranek 2016-01-07 10:15:48 EST
Reassigning to OpenShift docs. We need to document 'nodeName' in node config yaml file.

Fix: https://github.com/openshift/openshift-docs/pull/1401
(feel free to rephrase it in any way...)
Comment 4 Jan Safranek 2016-01-15 07:03:54 EST
The PR got merged.
Comment 5 John Fulton 2016-01-18 10:41:18 EST
(In reply to Jan Safranek from comment #1)
> This is typically caused by difference in hostname and OpenStack InstanceID.
> They do not need to be the same, but then please edit 'nodeName' item in
> node-config.yaml and set it to the OpenStack InstanceID.

Jan,

Would you mind specifying what you mean by OpenStack InstanceID? 

Do you mean the ID that one would get from what's below (e.g. ff74f164-cf3e-4650-81e1-5a46f2819097) or the display name (e.g. jj-float162)? 

[jfulton@runcible osp_scripts]$ openstack server list 
+--------------------------------------+-------------+--------+---------------------------------------------+
| ID                                   | Name        | Status | Networks                                    |
+--------------------------------------+-------------+--------+---------------------------------------------+
| ff74f164-cf3e-4650-81e1-5a46f2819097 | jj-float162 | ACTIVE | private_network=172.16.66.10, 10.19.139.162 |
| 83bf37d5-e5c9-49ed-b35d-bf3082e8ffc0 | jj-float163 | ACTIVE | private_network=172.16.66.9, 10.19.139.163  |
+--------------------------------------+-------------+--------+---------------------------------------------+
[jfulton@runcible osp_scripts]$ 

You also mentioned the hostname. I can set a record in DNS but I could also just create an entry in /etc/hosts. If the later sufficient? What if I simply make what `hostname` returns consistent with the above by specifying the hostname of an instance on boot as described in the following KCS: 

 https://access.redhat.com/solutions/1315953

So there seem to be a few options but I was wondering if you could say what lookup exactly is OpenShift running to verify it has what it needs in order to get past this issue? 

Thanks,
  John
Comment 6 Jan Safranek 2016-01-20 07:54:00 EST
It's the 'Name' field, "jj-float162".

nodeName in the configuration yaml is the authoritative source of OpenStack (or any cloud) instance name. If it is empty, Kubernetes tries (in this order):
1. hostname-override command line argument of kubelet. node-config.yaml then looks like:
  kubeletArguments:  
    cloud-provider:  
      - "openstack"  
    cloud-config:  
      - "/etc/cloud.conf"  
    hostname-override:  
      - "<hostname>"  

2. hostname of the machine (i.e. result of "uname -n").

It does not really matter how your DNS is set as long as the local hostname is fine and your openshift node can talk to rest of the cluster. E.g. I use "hostnamectl set-hostname jsafrane-k2" on my testing machine in internal OS1.
Comment 8 Vikram Goyal 2016-02-15 22:17:06 EST
The fix is published here: https://docs.openshift.com/enterprise/3.1/install_config/configuring_openstack.html#openstack-configuring-nodes.

Moving this bug to CLOSED-->CURRENTRELEASE.

Note You need to log in before you can comment on or make changes to this bug.