Bug 1474092 - host=localhost in /etc/nova/nova.conf and /etc/neutron/neutron.conf on the compute nodes
Summary: host=localhost in /etc/nova/nova.conf and /etc/neutron/neutron.conf on the co...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 9.0 (Mitaka)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Emilien Macchi
QA Contact: Amit Ugol
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-07-23 20:37 UTC by David Hill
Modified: 2020-12-14 09:12 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-03-26 15:49:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1499201 0 urgent CLOSED OSP9 -> OSP10: workloads created before upgrade are not reachable anymore after rebooting controller nodes 2022-08-02 18:03:19 UTC

Internal Links: 1499201

Description David Hill 2017-07-23 20:37:31 UTC
Description of problem:
host=localhost in /etc/nova/nova.conf and /etc/neutron/neutron.conf on the compute nodes and needs to be manually changed to the proper hostname

Version-Release number of selected component (if applicable):


How reproducible:
In this deployment

Steps to Reproduce:
1. Scale-up a new compute
2.
3.

Actual results:
grep ^host /etc/nova/nova.conf
grep ^host /etc/neutron/neutron.conf

Expected results:
Should have the proper hostname

Additional info:
We're using predictable IPs without a hostnamemap (just it case we should be)

Comment 1 Sven Anderson 2017-07-28 14:45:21 UTC
Can you please explain how to reproduce this?

BTW.: The compute nodes have to have a name, just the IP will not work. It doesn't have to be dns resolvable, but there must be a name set. So that could be actually the problem.

Comment 2 David Hill 2017-07-28 15:16:28 UTC
Hello sir,

   I do not really know how this happens.  We're deploying the computes and instead of having "host" commented out so it uses the hostname of the compute, it is uncommented and set to localhost.  We have to manually login the computes and controllers in order to simply comment out those values in nova and neutron.  I tried finding where "host" is set in the puppet manifests but I couldn't find where.

Thank you very much,

David Hill

Comment 3 Sven Anderson 2017-08-04 15:07:04 UTC
(In reply to David Hill from comment #2)
> Hello sir,
> 
>    I do not really know how this happens.  We're deploying the computes and
> instead of having "host" commented out so it uses the hostname of the
> compute, it is uncommented and set to localhost.  We have to manually login
> the computes and controllers in order to simply comment out those values in
> nova and neutron.  I tried finding where "host" is set in the puppet
> manifests but I couldn't find where.

It's set right here (at least in the current version):
https://github.com/openstack/puppet-nova/blob/master/manifests/init.pp#L655

So it is either commented out, or it comes from outside (THT) as the $host parameter. So probably TripleO-Heat-Templates are setting it to localhost, because there is no hostnamemap, as you said.

Can you check to set a hostnamemap?

Comment 4 Sven Anderson 2017-08-18 14:22:22 UTC
Closing, because there is not enough info to reproduce the issue here. Please feel free to reopen, if more info becomes available.

Comment 5 Jeremy 2017-08-28 19:37:57 UTC
Hello,
I see this same issue. Everything is working fine. Customer does scale up deploy and all compute nodes service fails. host=localhost. Also same issue in neutron.conf changing to the correct host fixes. 

What can we check to see if templates are setting host=localhost?

Thanks,

Comment 8 Ollie Walsh 2017-08-31 20:31:53 UTC
The title is "host=localhost in /etc/nova/nova.conf..."

Yet the description states:

Actual results:
grep ^host /etc/nova/nova.conf

i.e host isn't set in nova.conf at all.

So which is correct?

I'll assume grep is correct as we do not set host in nova.conf via puppet.

This suggests the hostname isn't configured correctly due to an invalid hostnamemap e.g the index has incremented due to a deleted node or a failed scale out but the hostnamemap has not been incremented to match.

Comment 9 Ollie Walsh 2017-08-31 23:10:46 UTC
(In reply to Ollie Walsh from comment #8) 
> .. as we do not set host in nova.conf via puppet.

Actually we do e.g this is the hieradata::
"neutron::host": "%{::fqdn}",
"nova::host": "%{::fqdn}",

Which still suggests that the hostname has not been configured correctly.

Comment 12 awaugama 2017-09-14 19:25:24 UTC
Closed due to insufficent data or not a bug

Comment 13 David Hill 2017-10-07 14:19:04 UTC
We're able to reproduce this with another customer ...

Comment 14 Mark McLoughlin 2017-10-07 14:34:40 UTC
David: please describe what "this" is? And also link to the customer case with reproducer

If you mean bug #1499398, the root cause for host=localhost in these configuration files was:

1. The host name assigned via DHCP and presumably in DNS is osrz110023

2. The host name in the template files, including the host map, is osrtpz11023

3. The 'hostname' command on this host returns 'osrz110023.localdomain'

4. Because /etc/hosts only contains osrtpz11023.localdomain, the 'hostname -f' command returns 'localhost'

5. The value returned from 'hostname -f' is what goes into these config files

In this case - there is no software error. The hostname should be resolvable to a FQDN using 'hostname -f'. The /etc/hosts config just needs to be fixed to match the assigned hostname.

Comment 15 Mark McLoughlin 2017-10-07 14:41:22 UTC
I messed up my example, let me try again


If you mean bug #1499398, the root cause for host=localhost in these configuration files was:

1. The host name assigned via DHCP and presumably in DNS is foo0023

2. The host name in the template files, including the host map, is foo023
 
3. The 'hostname' command on this host returns 'foo0023.localdomain'
 
4. Because /etc/hosts only contains foo023.localdomain, the 'hostname -f' command returns 'localhost'

5. The value returned from 'hostname -f' is what goes into these config files

Comment 21 Alex Schultz 2017-10-18 20:04:33 UTC
I verified that this appears to no longer be an issue in 11. We'll have to look as to why this is occurring in 9.0.

Comment 22 Ben Nemec 2017-10-18 22:06:43 UTC
So in general changing host or domain names after the initial deployment is a bad idea.  Even if you don't hit this localhost problem (which seems to be a result of manual edits of /etc/hosts), any networks or instances that were created before the name change will be inaccessible afterward.  At least this has been my experience.

I'm not sure we document that and we should.  An RFE to disallow hostname or domain changes might also be appropriate, although I'm not entirely sure how we would do that.

Comment 23 Emilien Macchi 2018-03-26 15:49:33 UTC
As Alex said, this bug no longer appears in 11, also all related 3 cases are now closed, therefore we'll close the bug unless there is anything you want us to do, then please re-open.


Note You need to log in before you can comment on or make changes to this bug.