Bug 1647772 - Consider vendordata_ timeouts to 10 seconds by default for nova-join
Summary: Consider vendordata_ timeouts to 10 seconds by default for nova-join
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Rajesh Tailor
QA Contact: OSP DFG:Compute
URL:
Whiteboard:
Depends On:
Blocks: 1644747
TreeView+ depends on / blocked
 
Reported: 2018-11-08 10:42 UTC by Eduard Barrera
Modified: 2023-03-21 19:07 UTC (History)
10 users (show)

Fixed In Version: puddle: 2018-12-13.4
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-03 11:50:51 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 565130 0 None None None 2019-01-03 11:50:50 UTC
Red Hat Issue Tracker OSP-11749 0 None None None 2021-12-10 18:24:22 UTC

Description Eduard Barrera 2018-11-08 10:42:38 UTC 
Description of problem:

Recently we had a customer having random timeouts  with nova-join

 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]   File "/usr/lib/python2.7/site-packages/nova/api/metadata/vendordata_dynamic.py", line 118, in _do_request
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]     six.reraise(type(e), e, sys.exc_info()[2])
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]   File "/usr/lib/python2.7/site-packages/nova/api/metadata/vendordata_dynamic.py", line 100, in _do_request
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]     timeout=timeout)
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 698, in request
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]     resp = send(**kwargs)
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 763, in _send_request
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111]     raise exceptions.ConnectTimeout(msg)
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111] ConnectTimeout: Request to http://1.1.1.1:9090/v1/ timed out
2018-11-06 15:09:17.226 7921 ERROR nova.compute.manager [instance: 555cbad6-f81a-4828-b5f1-111] 

Incresing these timeouts from 5 to 10 solved the issue

vendordata_dynamic_connect_timeout=10
vendordata_dynamic_read_timeout=10

Would you consider changing the default value to 10 ?

Version-Release number of selected component (if applicable):
13

How reproducible:
random

Steps to Reproduce:
1. Use nova-join when deploying the overcloud
2.
3.

Actual results:
Deploy fails due timeout on nova-join

Expected results:
deploy success
Comment 1 Rajesh Tailor 2019-01-03 11:50:51 UTC 
Closing the BZ, as the required changes are already present in latest osp13 puddle.

grep -ri vendordatadynamic.*_timeout /etc/nova/nova.conf
[...]
vendordata_dynamic_connect_timeout=5

vendordata_dynamic_connect_timeout=30
vendordata_dynamic_read_timeout=5

vendordata_dynamic_read_timeout=30

got above output from /etc/nova/nova.conf on undercloud node, where timeout is set to 30 seconds, instead of 5 seconds.
Note You need to log in before you can comment on or make changes to this bug.