RDO tickets are now tracked in Jira https://issues.redhat.com/projects/RDO/issues/
Bug 1344507 - Nova novnc console doesn't load 2/3 times: Failed to connect to server (code: 1006)
Summary: Nova novnc console doesn't load 2/3 times: Failed to connect to server (code:...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: RDO
Classification: Community
Component: openstack-tripleo
Version: trunk
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: trunk
Assignee: James Slagle
QA Contact: Shai Revivo
URL:
Whiteboard:
Depends On:
Blocks: 1348558
TreeView+ depends on / blocked
 
Reported: 2016-06-09 20:04 UTC by Marius Cornea
Modified: 2017-09-04 20:25 UTC (History)
3 users (show)

Fixed In Version: openstack-tripleo-heat-templates-5.0.1-0.20161021142101.fa5c89c.el7.centos
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1348558 (view as bug list)
Environment:
Last Closed: 2017-06-22 15:24:50 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 328347 0 None None None 2016-10-21 11:55:31 UTC

Description Marius Cornea 2016-06-09 20:04:15 UTC
Description of problem:
Nova novnc console fails to load 2/3 requests in an HA environment with 3 controllers. The issue seems pretty much the same as the one reported in BZ#1238336

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-2.0.1-0.20160602175532.bcd726f.el7.centos.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy overcloud 
2. Log in to Horizon
3. Access instance console

Actual results:
2/3 times you get the following message:
Failed to connect to server (code: 1006)

Expected results:
The noVNC console always gets loaded.

Additional info:
Haproxy config:
listen nova_novncproxy
  bind 10.0.0.11:6080 transparent
  bind 172.16.18.25:6080 transparent
  balance source
  timeout tunnel 1h
  server overcloud-controller-0 10.0.0.15:6080 check fall 5 inter 2000 rise 2
  server overcloud-controller-1 10.0.0.14:6080 check fall 5 inter 2000 rise 2
  server overcloud-controller-2 10.0.0.13:6080 check fall 5 inter 2000 rise 2


nova-novncproxy.log shows the following messages for a failed connection:

2016-06-09 20:03:18.049 14778 INFO nova.console.websocketproxy [-] 10.0.0.15 - - [09/Jun/2016 20:03:18] 10.0.0.15: Plain non-SSL (ws://) WebSocket connection
2016-06-09 20:03:18.050 14778 INFO nova.console.websocketproxy [-] 10.0.0.15 - - [09/Jun/2016 20:03:18] 10.0.0.15: Version hybi-13, base64: 'False'
2016-06-09 20:03:18.050 14778 INFO nova.console.websocketproxy [-] 10.0.0.15 - - [09/Jun/2016 20:03:18] 10.0.0.15: Path: '/websockify'
2016-06-09 20:03:18.054 14778 WARNING oslo_config.cfg [req-626d918a-c898-470e-ac45-4388e935882d - - - - -] Option "rabbit_hosts" from group "oslo_messaging_rabbit" is deprecated for removal.  Its value may be silently ignored in the future.
2016-06-09 20:03:18.055 14778 WARNING oslo_config.cfg [req-626d918a-c898-470e-ac45-4388e935882d - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal.  Its value may be silently ignored in the future.
2016-06-09 20:03:18.055 14778 WARNING oslo_config.cfg [req-626d918a-c898-470e-ac45-4388e935882d - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal.  Its value may be silently ignored in the future.
2016-06-09 20:03:18.146 14778 INFO nova.console.websocketproxy [req-626d918a-c898-470e-ac45-4388e935882d - - - - -] handler exception: The token 'ccadff42-91ef-4716-9882-eba5edad987b' is invalid or has expired

Comment 1 Marius Cornea 2016-06-09 20:27:49 UTC
Workaround:

On controller nodes in /etc/nova/nova.conf add this cache section:

[cache]
backend = oslo_cache.memcache_pool
enabled = true
memcache_servers = 10.0.0.15:11211,10.0.0.14:11211,10.0.0.13:11211

Comment 2 Sven Anderson 2016-10-21 11:55:32 UTC
This should be fixed since https://review.openstack.org/#/c/328347/

Please verify with latest builds.

Comment 3 Christopher Brown 2017-06-22 15:24:50 UTC
Fixed, closing.


Note You need to log in before you can comment on or make changes to this bug.