Bug 901968 - openstack-nova-compute service fails to start when ipv6 is enabled
openstack-nova-compute service fails to start when ipv6 is enabled
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova (Show other bugs)
2.0 (Folsom)
Unspecified Unspecified
low Severity high
: rc
: 5.0 (RHEL 7)
Assigned To: Brent Eagles
Toni Freger
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-01-20 04:49 EST by Rami Vaknin
Modified: 2016-04-26 12:40 EDT (History)
6 users (show)

See Also:
Fixed In Version: 2013.2.2-4.el6ost
Doc Type: Known Issue
Doc Text:
ipv6 is supported in RHELOSP 5.0, but is not supported in RHOS 2.1. If you configure Compute in RHOS 2.1 with 'use_ipv6=true', the service will not start properly.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-07-08 11:26:12 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
compute.log (13.06 KB, application/x-tar)
2013-01-20 04:53 EST, Rami Vaknin
no flags Details

  None (edit)
Description Rami Vaknin 2013-01-20 04:49:54 EST
Version:
Folsom on RHEL6.4
openstack-nova-scheduler-2012.2.2-5.el6ost.noarch
openstack-nova-compute-2012.2.2-5.el6ost.noarch
python-nova-2012.2.2-5.el6ost.noarch
openstack-nova-network-2012.2.2-5.el6ost.noarch
openstack-nova-objectstore-2012.2.2-5.el6ost.noarch
openstack-nova-volume-2012.2.2-5.el6ost.noarch
openstack-nova-2012.2.2-5.el6ost.noarch
python-novaclient-2.10.0-1.el6ost.noarch
openstack-nova-common-2012.2.2-5.el6ost.noarch
openstack-nova-cert-2012.2.2-5.el6ost.noarch
openstack-nova-console-2012.2.2-5.el6ost.noarch
openstack-nova-novncproxy-0.4-2.el6.noarch
openstack-nova-api-2012.2.2-5.el6ost.noarch

Scenario:
I've enabled ipv6 by adding "use_ipv6=true" to /etc/nova/nova.conf, then I've restarted nova services and noticed that openstack-nova-compute fails to start - "dead but pid file exists" - Unexpected error while running command "sudo nova-rootwrap /etc/nova/rootwrap.conf ip6tables-restore -c", exit code 2:

Additional Information:

From ifconfig
=============
br100     Link encap:Ethernet  HWaddr 44:1E:A1:73:40:42  
          inet addr:192.168.7.1  Bcast:192.168.7.255  Mask:255.255.255.0
          inet6 addr: fe80::784d:94ff:fe44:a8e1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:594 (594.0 b)


From /var/log/nova/compute.log
==============================
2013-01-20 09:23:06 DEBUG nova.utils [req-62876277-0031-46fe-b2b6-c0978803831d None None] Result was 2 execute /usr/lib/python2.6/site-packages/nova/utils.py:199
2013-01-20 09:23:06 23738 CRITICAL nova [-] Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip6tables-restore -c
Exit code: 2
Stdout: ''
Stderr: "ip6tables-restore v1.4.7: host/network `None' not found\nError occurred at line: 28\nTry `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.\n"
2013-01-20 09:23:06 23738 TRACE nova Traceback (most recent call last):
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/bin/nova-compute", line 48, in <module>
2013-01-20 09:23:06 23738 TRACE nova     service.wait()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/service.py", line 659, in wait
2013-01-20 09:23:06 23738 TRACE nova     _launcher.wait()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/service.py", line 192, in wait
2013-01-20 09:23:06 23738 TRACE nova     super(ServiceLauncher, self).wait()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/service.py", line 162, in wait
2013-01-20 09:23:06 23738 TRACE nova     service.wait()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 166, in wait
2013-01-20 09:23:06 23738 TRACE nova     return self._exit_event.wait()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/eventlet/event.py", line 116, in wait
2013-01-20 09:23:06 23738 TRACE nova     return hubs.get_hub().switch()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch
2013-01-20 09:23:06 23738 TRACE nova     return self.greenlet.switch()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 192, in main
2013-01-20 09:23:06 23738 TRACE nova     result = function(*args, **kwargs)
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/service.py", line 132, in run_server
2013-01-20 09:23:06 23738 TRACE nova     server.start()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/service.py", line 398, in start
2013-01-20 09:23:06 23738 TRACE nova     self.manager.init_host()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 324, in init_host
2013-01-20 09:23:06 23738 TRACE nova     self._legacy_nw_info(net_info))
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2479, in ensure_filtering_rules_for_instance
2013-01-20 09:23:06 23738 TRACE nova     network_info)
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/virt/firewall.py", line 188, in prepare_instance_filter
2013-01-20 09:23:06 23738 TRACE nova     self.refresh_provider_fw_rules()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/virt/firewall.py", line 454, in refresh_provider_fw_rules
2013-01-20 09:23:06 23738 TRACE nova     self.iptables.apply()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/network/linux_net.py", line 345, in apply
2013-01-20 09:23:06 23738 TRACE nova     self._apply()
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/utils.py", line 752, in inner
2013-01-20 09:23:06 23738 TRACE nova     retval = f(*args, **kwargs)
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/network/linux_net.py", line 371, in _apply
2013-01-20 09:23:06 23738 TRACE nova     attempts=5)
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/network/linux_net.py", line 949, in _execute
2013-01-20 09:23:06 23738 TRACE nova     return utils.execute(*cmd, **kwargs)
2013-01-20 09:23:06 23738 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/utils.py", line 206, in execute
2013-01-20 09:23:06 23738 TRACE nova     cmd=' '.join(cmd))
2013-01-20 09:23:06 23738 TRACE nova ProcessExecutionError: Unexpected error while running command.
2013-01-20 09:23:06 23738 TRACE nova Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip6tables-restore -c
2013-01-20 09:23:06 23738 TRACE nova Exit code: 2
2013-01-20 09:23:06 23738 TRACE nova Stdout: ''
2013-01-20 09:23:06 23738 TRACE nova Stderr: "ip6tables-restore v1.4.7: host/network `None' not found\nError occurred at line: 28\nTry `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.\n"
2013-01-20 09:23:06 23738 TRACE nova
Comment 1 Rami Vaknin 2013-01-20 04:53:22 EST
Created attachment 683505 [details]
compute.log
Comment 4 Stephen Gordon 2013-03-18 12:45:46 EDT
Removing the release note flag, we're covering this under Bug # 894440.
Comment 9 Brent Eagles 2014-05-15 12:19:04 EDT
This appears to have been resolved in 2013.2.2-4.el6ost (and possibly earlier). Manually verified by creating an IPv6 network and booting a VM. 

Some notes:
 - If IPv6 is initially set as disabled and networks are created that do not include IPv6 information, enabling IPv6 breaks things as nova attempts to use the empty IPv6 related fields and results in errors.
 - IPv6 forwarding was not enabled by packstack 2013.2.1-0.28.dev989.el6ost2013.2.1-0.28.dev989.el6ost so I had to modify this manually.
Comment 11 Toni Freger 2014-06-16 04:13:42 EDT
The nova-compute seems stable after IPv6 is set to "true" and all Nova's services are restarted.
After the changes the VMs booted successfully without any errors.

Also, I have changed the IPv6 to the default value,and  didn't see any errors within nova log file. Nova's services restarted and VMs work fine.
Comment 14 Stephen Gordon 2014-06-27 11:49:35 EDT
Hi Summer, I'd actually like to drop this release note - I think it confuses matters w.r.t. the other IPv6 note.
Comment 16 errata-xmlrpc 2014-07-08 11:26:12 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0853.html

Note You need to log in before you can comment on or make changes to this bug.