Bug 1467496
Summary: | zero byte ifcfg files after overcloud deployment in Ravello | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Michael Jarrett <mjarrett> | ||||
Component: | openstack-neutron | Assignee: | Assaf Muller <amuller> | ||||
Status: | CLOSED DUPLICATE | QA Contact: | Toni Freger <tfreger> | ||||
Severity: | urgent | Docs Contact: | |||||
Priority: | unspecified | ||||||
Version: | 10.0 (Newton) | CC: | amuller, beagles, bfournie, chrisw, dbecker, ftaylor, jlibosva, jslagle, mburns, mcornea, mjarrett, morazi, mweetman, nyechiel, rhel-osp-director-maint, rlocke, sclewis, srevivo | ||||
Target Milestone: | --- | Keywords: | ZStream | ||||
Target Release: | --- | ||||||
Hardware: | Unspecified | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2017-07-20 13:14:03 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
Michael Jarrett
2017-07-04 04:23:35 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release. Michael or Forrest - can you provide a sosreport when this occurs and/or set me up in an environment where I can duplicate this? SSO ID is bfournie. Thanks. I was able to reproduce this bug doing the training, specifically in Chapter 6 - Scaling Overcloud Nodes. The steps described in the video cause the failure as described on compute1: - ifcfg files are empty - "ip a" shows no address for eth0 or any other interfaces It appears that the deployment to add an overcloud node has not completed successfully. Looking at the logs on the undercloud, this appears to be a Neutron bug that has been fixed. I see this in /var/log/neutron/openvswitch-agent.log: 2017-07-01 08:07:47.039 24375 ERROR ryu.lib.hub [-] hub: uncaught exception: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ryu/lib/hub.py", line 54, in _launch return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/ryu/base/app_manager.py", line 545, in close self.uninstantiate(app_name) File "/usr/lib/python2.7/site-packages/ryu/base/app_manager.py", line 528, in uninstantiate app = self.applications.pop(name) KeyError: 'ofctl_service' 2017-07-01 08:07:47.203 24375 INFO oslo_rootwrap.client [-] Stopping rootwrap daemon process with pid=24451 The bug is https://bugzilla.redhat.com/show_bug.cgi?id=1425507 See https://bugzilla.redhat.com/show_bug.cgi?id=1425507#c15 the stack trace in the comment matches this trace exactly and the comment describes the issue being seen: "Moreover this issue appears to be affecting not only overcloud upgrades but undercloud upgrade as well, preventing operations such as adding overcloud nodes." Note that packages installed in undercloud: [stack@director log]$ rpm -aq | grep openvswitch openstack-neutron-openvswitch-9.2.0-2.el7ost.noarch openvswitch-2.5.0-14.git20160727.el7fdp.x86_64 python-openvswitch-2.5.0-14.git20160727.el7fdp.noarch I'm attaching the openvswitch_agent.log file that shows the error Created attachment 1301448 [details]
openvswitch_agent.log with ryu exception
I recommend upgrading the training system to latest OSP-11 build to pick up this bug fix. If that's not possible for this training setup there may be workarounds available. I'm going to add some people who were involved in https://bugzilla.redhat.com/show_bug.cgi?id=1425507 in case they can recommend a workaround. I will leave this bug open for a short while before closing it as a duplicate. So, upgrading to OSP 11 is not an option for our training environment. We have made a commitment to the "extended release" versions so need to stay on OSP 10 (course will likely be revised for OSP 13). The current classroom environment is running OSP 10.0.2. Has this been resolved in 10.0.4 or a subsequent maintenance release of 10.0.z? >So, upgrading to OSP 11 is not an option for our training environment. We have made a >commitment to the "extended release" versions so need to stay on OSP 10 (course will >likely be revised for OSP 13). I understand. >The current classroom environment is running OSP 10.0.2. Has this been resolved in >10.0.4 or a subsequent maintenance release of 10.0.z? Marius or Jakub - can you indicate if there is a patch in OSP 10 for this openvswitch ryu issue? Thanks. (In reply to Bob Fournier from comment #9) > >So, upgrading to OSP 11 is not an option for our training environment. We have made a >commitment to the "extended release" versions so need to stay on OSP 10 (course will >likely be revised for OSP 13). > > I understand. > > >The current classroom environment is running OSP 10.0.2. Has this been resolved in >10.0.4 or a subsequent maintenance release of 10.0.z? > > Marius or Jakub - can you indicate if there is a patch in OSP 10 for this > openvswitch ryu issue? Thanks. There is a solved bug for OSP10 overcloud [1], RDO patch backported to OSP10 is here [2]. For undercloud, reboot of the node is required as per [3]. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1450223 [2] https://review.rdoproject.org/r/#/c/6648 [3] https://bugzilla.redhat.com/show_bug.cgi?id=1444883#c14 >There is a solved bug for OSP10 overcloud [1], RDO patch backported to OSP10 is here >[2]. For undercloud, reboot of the node is required as per [3]. >[1] https://bugzilla.redhat.com/show_bug.cgi?id=1450223 >[2] https://review.rdoproject.org/r/#/c/6648 >[3] https://bugzilla.redhat.com/show_bug.cgi?id=1444883#c14 Thanks a lot Jakub. I'm closing this as a duplicate, note that according to [1] the fix is in 10.0.z3 Please pick up release and reboot undercloud. *** This bug has been marked as a duplicate of bug 1450223 *** |