Bug 1567917 - Scale out failed ironic node introspection timed out, mixed versions upgrade
Summary: Scale out failed ironic node introspection timed out, mixed versions upgrade
Keywords:
Status: CLOSED DUPLICATE of bug 1570386
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: James Slagle
QA Contact: Arik Chernetsky
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-16 12:30 UTC by Ronnie Rasouli
Modified: 2018-04-23 14:13 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-23 14:13:34 UTC
Target Upstream Version:


Attachments (Terms of Use)
iptables rules (13.02 KB, text/plain)
2018-04-16 12:30 UTC, Ronnie Rasouli
no flags Details
ironic conductor log (8.78 MB, text/plain)
2018-04-16 12:36 UTC, Ronnie Rasouli
no flags Details

Description Ronnie Rasouli 2018-04-16 12:30:12 UTC
Created attachment 1422426 [details]
iptables rules

Description of problem:

performed mixed versions stage upgrade, undercloud upgrade to RHOS13 and scaled out 1 extra compute node, the scale out of that node timed out.

node did not received dhcp, then the scale out stuck 

06:59:49.626034 52:54:00:0f:20:b7 > Broadcast, ethertype IPv4 (0x0800), length 438: 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:0f:20:b7, length 396




Version-Release number of selected component (if applicable):
openstack-ironic-api-10.1.2-2.el7ost.noarch
openstack-ironic-conductor-10.1.2-2.el7ost.noarch
openstack-ironic-common-10.1.2-2.el7ost.noarch

instack-undercloud-8.4.0-3.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. deploy RHOS12 
2. upgrade the undercloud
3. perform scale out for additional compute node

Actual results:
Scale out timeout after 1 hour

Expected results:


Additional info:

var/log/ironic-inspector/ironic-inspector.log:198:2018-04-16 02:58:18.160 27985 DEBUG ironic_inspector.firewall [-] Ignoring failed iptables ('-D', 'INPUT', '-i', 'br-ctlplane', '-p', 'udp', '--dport', '67', '-j', 'ironic-inspector'): iptables v1.4.21: Couldn't load target `ironic-inspector':No such file or directory. . Try `iptables -h' or 'iptables...

Comment 1 Ronnie Rasouli 2018-04-16 12:36:26 UTC
Created attachment 1422427 [details]
ironic conductor log

Comment 3 Ronnie Rasouli 2018-04-17 08:00:02 UTC
Disabling ipatbles does the trick - the ironic introspection pass

Comment 4 Bob Fournier 2018-04-23 14:13:34 UTC
This is being tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1570386, patches are upstream.

*** This bug has been marked as a duplicate of bug 1570386 ***


Note You need to log in before you can comment on or make changes to this bug.