Bug 1629331

Summary: docker container ironic_inspector_dnsmasq continuously restarts
Product: [Community] RDO Reporter: mike.oliveras
Component: openstack-tripleoAssignee: James Slagle <jslagle>
Status: CLOSED EOL QA Contact: Shai Revivo <srevivo>
Severity: high Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: jbiao, tahirgreat
Target Milestone: ---   
Target Release: trunk   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-06-15 20:12:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
undercloud.conf file used for "openstack undercloud install"
none
file imported wirh "openstack overcloud node import"
none
dnsmasq.conf file none

Description mike.oliveras 2018-09-14 21:57:20 UTC
Created attachment 1483412 [details]
undercloud.conf file used for "openstack undercloud install"

**NOTE** Version is Rocky (not listed in dropdown)

Description of problem:
I performed a tripleo install using settings in undercloud.conf (attached).
My nodes are using the manual-management driver.

When trying to introspect my nodes it was failing and I noticed that the docker
container is continuously restarting.



Version-Release number of selected component (if applicable):
python-tripleoclient-heat-installer-10.5.1-0.20180906012840.5b00837.el7.noarch
python2-tripleo-common-9.3.1-0.20180906092202.00906ff.el7.noarch
openstack-tripleo-puppet-elements-9.0.0-0.20180906013709.daf9069.el7.noarch
openstack-tripleo-heat-templates-9.0.0-0.20180912165049.31a97ad.el7.noarch
openstack-tripleo-common-containers-9.3.1-0.20180906092202.00906ff.el7.noarch
python-tripleoclient-10.5.1-0.20180906012840.5b00837.el7.noarch
openstack-tripleo-common-9.3.1-0.20180906092202.00906ff.el7.noarch
puppet-tripleo-9.3.1-0.20180831202649.8ec6c86.el7.noarch
openstack-tripleo-image-elements-9.0.0-0.20180831210308.2dc678a.el7.noarch
openstack-tripleo-validations-9.3.1-0.20180831205305.fbfd253.el7.noarch
python2-tripleo-repos-0.0.1-0.20180906025311.37b6276.el7.noarch
ansible-role-tripleo-modify-image-1.0.1-0.20180911205845.b47b569.el7.noarch
ansible-tripleo-ipsec-9.0.1-0.20180827143021.d2b9234.el7.noarch



How reproducible:
100%

Steps to Reproduce:
Install a tripleO on a baremetal server using the undercloud.conf provided
Actual results:


Expected results:
All docker containers should start and I should be able to introspect the nodes

Additional info:
I am also attaching my instackenv.json file.  I was able to import the nodes successfully.  I can provide any additional information that is needed, just let me know what you need.

Comment 1 mike.oliveras 2018-09-14 21:58:25 UTC
Created attachment 1483413 [details]
file imported wirh "openstack overcloud node import"

Comment 2 mike.oliveras 2018-09-14 22:02:38 UTC
Created attachment 1483414 [details]
dnsmasq.conf file

/var/lib/config-data/ironic_inspector/etc/ironic-inspector/dnsmasq.conf file (which I believe is the file used by the container)

Comment 3 mike.oliveras 2018-09-17 16:56:45 UTC
I ran "docker container logs  ironic_inspector_dnsmasq" and see:

+ echo 'Running command: '\''/sbin/dnsmasq --conf-file=/etc/ironic-inspector/dnsmasq.conf -k --log-facility=/var/log/ironic-inspector/dnsmasq.log'\'''
Running command: '/sbin/dnsmasq --conf-file=/etc/ironic-inspector/dnsmasq.conf -k --log-facility=/var/log/ironic-inspector/dnsmasq.log'
+ exec /sbin/dnsmasq --conf-file=/etc/ironic-inspector/dnsmasq.conf -k --log-facility=/var/log/ironic-inspector/dnsmasq.log
dnsmasq: failed to bind DHCP server socket: Address already in use


I stopped the container and saw that the tripleo server is still listening on the port, also verified that dnsmasq was stopped"

[root@director dhcp-hostsdir]# systemctl status dhcpd 
Unit dhcpd.service could not be found.
[root@director dhcp-hostsdir]# systemctl status dnsmasq
● dnsmasq.service - DNS caching server.
   Loaded: loaded (/usr/lib/systemd/system/dnsmasq.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

[root@director dhcp-hostsdir]# netstat -anup | grep :67
udp        0      0 0.0.0.0:67              0.0.0.0:*                           1956/dnsmasq        


so I did a "kill 1956" and we are no longer listening.

I then started the ironic_inspector_dnsmasq container again and it seems happy.

Comment 4 mike.oliveras 2018-09-17 17:36:06 UTC
after rebooting the tripleo machine, the ironic_inspector_dnsmasq container keeps restarting again for the same reason as noted above. I again have to kill the dnsmasq process before the container will successfully come up.

Comment 6 Tahir 2020-08-20 16:20:45 UTC
Hi All,

I am facing the same issue and solved it by the method mentioned above i.e. kill dnsmaq and restart the container it worked but i am still unable to do the introspection successfully.

do you have any idea about the next steps.


br,
tahir