Bug 1564227 - dnsmasq not showing accurate result [NEEDINFO]
Summary: dnsmasq not showing accurate result
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.6.0
Hardware: Unspecified
OS: Linux
unspecified
high
Target Milestone: ---
: 3.11.0
Assignee: Ben Bennett
QA Contact: Meng Bo
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-05 17:59 UTC by sudhir agarwal
Modified: 2018-05-15 15:51 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-05-15 15:51:16 UTC
Target Upstream Version:
bbennett: needinfo? (sudhiragarwal619)


Attachments (Terms of Use)

Description sudhir agarwal 2018-04-05 17:59:29 UTC
Description of problem:
When I am trying to curl the route of an application which is hosted in another host and another project, the route is getting resolved by the dnsmasq. I am able to see the packets with DNS query and reply in source pod's eth0 interface which is getting resolved by dnsmasq (node's eth0 interface with port 53). But when I collect packets in dnsmasq (node's eth0 interface with port 53), I don't see the packets with DNS query and reply of route being resolved. 

Version-Release number of selected component (if applicable):
3.6

How reproducible:
100%

Steps to Reproduce:
1. Create 2 applications in different nodes and different projects.
2. Login to one of the application and try to curl the route of other application
3. At the same time, collect the tcpdump packets in:
   (a) first in the pod's eth0 interface
   (b) then in the node's eth0 interface with port 53 (dnsmasq)

Actual results:
Here, we can see the DNS query and reply of route being resolved (which is getting resolved by dnsmasq) in the pod's eth0 interface but not in the packets captured in the dnsmasq.

Expected results:
We should be able to see the DNS query and reply packets(of route being resolved) in the dnsmasq (node's eth0 interface with port 53) also.

Additional info:

Comment 1 Ben Bennett 2018-04-09 19:20:34 UTC
What does /etc/resolv.conf on the pod say?

Comment 2 sudhir agarwal 2018-04-10 06:24:02 UTC
I have installed 2 applications - jenkins and redmine, in 2 different nodes and different projects. And i am doing curl of the route of redmine application from the jenkins application. So in the jenkins application, the /etc/resolv.conf file is :
sh-4.2$ cat /etc/resolv.conf
nameserver 172.30.70.115
search project-b.svc.cluster.local svc.cluster.local cluster.local nectechnologies.in apps67.nectechnologies.in
options ndots:5


And in the redmine application, the /etc/resolv.conf file is:
# cat /etc/resolv.conf
nameserver 172.30.70.78
search project-a.svc.cluster.local svc.cluster.local cluster.local nectechnologies.in apps67.nectechnologies.in
options ndots:5

Comment 3 sudhir agarwal 2018-04-10 07:04:06 UTC
(In reply to sudhir agarwal from comment #2)
> I have installed 2 applications - jenkins and redmine, in 2 different nodes
> and different projects. And i am doing curl of the route of redmine
> application from the jenkins application. So in the jenkins application, the
> /etc/resolv.conf file is :
> sh-4.2$ cat /etc/resolv.conf
> nameserver 172.30.70.115
> search project-b.svc.cluster.local svc.cluster.local cluster.local
> example.com apps67.example.com
> options ndots:5
> 
> 
> And in the redmine application, the /etc/resolv.conf file is:
> # cat /etc/resolv.conf
> nameserver 172.30.70.78
> search project-a.svc.cluster.local svc.cluster.local cluster.local
> example.com apps67.example.com
> options ndots:5

Comment 4 Ben Bennett 2018-04-10 20:03:47 UTC
What does 'ip addr' on the node the redmine pod is running on show?

Comment 5 sudhir agarwal 2018-04-11 09:49:56 UTC
(In reply to Ben Bennett from comment #4)
> What does 'ip addr' on the node the redmine pod is running on show?

Please find the output of the "ip addr" on the node on which redmine pod is running:
http://paste.openstack.org/show/718884/


Also, please find the /etc/dnsmasq.conf file contents in the following link. you might need this information as well:
http://paste.openstack.org/show/718885/

Comment 6 Ben Bennett 2018-04-16 19:32:19 UTC
What command are you using to capture the packets on the node?  It should be looking at interface enp3s0, and note that positive responses could have been cached in the pod.  So you may need to try resolving new domains to see the traffic on the node.

Comment 7 sudhir agarwal 2018-04-24 05:51:35 UTC
(In reply to Ben Bennett from comment #6)
> What command are you using to capture the packets on the node?  It should be
> looking at interface enp3s0, and note that positive responses could have
> been cached in the pod.  So you may need to try resolving new domains to see
> the traffic on the node.

i am using tcpdump command:
tcpdump -i enp3s0 -w node_interface.pcap

Comment 8 Ben Bennett 2018-05-01 19:56:04 UTC
Can you share that node_interface.pcap file please?

(Assuming you have captured a time you attempt to resolve from a pod on the same node)

Please make sure that you try to resolve new names so that they are not cached somewhere.


Note You need to log in before you can comment on or make changes to this bug.