Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1607967

Summary: UnexpectedSource: got a response from ('2001:4860:4860:0:0:0:0:8888', 53, 0, 0) instead of ('2001:4860:4860::8888', 53, 0, 0)
Product: Red Hat OpenStack Reporter: David Vallee Delisle <dvd>
Component: python-eventletAssignee: Lon Hohberger <lhh>
Status: CLOSED ERRATA QA Contact: Shai Revivo <srevivo>
Severity: high Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: bcafarel, dvd, joflynn, jschluet, lhh, marjones, mburns, mkrcmari, pforsber, ragiman, srevivo
Target Milestone: z2Keywords: Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-eventlet-0.20.1-5.1.el7ost Doc Type: Bug Fix
Doc Text:
There was an issue in python-eventlet UDP address handling that resulted in some IPv6 addresses being handled incorrectly in some cases. As a result, when receiving DNS responses via UDP, python-eventlet ignored the response and stalled for several seconds, severely impacting performance. This issue is now resolved.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2018-08-29 16:33:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Backport (part 1)
none
Backport (part 2)
none
Unit tests (part 3) none

Description David Vallee Delisle 2018-07-24 15:54:18 UTC 
Description of problem:

When using short notation in resolv.conf, dns resolution times out after 30 seconds

Version-Release number of selected component (if applicable):


How reproducible:
All the time

Steps to Reproduce:
1. Use short notation in resolv.conf
2. Snoop dns queries with tcpdump
3. Find a bunch of NXRECORD everytime a hostname is looked up, thus slowing down deployment and other functions
~~~
82	2018-07-06 14:17:03.637557	2001:4860:4860:0:0:0:0:8888	2001:4860:4860:0:0:0:0:8888	DNS	199	Standard query response 0x2c34 No such name AAAA blabla.com SOA blablabla.com
~~~

Actual results:
Everytime a hostname lookup is executed, it takes 30 seconds before falling back on ipv4 because the reply is ignored because the source is considered as being different, even though it's the right source.

Expected results:
We should accept results that that match the source, no matter if we use short or long notation in resolv.conf

Additional info:
I think we should probably develop a better validation than just == in eventlet/support/greendns.py:

~~~
627         while 1:
628             try:
629                 (wire, from_address) = s.recvfrom(65535)
630             except socket.timeout:
631                 # Q: Do we also need to catch coro.CoroutineSocketWake and pass?
632                 if expiration - time.time() <= 0.0:
633                     raise dns.exception.Timeout
634             if from_address == destination:
635                 break
636             if not ignore_unexpected:
637                 raise dns.query.UnexpectedSource(
638                     'got a response from %s instead of %s'
639                     % (from_address, destination))
~~~
Comment 2 David Vallee Delisle 2018-07-24 16:02:39 UTC 
For information, our customer is having this error in the logs:

2018-07-24 12:39:48.515 55 ERROR dns.resolver [req-c444b6ff-a64c-41a8-bc81-801081c59832 - - - - -] : UnexpectedSource: got a response from ('2001:4860:4860:0:0:0:0:8888', 53, 0, 0) instead of ('2001:4860:4860::8888', 53, 0, 0)
Comment 5 David Vallee Delisle 2018-07-25 23:04:54 UTC 
Something like this might be a better way to compare IPv6

~~~
Python 2.7.15 (default, May 15 2018, 15:37:31) 
[GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from socket import inet_pton, AF_INET6
>>> source = inet_pton(AF_INET6, "2001:4860:4860::8888")
>>> dest = inet_pton(AF_INET6, "2001:4860:4860:0:0:0:0:8888")
>>> if source == dest:
...     print("same")
... 
same
>>> print("Source %s Dest: %s", (source, dest))
('Source %s Dest: %s', (' \x01H`H`\x00\x00\x00\x00\x00\x00\x00\x00\x88\x88', ' \x01H`H`\x00\x00\x00\x00\x00\x00\x00\x00\x88\x88'))
~~~
Comment 9 Lon Hohberger 2018-07-27 20:32:28 UTC 
https://github.com/eventlet/eventlet/pull/510/commits/a2d40db2074ddb8039e414fa969c13aca006590e
Comment 10 Lon Hohberger 2018-07-31 18:25:04 UTC 
Merged upstream: https://github.com/eventlet/eventlet/pull/510
Comment 11 Lon Hohberger 2018-07-31 18:58:51 UTC 
Created attachment 1471908 [details]
Backport (part 1)
Comment 12 Lon Hohberger 2018-07-31 18:59:14 UTC 
Created attachment 1471909 [details]
Backport (part 2)
Comment 13 Lon Hohberger 2018-07-31 18:59:41 UTC 
Created attachment 1471910 [details]
Unit tests (part 3)
Comment 23 Joanne O'Flynn 2018-08-13 11:18:36 UTC 
This bug is marked for inclusion in the errata but does not currently contain draft documentation text. To ensure the timely release of this advisory please provide draft documentation text for this bug as soon as possible.

If you do not think this bug requires errata documentation, set the requires_doc_text flag to "-".


To add draft documentation text:

* Select the documentation type from the "Doc Type" drop down field.

* A template will be provided in the "Doc Text" field based on the "Doc Type" value selected. Enter draft text in the "Doc Text" field.
Comment 31 Bernard Cafarelli 2018-08-27 15:11:04 UTC 
eventlet upstream unit tests pass with the fix + no relevant regressions identified in OSP CI, so marking this VERIFIED
Comment 34 errata-xmlrpc 2018-08-29 16:33:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2573