Bug 1570615

Summary: random scenario tempest tests fail due to Floating IP not reachable
Product: Red Hat OpenStack Reporter: Waldemar Znoinski <wznoinsk>
Component: opendaylightAssignee: Aswin Suryanarayanan <asuryana>
Status: CLOSED ERRATA QA Contact: Waldemar Znoinski <wznoinsk>
Severity: high Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: asuryana, jschluet, knylande, mkolesni, nyechiel, sclewis, trozet, vthapar
Target Milestone: betaKeywords: Triaged
Target Release: 13.0 (Queens)Flags: asuryana: needinfo-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: odl_netvirt
Fixed In Version: opendaylight-8.0.0-8.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
N/A
Last Closed: 2018-06-27 13:52:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
opendaylight_osp13_flapping_fip.html
none
new random FIP issue none

Description Waldemar Znoinski 2018-04-23 10:45:35 UTC
Created attachment 1425641 [details]
opendaylight_osp13_flapping_fip.html

Description of problem:
Random scenario tempest tests fail due to 'connection to Floating IP timeout'.
errors:
- Connection to the 10.0.0.219 via SSH timed out (see 'test_snapshot_pattern' in 'opendaylight_osp13_flapping_fip.html' attachment)
- Timed out waiting for 10.0.0.215 to become reachable (see 'test_server_connectivity_stop_start' in 'opendaylight_osp13_flapping_fip.html' attachment)
- other relating to not being able to ssh connect to FIP (while VM already has received fixed IP)

Version-Release number of selected component (if applicable):
opendaylight-8.0.0-5
osp13, puddle 2018-04-19.2

How reproducible:
10%, can be reproduced only if multiple scenario tempest tests are run at the same time (even in serial, concurrenty=1, mode)

Steps to Reproduce:
1. deploy OSP with opendaylight
2. run tempest with --regex tempest.scenario.*
3.

Actual results:
random scenario tempest tests fail to connect to floating IP of VMs

Expected results:
connectivity to FIP of VMs work, tests to pass

Additional info:

Comment 1 Waldemar Znoinski 2018-04-23 10:47:06 UTC
this bug may be as a result of https://bugzilla.redhat.com/show_bug.cgi?id=1556686 but it hasn't been confirmed

Comment 2 Aswin Suryanarayanan 2018-04-23 12:12:31 UTC
This bug is expected to be fixed by [1]. Now running the CI with the patch to see if there is any improvements in the no of failures.

[1]https://code.engineering.redhat.com/gerrit/#/c/136293/

Comment 8 Aswin Suryanarayanan 2018-04-30 07:07:14 UTC
The Flapping FIP issue is not observed after merging [1]

[1]https://bugzilla.redhat.com/show_bug.cgi?id=1570615

Comment 16 Waldemar Znoinski 2018-05-09 11:58:52 UTC
Created attachment 1433766 [details]
new random FIP issue

Comment 17 Waldemar Znoinski 2018-05-09 11:59:53 UTC
it looked like the random FIP problems went away (didn't appear in last ~2 weeks), it happened last night (it may be same or a new FIP issue) - see the above attachment

Comment 20 errata-xmlrpc 2018-06-27 13:52:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086