Bug 1777768 - [OSP14] Packets dropped during minor update
Summary: [OSP14] Packets dropped during minor update
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: async
: 14.0 (Rocky)
Assignee: Saravanan KR
QA Contact: Ronnie Rasouli
URL:
Whiteboard:
Depends On:
Blocks: 1770345
TreeView+ depends on / blocked
 
Reported: 2019-11-28 09:59 UTC by Jose Luis Franco
Modified: 2019-12-20 16:49 UTC (History)
7 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.3.1-0.20190513171773.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-20 16:49:31 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 696482 0 'None' 'NEW' '[QueensOnly] Update ovs to 2.11 without network loss' 2019-12-08 11:33:07 UTC
Red Hat Product Errata RHBA-2019:4339 0 None None None 2019-12-20 16:49:38 UTC

Description Jose Luis Franco 2019-11-28 09:59:02 UTC
Description of problem:

The ping test running during the OSP14 minor update procedure in the CI is reporting a 4% packages loss:

TASK [tripleo-upgrade : stop l3 agent connectivity check] **********************
task path: /home/rhos-ci/jenkins/workspace/DFG-upgrades-updates-14-from-GA-composable-ipv6-scale-up/infrared/plugins/tripleo-upgrade/infrared_plugin/roles/tripleo-upgrade/tasks/common/l3_agent_connectivity_check_stop_script.yml:2
Thursday 28 November 2019  02:22:12 +0000 (1:41:16.677)       2:10:47.192 ***** 
fatal: [undercloud-0]: FAILED! => {
    "changed": true, 
    "cmd": "source /home/stack/qe-Cloud-0rc\n /home/stack/l3_agent_stop_ping.sh", 
    "delta": "0:00:00.105060", 
    "end": "2019-11-27 21:22:13.038575", 
    "rc": 1, 
    "start": "2019-11-27 21:22:12.933515"
}

STDOUT:

6078 packets transmitted, 5802 received, +168 errors, 4% packet loss, time 6083460ms
rtt min/avg/max/mdev = 0.541/1.409/1005.763/13.211 ms, pipe 4
Ping loss higher than 1% detected


MSG:

non-zero return code

	to retry, use: --limit @/home/rhos-ci/jenkins/workspace/DFG-upgrades-updates-14-from-GA-composable-ipv6-scale-up/infrared/plugins/tripleo-upgrade/infrared_plugin/main.retry

PLAY RECAP *********************************************************************
undercloud-0               : ok=71   changed=33   unreachable=0    failed=1   

Log: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/DFG-upgrades-updates-14-from-GA-composable-ipv6-scale-up/4/artifact/.sh/ir-tripleo-overcloud-update.log

The job logs were not stored, probably due to the Jenkins restart happening yesterday evening. One more logs are collected they will be added into the BZ.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Trigger CI job https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/upgrades/view/update/job/DFG-upgrades-updates-14-from-GA-composable-ipv6-scale-up/
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Lukas Bezdicka 2019-11-29 10:49:59 UTC
With patch https://review.opendev.org/#/c/696482/ applied we got 5879 packets transmitted, 5879 received, 0% packet loss, time 5884614ms

Comment 18 Ronnie Rasouli 2019-12-08 11:56:40 UTC
12862 packets transmitted, 12860 received, 0% packet loss, time 12874402ms on ping test during the upgrade process

Comment 20 errata-xmlrpc 2019-12-20 16:49:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:4339


Note You need to log in before you can comment on or make changes to this bug.