Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 1954855

Summary:	northd crashes with OCP on upgrade
Product:	Red Hat Enterprise Linux Fast Datapath	Reporter:	Tim Rozet <trozet>
Component:	OVN	Assignee:	OVN Team <ovnteam>
Status:	CLOSED DUPLICATE	QA Contact:	Jianlin Shi <jishi>
Severity:	high	Docs Contact:
Priority:	medium
Version:	RHEL 8.0	CC:	ctrautma, dcbw
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-02-16 21:53:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1953613, 1956358

Description Tim Rozet 2021-04-28 22:16:54 UTC

Description of problem:
On multiple master nodes I see northd cores. In dmesg:
[327088.861386] traps: ovn-northd[1074285] general protection fault ip:7f34897f1d21 sp:7ffeea050a80 error:0 in libc-2.28.so[7f34897d0000+1b9000]

Version-Release number of selected component (if applicable):
ovn2.13-20.12.0-24.el8fdp.x86_64

OCP upgrade is from 4.7.3->4.7.6 which uses the same version of OVN.

Will attach core.

Comment 2 Dan Williams 2021-04-29 00:43:39 UTC

Note that there is no relevant difference between 4.7.3 and 4.7.6 in terms of RPMs. Only things that changed were NetworkManager, crio, and a few others.

Comment 3 Dan Williams 2021-04-29 01:31:27 UTC

Core was generated by `ovn-northd --no-chdir -vconsole:info -vfile:off --ovnnb-db ssl:172.21.8.40:9641'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f73b8c11d21 in __gconv_lookup_cache () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libcap-ng-0.7.5-4.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) bt
#0  0x00007f73b8c11d21 in __gconv_lookup_cache () from /lib64/libc.so.6
#1  0x0000000000000000 in ?? ()
(gdb)

Comment 4 Dan Williams 2021-04-29 01:43:46 UTC

core.ovn-northd.0.f78fa98b65584355b04b61fdb11da53e.4029.1619103625000000 (same as above)
---
Core was generated by `ovn-northd --no-chdir -vconsole:info -vfile:off --ovnnb-db ssl:172.21.8.40:9641'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f73b8c11d21 in __gconv_lookup_cache () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libcap-ng-0.7.5-4.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) bt
#0  0x00007f73b8c11d21 in __gconv_lookup_cache () from /lib64/libc.so.6
#1  0x0000000000000000 in ?? ()
(gdb)


core.ovn-northd.0.8e76dd4e0f6b41e5acc7184ebb6282db.415827.1619172149000000
---
Core was generated by `ovn-northd --no-chdir -vconsole:info -vfile:off --ovnnb-db ssl:172.21.8.40:9641'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fc063c55d21 in __gconv_lookup_cache () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libcap-ng-0.7.5-4.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) bt
#0  0x00007fc063c55d21 in __gconv_lookup_cache () from /lib64/libc.so.6
#1  0x0000000000000000 in ?? ()
(gdb) 


core.ovn-northd.0.ece4d0223aa04befa987e38ecb7b9869.1063219.1619104182000000
---
Core was generated by `ovn-northd --no-chdir -vconsole:info -vfile:off --ovnnb-db ssl:172.21.8.40:9641'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f000035ed21 in __gconv_lookup_cache () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libcap-ng-0.7.5-4.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) bt
#0  0x00007f000035ed21 in __gconv_lookup_cache () from /lib64/libc.so.6
#1  0x0000000000000000 in ?? ()
(gdb)

Comment 5 Dan Williams 2022-02-16 21:53:55 UTC


*** This bug has been marked as a duplicate of bug 1957030 ***