Bug 1957030 - core dumps for ovsdb-sever and northd seen in OCP 4.7->4.8 upgrade
Summary: core dumps for ovsdb-sever and northd seen in OCP 4.7->4.8 upgrade
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux Fast Datapath
Classification: Red Hat
Component: ovsdb2.13
Version: RHEL 8.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Open vSwitch development team
QA Contact: ovs-qe
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-04 21:40 UTC by Tim Rozet
Modified: 2021-05-05 14:22 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)

Comment 1 Dan Williams 2021-05-05 03:59:58 UTC
Yet again, I get:

Core was generated by `ovn-northd --no-chdir -vconsole:info -vfile:off --ovnnb-db ssl:10.0.172.29:9641'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fa41b781d21 in __gconv_lookup_cache () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libcap-ng-0.7.5-4.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) t a a bt

Thread 1 (LWP 1):
#0  0x00007fa41b781d21 in __gconv_lookup_cache () from /lib64/libc.so.6
#1  0x0000000000000000 in ?? ()

Somebody should really try Dumitru's instructions in https://bugzilla.redhat.com/show_bug.cgi?id=1943413#c36 with one of the cores and make sure I'm doing it right.

Comment 2 Dan Williams 2021-05-05 14:16:17 UTC
ovsdb-server crash:

Core was generated by `ovsdb-server -vconsole:info -vfile:off --log-file=/var/log/ovn/ovsdb-server-nb.'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f158c800d21 in __gconv_lookup_cache () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libcap-ng-0.7.5-4.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) bt
#0  0x00007f158c800d21 in __gconv_lookup_cache () from /lib64/libc.so.6
#1  0x0000000000000000 in ?? ()
(gdb) t a a bt

Thread 2 (LWP 1):
#0  json_destroy (json=0x5590009c9960) at ../lib/json.c:354
#1  json_destroy_array (array=<optimized out>, array=<optimized out>) at ../lib/json.c:403
#2  json_destroy.part.10 () at ../lib/json.c:361
#3  0x0000558ffdc56b55 in json_destroy (json=<optimized out>) at ../lib/json.c:354
#4  json_destroy_array (array=<optimized out>, array=<optimized out>) at ../lib/json.c:403
#5  json_destroy.part.10 () at ../lib/json.c:361
#6  0x0000558ffdc56aac in json_destroy (json=<optimized out>) at ../lib/json.c:390
#7  json_destroy_object (object=0x559000b81b30) at ../lib/json.c:390
#8  json_destroy.part.10 () at ../lib/json.c:357
#9  0x0000558ffdc56aac in json_destroy (json=<optimized out>) at ../lib/json.c:390
#10 json_destroy_object (object=0x559000a2ea80) at ../lib/json.c:390
#11 json_destroy.part.10 () at ../lib/json.c:357
#12 0x0000558ffdc56aac in json_destroy (json=<optimized out>) at ../lib/json.c:390
#13 json_destroy_object (object=0x5590008c7fe0) at ../lib/json.c:390
#14 json_destroy.part.10 () at ../lib/json.c:357
#15 0x0000558ffdc56b55 in json_destroy (json=<optimized out>) at ../lib/json.c:354
#16 json_destroy_array (array=<optimized out>, array=<optimized out>) at ../lib/json.c:403
#17 json_destroy.part.10 () at ../lib/json.c:361
#18 0x0000558ffdc57f25 in json_destroy (json=<optimized out>) at ../lib/json.c:354
#19 0x0000558ffdc50136 in raft_entry_uninit (e=0x7f158ec860e0) at ../ovsdb/raft-private.c:294
#20 raft_entry_uninit (e=0x7f158ec860e0) at ../ovsdb/raft-private.c:291
#21 0x0000558ffdc4bc85 in raft_close (raft=0x558fffc92d70) at ../ovsdb/raft.c:1313
#22 0x0000558ffdc3a15e in ovsdb_storage_close (storage=0x558fffc966a0) at ../ovsdb/storage.c:138
#23 0x0000558ffdc3363f in ovsdb_destroy (db=0x558fffc966e0) at ../ovsdb/ovsdb.c:457
#24 0x0000558ffdc2cd8e in close_db (db=0x558fffc92fa0, comment=<optimized out>, config=<optimized out>) at ../ovsdb/ovsdb-server.c:540
#25 0x0000558ffdc2ab7f in main () at ../ovsdb/ovsdb-server.c:481
#26 0x00007f158c802803 in _nl_find_locale () from /lib64/libc.so.6
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 1 (LWP 214):
#0  0x00007f158c800d21 in __gconv_lookup_cache () from /lib64/libc.so.6
#1  0x0000000000000000 in ?? ()

Comment 3 Dan Williams 2021-05-05 14:17:35 UTC
I think these are the same as we've seen before in the upgrade tests; a crash on shutdown which should be harmless but still should be root-caused.


Note You need to log in before you can comment on or make changes to this bug.