Description of problem: Machine powered off Start machine Login Crash occurs every time shortly after login Version-Release number of selected component: evolution-3.46.1-1.fc37 Additional info: reporter: libreport-2.17.4 backtrace_rating: 4 cgroup: 0::/user.slice/user-1000.slice/user/app.slice/app-gnome-org.gnome.Evolution-3814.scope cmdline: evolution crash_function: g_bsearch_array_free executable: /usr/bin/evolution journald_cursor: s=45ff2b6d98b741c1893ec23a4e98d9c0;i=259ae;b=6db8d6275b384c61b9f2a101e0b3adf7;m=6863cde7;t=5ee0f9ae37833;x=83580080ac05e4f7 kernel: 6.0.9-300.fc37.x86_64 rootdir: / runlevel: N 5 type: CCpp uid: 1000
Created attachment 1926567 [details] File: backtrace
Created attachment 1926568 [details] File: core_backtrace
Created attachment 1926569 [details] File: cpuinfo
Created attachment 1926570 [details] File: dso_list
Created attachment 1926571 [details] File: environ
Created attachment 1926572 [details] File: limits
Created attachment 1926573 [details] File: maps
Created attachment 1926574 [details] File: mountinfo
Created attachment 1926575 [details] File: open_fds
Created attachment 1926576 [details] File: proc_pid_status
Created attachment 1926577 [details] File: var_log_messages
Thanks for a bug report. I do not face this crash myself, but there's a user facing it in Ubuntu, which is reported upstream at: https://gitlab.gnome.org/GNOME/evolution/-/issues/2131 It looks like a problem in the libsoup3. Could you install debuginfo packages for glib2, glib-networking, libsoup3, gnutls, evolution-data-server, evolution and evolution-ews, and then run evolution under valgrind, which can catch some memory issues, please? To install the debuginfo packages you can run: sudo dnf install glib2-debuginfo glib-networking-debuginfo libsoup3-debuginfo gnutls-debuginfo evolution-data-server-debuginfo evolution-debuginfo evolution-ews-debuginfo --enablerepo=fedora-debuginfo --enablerepo=updates-debuginfo only make sure the versions installed will match the installed packages. Then you can run Evolution as a regular user under valgrind (it can be installed with `sudo dnf install valgrind`) with these commands: export GIGACAGE_ENABLED=0 export WEBKIT_FORCE_SANDBOX=0 export Malloc=1 G_SLICE=always-malloc valgrind --num-callers=30 --leak-check=no --track-origins=yes --aspace-minaddr=0x100000000 evolution &>log.txt Note the evolution will be very slow, due to all the memory checking, and there will be a high CPU use as well due to it. Valgrind can prevent certain crashes, it "only" logs about them, thus even if the evolution will be slow and running, the log.txt file can contain valuable information about memory misuse. If it'll run, then keep it for some time and eventually close it regularly, then upload the file here, please. Check it for any private information first, but I suppose there will be none. It can also happen that the slowness prevents conditions for the problem if it is related to certain timing.
Looking at the attached backtrace, it seems to be crashing at the same place for me, i.e., soup_connection_manager_get_connection. See https://gitlab.gnome.org/GNOME/evolution/-/issues/2131#note_1596408.
I have been running evolution under valgrind all day and not experienced a single crash. Tomorrow ill try running it as usual again and see if crashes reappear or if it has disappeared.
(In reply to Ulf Gunnarson from comment #14) > I have been running evolution under valgrind all day and not experienced a > single crash. Does the log show anything useful? See the last paragraph of the comment #12.
Log output: ==4314== Memcheck, a memory error detector ==4314== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==4314== Using Valgrind-3.20.0 and LibVEX; rerun with -h for copyright info ==4314== Command: evolution ==4314== ==4815== ==4815== HEAP SUMMARY: ==4815== in use at exit: 15,893,115 bytes in 145,500 blocks ==4815== total heap usage: 468,077 allocs, 322,577 frees, 45,283,636 bytes allocated ==4815== ==4815== For a detailed leak analysis, rerun with: --leak-check=full ==4815== ==4815== For lists of detected and suppressed errors, rerun with: -s ==4815== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) ==4314== Warning: set address range perms: large range [0x12571a000, 0x16571c000) (defined) ==4314== Warning: set address range perms: large range [0x16d71c000, 0x36d71c000) (defined) ==4314== Warning: set address range perms: large range [0x16d71c000, 0x200000000) (noaccess) ==4314== Warning: set address range perms: large range [0x300000000, 0x36d71c000) (noaccess) --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. (evolution:4314): camel-ews-provider-WARNING **: 10:34:09.948: camel_ews_folder_update_thread: The specified object was not found in the store., The process failed to get the correct properties. --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. Opening in existing browser session. libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null) ==51756== ==51756== HEAP SUMMARY: ==51756== in use at exit: 68,097,898 bytes in 473,879 blocks ==51756== total heap usage: 205,133,367 allocs, 204,659,488 frees, 39,087,303,581 bytes allocated ==51756== ==51756== For a detailed leak analysis, rerun with: --leak-check=full ==51756== ==51756== For lists of detected and suppressed errors, rerun with: -s ==51756== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. --4314-- WARNING: unhandled amd64-linux syscall: 434 --4314-- You may be able to write your own handler. --4314-- Read the file README_MISSING_SYSCALL_OR_IOCTL. --4314-- Nevertheless we consider this a bug. Please report --4314-- it at http://valgrind.org/support/bug_reports.html. (evolution:4314): GLib-GIO-WARNING **: 14:28:04.663: Your application did not unregister from D-Bus before destruction. Consider using g_application_run(). ==4314== ==4314== HEAP SUMMARY: ==4314== in use at exit: 45,778,384 bytes in 322,389 blocks ==4314== total heap usage: 223,963,228 allocs, 223,640,839 frees, 42,787,362,213 bytes allocated ==4314== ==4314== For a detailed leak analysis, rerun with: --leak-check=full ==4314== ==4314== For lists of detected and suppressed errors, rerun with: -s ==4314== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
I see, thanks. It looks like the slowness caused by the memory checking prevented the problem to occur.
(In reply to Milan Crha from comment #17) > I see, thanks. It looks like the slowness caused by the memory checking > prevented the problem to occur. I had a similar experience with valgrind.
I cannot tell whether this will catch anything, but it may worth a try. Install libasan, please, and then run Evolution as this: export GIGACAGE_ENABLED=0 export WEBKIT_FORCE_SANDBOX=0 export Malloc=1 export G_SLICE=always-malloc LD_PRELOAD=/usr/lib64/libasan.so.8 evolution
(In reply to Milan Crha from comment #19) > I cannot tell whether this will catch anything, but it may worth a try. > Install libasan, please, and then run Evolution as this: > > export GIGACAGE_ENABLED=0 > export WEBKIT_FORCE_SANDBOX=0 > export Malloc=1 > export G_SLICE=always-malloc > LD_PRELOAD=/usr/lib64/libasan.so.8 evolution Just tried it. Crashed almost right away with no additional output other than what I've seen before.
Tried as well, sofar no crash. I dont experience the frequent crashes anymore, only "add password to keyring for exchange server" every reboot. Have full log if needed. ~$ export GIGACAGE_ENABLED=0 export WEBKIT_FORCE_SANDBOX=0 export Malloc=1 export G_SLICE=always-malloc LD_PRELOAD=/usr/lib64/libasan.so.8 evolution ================================================================= ==7593==ERROR: LeakSanitizer: detected memory leaks Direct leak of 760 byte(s) in 95 object(s) allocated from: #0 0x7f19eecba097 in calloc (/usr/lib64/libasan.so.8+0xba097) #1 0x7f19edf415f0 in g_malloc0 (/lib64/libglib-2.0.so.0+0x5f5f0) SUMMARY: AddressSanitizer: 760 byte(s) leaked in 95 allocation(s). ==8319==Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING. ==8319==ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range. ==8319==This might be related to ELF_ET_DYN_BASE change in Linux 4.12. ==8319==See https://github.com/google/sanitizers/issues/856 for possible workarounds. ==8319==Process memory map follows: . . .
Hrm, I wish I could reproduce it locally. It would be much easier to debug. I've no idea what else to try.
Before reporting i had alot of crashes, after installing the various software to try to log stuff and other updates i have not had a single crash.
I still have a lot of crashes with the installed software. Could this be a stack smasher?
(In reply to Ulf Gunnarson from comment #23) > Before reporting i had alot of crashes, after installing the various > software to try to log stuff and other updates i have not had a single crash. Could you confirm how you are running this not to get a crash?
I ran evolution like suggested previously with extra software installed to try to log more info : export GIGACAGE_ENABLED=0 export WEBKIT_FORCE_SANDBOX=0 export Malloc=1 export G_SLICE=always-malloc LD_PRELOAD=/usr/lib64/libasan.so.8 evolution When i experienced no crashes for several days i begun starting evolution like i normally do from app menu and sofar no crash. I thought id go back to "debugway" if i once more start experiencing crashes.
(In reply to Raffi Khatchadourian from comment #24) > I still have a lot of crashes with the installed software. Could this be a > stack smasher? There's definitely something breaking the data structures, but the tools which can help to find the cause do not help in this particular case. The first step can be to make sure libsoup3 3.2.2 or newer is installed, because it fixes one similar problem with thread safety.
(In reply to Ulf Gunnarson from comment #26) > I ran evolution like suggested previously with extra software installed to > try to log more info : > export GIGACAGE_ENABLED=0 > export WEBKIT_FORCE_SANDBOX=0 > export Malloc=1 > export G_SLICE=always-malloc > LD_PRELOAD=/usr/lib64/libasan.so.8 evolution > > When i experienced no crashes for several days i begun starting evolution > like i normally do from app menu and sofar no crash. Could you confirm the Evolution version you are using?
(In reply to Milan Crha from comment #27) > (In reply to Raffi Khatchadourian from comment #24) > > I still have a lot of crashes with the installed software. Could this be a > > stack smasher? > > There's definitely something breaking the data structures, but the tools > which can help to find the cause do not help in this particular case. > > The first step can be to make sure libsoup3 3.2.2 or newer is installed, > because it fixes one similar problem with thread safety. I tried upgrading, no good. I also tried downgrading with no luck also. I am now running the stock version of libsoup in Ubuntu and I downgraded my Evolution (using flatpak) to 3.44.4. Now, I am not experiencing crashing.
(In reply to Raffi Khatchadourian from comment #28) > (In reply to Ulf Gunnarson from comment #26) > > I ran evolution like suggested previously with extra software installed to > > try to log more info : > > export GIGACAGE_ENABLED=0 > > export WEBKIT_FORCE_SANDBOX=0 > > export Malloc=1 > > export G_SLICE=always-malloc > > LD_PRELOAD=/usr/lib64/libasan.so.8 evolution > > > > When i experienced no crashes for several days i begun starting evolution > > like i normally do from app menu and sofar no crash. > > Could you confirm the Evolution version you are using? Version im currently: 3.46.1 (3.46.1-1.fc37)
Interesting. On 3.46.1-0ubuntu1, I am constantly crashing. Upgrading or downgrading libsoup doesn't make a difference. Now I'm on 3.44.4 using flatpak (see below), and I have no crashed: $ flatpak info org.gnome.Evolution Evolution - Manage your email, contacts and schedule ID: org.gnome.Evolution Ref: app/org.gnome.Evolution/x86_64/stable Arch: x86_64 Branch: stable Version: 3.44.4 License: GPL-2.0+ and GFDL-1.3 Origin: flathub Collection: org.flathub.Stable Installation: system Installed: 61.9 MB Runtime: org.gnome.Platform/x86_64/42 Sdk: org.gnome.Sdk/x86_64/42 Commit: def74e5bce7c7a789f878f9d1562acfaa431edd6ff29902525d8e37dd1b8a736 Parent: c7a8d2741616cefb7faa53bbd8ec2ae27a20244e5cadbecff2dc82416b36f8d2 Subject: Update to 3.44.4 (93088d3b) Date: 2022-08-08 17:15:08 +0000
After receiving updated evolution, 3.46.2 (3.46.2-1.fc37), I've begun to experience crashes once more. This morning it crashed on startup while running using libasan. Output below ~$ export GIGACAGE_ENABLED=0 export WEBKIT_FORCE_SANDBOX=0 export Malloc=1 export G_SLICE=always-malloc LD_PRELOAD=/usr/lib64/libasan.so.8 evolution ================================================================= ==9406==ERROR: LeakSanitizer: detected memory leaks Direct leak of 760 byte(s) in 95 object(s) allocated from: #0 0x7fc96c6ba097 in calloc (/usr/lib64/libasan.so.8+0xba097) #1 0x7fc96b9415f0 in g_malloc0 (/lib64/libglib-2.0.so.0+0x5f5f0) SUMMARY: AddressSanitizer: 760 byte(s) leaked in 95 allocation(s). ================================================================= (evolution:9387): GLib-CRITICAL **: 08:23:25.920: iter_remove_or_steal: assertion 'ri->version == ri->hash_table->version' failed (evolution:9387): GLib-CRITICAL **: 08:23:25.920: g_hash_table_iter_next: assertion 'ri->version == ri->hash_table->version' failed ==9387==ERROR: AddressSanitizer: attempting double-free on 0x60e002b1c3a0 in thread T10: #0 0x7fc3b98b9388 in __interceptor_free.part.0 (/usr/lib64/libasan.so.8+0xb9388) #1 0x7fc3b95ae8cc in g_free (/lib64/libglib-2.0.so.0+0x5b8cc) #2 0x7fc3b3e4aa72 in soup_auth_ntlm_free_connection_state ../libsoup/auth/soup-auth-ntlm.c:298 #3 0x7fc3b88af553 in _g_closure_invoke_va ../gobject/gclosure.c:895 #4 0x7fc3b88af553 in g_signal_emit_valist ../gobject/gsignal.c:3456 #5 0x7fc3b88af632 in g_signal_emit (/lib64/libgobject-2.0.so.0+0x31632) #6 0x7fc3b3e903b3 in soup_connection_list_disconnect_all ../libsoup/soup-connection-manager.c:292 #7 0x7fc3b3e903b3 in soup_connection_manager_get_connection_locked ../libsoup/soup-connection-manager.c:420 #8 0x7fc3b3e903b3 in soup_connection_manager_get_connection ../libsoup/soup-connection-manager.c:502 #9 0x7fc3b3e903b3 in soup_session_ensure_item_connection ../libsoup/soup-session.c:1696 #10 0x7fc3b3e903b3 in soup_session_process_queue_item ../libsoup/soup-session.c:1747 #11 0x7fc3b3e90d2f in async_run_queue ../libsoup/soup-session.c:1854 #12 0x7fc3b3e90de6 in queue_dispatch ../libsoup/soup-session.c:187 #13 0x7fc3b95a9cbe in g_main_dispatch ../glib/gmain.c:3444 #14 0x7fc3b95ff597 in g_main_context_iterate.constprop.0 ../glib/gmain.c:4238 #15 0x7fc3b95a928e in g_main_loop_run (/lib64/libglib-2.0.so.0+0x5628e) #16 0x7fc39d02d36d in e_ews_soup_worker_thread (/usr/lib64/evolution-ews/libevolution-ews.so+0x2a36d) #17 0x7fc3b95d39c1 in g_thread_proxy ../glib/gthread.c:831 #18 0x7fc3b50ae14c in start_thread (/lib64/libc.so.6+0x8b14c) #19 0x7fc3b512f9ff in clone3 (/lib64/libc.so.6+0x10c9ff) 0x60e002b1c3a0 is located 0 bytes inside of 154-byte region [0x60e002b1c3a0,0x60e002b1c43a) freed by thread T13 here: #0 0x7fc3b98b9388 in __interceptor_free.part.0 (/usr/lib64/libasan.so.8+0xb9388) #1 0x7fc3b95ae8cc in g_free (/lib64/libglib-2.0.so.0+0x5b8cc) #2 0x60b0001ab8df (<unknown module>) previously allocated by thread T10 here: #0 0x7fc3b98ba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7fc3b95b2168 in g_malloc (/lib64/libglib-2.0.so.0+0x5f168) Thread T10 created by T2 here: #0 0x7fc3b984b3e6 in __interceptor_pthread_create (/usr/lib64/libasan.so.8+0x4b3e6) #1 0x7fc3b95d6b60 in g_system_thread_new ../glib/gthread-posix.c:1324 #2 0x7fc3b95d6b60 in g_thread_new_internal ../glib/gthread.c:936 Thread T2 created by T0 here: #0 0x7fc3b984b3e6 in __interceptor_pthread_create (/usr/lib64/libasan.so.8+0x4b3e6) #1 0x7fc3b95d6b60 in g_system_thread_new ../glib/gthread-posix.c:1324 #2 0x7fc3b95d6b60 in g_thread_new_internal ../glib/gthread.c:936 Thread T13 created by T0 here: #0 0x7fc3b984b3e6 in __interceptor_pthread_create (/usr/lib64/libasan.so.8+0x4b3e6) #1 0x7fc3b95d6b60 in g_system_thread_new ../glib/gthread-posix.c:1324 #2 0x7fc3b95d6b60 in g_thread_new_internal ../glib/gthread.c:936 SUMMARY: AddressSanitizer: double-free (/usr/lib64/libasan.so.8+0xb9388) in __interceptor_free.part.0 ==9387==ABORTING ~$ ================================================================= ==9441==ERROR: LeakSanitizer: detected memory leaks Direct leak of 17888 byte(s) in 117 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb9cf039f in bmalloc_try_allocate_impl_casual_case.constprop.0 (/lib64/libjavascriptcoregtk-4.1.so.0+0xef039f) Direct leak of 12284 byte(s) in 561 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb9cf00af in bmalloc_allocate_impl_casual_case.constprop.0 (/lib64/libjavascriptcoregtk-4.1.so.0+0xef00af) Direct leak of 5376 byte(s) in 7 object(s) allocated from: #0 0x7f5ebdcb95b5 in __interceptor_realloc.part.0 (/usr/lib64/libasan.so.8+0xb95b5) #1 0x7f5eb8d6aba8 in FcPatternObjectInsertElt (/lib64/libfontconfig.so.1+0x20ba8) Direct leak of 48 byte(s) in 1 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb7039168 in g_malloc (/lib64/libglib-2.0.so.0+0x5f168) Indirect leak of 9123 byte(s) in 131 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb9cf00af in bmalloc_allocate_impl_casual_case.constprop.0 (/lib64/libjavascriptcoregtk-4.1.so.0+0xef00af) Indirect leak of 5152 byte(s) in 161 object(s) allocated from: #0 0x7f5ebdcba097 in calloc (/usr/lib64/libasan.so.8+0xba097) #1 0x7f5eb8d6a5ba in FcValueListDuplicate (/lib64/libfontconfig.so.1+0x205ba) Indirect leak of 2432 byte(s) in 76 object(s) allocated from: #0 0x7f5ebdcba097 in calloc (/usr/lib64/libasan.so.8+0xba097) #1 0x7f5eb8d6ecac in FcPatternObjectAddWithBinding (/lib64/libfontconfig.so.1+0x24cac) Indirect leak of 1344 byte(s) in 42 object(s) allocated from: #0 0x7f5ebdcba097 in calloc (/usr/lib64/libasan.so.8+0xba097) #1 0x7f5eb8d6a46f in FcValueListPrepend (/lib64/libfontconfig.so.1+0x2046f) Indirect leak of 1341 byte(s) in 98 object(s) allocated from: #0 0x7f5ebdc7243b in strdup (/usr/lib64/libasan.so.8+0x7243b) #1 0x7f5eb8d6a3b7 in FcValueSave (/lib64/libfontconfig.so.1+0x203b7) Indirect leak of 336 byte(s) in 7 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb8d60877 in FcLangSetCopy (/lib64/libfontconfig.so.1+0x16877) Indirect leak of 96 byte(s) in 3 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb8d5679f in FcConfigValues (/lib64/libfontconfig.so.1+0xc79f) Indirect leak of 81 byte(s) in 2 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb7039168 in g_malloc (/lib64/libglib-2.0.so.0+0x5f168) Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f5ebdcba6af in __interceptor_malloc (/usr/lib64/libasan.so.8+0xba6af) #1 0x7f5eb8d5656e in FcConfigEvaluate (/lib64/libfontconfig.so.1+0xc56e) SUMMARY: AddressSanitizer: 55533 byte(s) leaked in 1207 allocation(s). ~$
Thanks for the update. The `soup_auth_ntlm_free_connection_state` is bug #2150603, which links to an upstream bug on the libsoup3 side. I hope they'll look on it soon. Let's mark this one a duplicate of the other. *** This bug has been marked as a duplicate of bug 2150603 ***
Could you follow bug #2144358 comment #16 , please? Maybe it'll help here as well, I do not know.
I have installed the update. Wont be able to run for extended time until next week though