RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1782177 - Calendar refresh blocks factory's processing thread
Summary: Calendar refresh blocks factory's processing thread
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: evolution-ews
Version: 8.1
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: rc
: 8.3
Assignee: Milan Crha
QA Contact: Desktop QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-11 10:49 UTC by Grant Williamson
Modified: 2020-03-18 09:44 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-18 09:44:54 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
gdb of process (33.70 KB, text/plain)
2019-12-11 10:49 UTC, Grant Williamson 		no flags Details
evolution source registry debug (47.42 KB, text/plain)
2019-12-11 10:51 UTC, Grant Williamson 		no flags Details
gdb of calendar hang (29.68 KB, text/plain)
2019-12-11 12:06 UTC, Grant Williamson 		no flags Details
gdb of calendar hang # 3 (41.94 KB, text/plain)
2019-12-11 12:16 UTC, Grant Williamson 		no flags Details
Evolution Calendar Factory Subprocess (27.99 KB, text/plain)
2019-12-11 12:33 UTC, Grant Williamson 		no flags Details
View All

Description Grant Williamson 2019-12-11 10:49:18 UTC 
Created attachment 1643840 [details]
gdb of process

Description of problem:
 

Create a account, when calendar is retrieved hangs evolution-ews

evolution-3.28.5-9.el8.x86_64

"
How does it hang it? If you mean evolution itself, then could you install debuginfo for evolution-data-server, evolution, evolution-ews, glib2 and gtk3 (make sure the package versions will match those already installed), and then get a backtrace of the stuck process(-es), please? You can get the backtrace with command like this:
   $ gdb --batch --ex "t a a bt" -pid=`pidof evolution` &>bt.txt
Please check the bt.txt for any private information, like passwords, email address, server addresses,... I usually search for "pass" at least (quotes for clarity only).
"
Comment 1 Grant Williamson 2019-12-11 10:51:47 UTC 
Created attachment 1643841 [details]
evolution source registry debug
Comment 2 Milan Crha 2019-12-11 11:16:46 UTC 
Thanks for a bug report (follow up of bug #1772103). Despite having not installed any debuginfo packages, the backtrace shows that the evolution process is waiting for a response from the evolution-calendar-factory-subprocess process. Those responses should be relatively quick, at least when asking for a timezone, which the main (GUI) thread does. I guess the main thread of the factory process is blocked by an ongoing refresh of the EWS calendar(s).

That would be shown in a backtrace of that evolution-calendar-factory-subprocess and the time needed for the refresh depends on the size of the calendars. Could you run the gdb command above and instead of using `pidof evolution` use a process ID of the running evolution-calendar-factory-subprocess, please? You can get the process ID with:

   $ ps ax | grep evolution-cale

where one process will be 'evolution-calendar-factory' and the other will be 'evolution-calendar-factory-subprocess', which we are interested in. The same rules about privacy from comment #0 apply here too.
Comment 3 Milan Crha 2019-12-11 11:50:16 UTC 
I'm moving this bug to RHEL7, as its counterpart had been moved there too.
Comment 5 Grant Williamson 2019-12-11 11:51:42 UTC 
I am testing this on EL8. 

"Despite having not installed any debuginfo packages"
You requested these packages. they were installed.
evolution-data-server-debuginfo.x86_64 evolution-debuginfo.x86_64 glib2-debuginfo gtk3-debuginfo
The debug packages and the packages are not in sync, not your problem. Might be handy if they were.

Let me see if I can get them in sync.
Comment 6 Grant Williamson 2019-12-11 12:04:51 UTC 
This is what is installed.
evolution-3.28.5-6.el8.x86_64
evolution-debuginfo-3.28.5-6.el8.x86_64
evolution-debugsource-3.28.5-6.el8.x86_64

evolution-data-server-3.28.5-9.el8.x86_64
evolution-data-server-debuginfo-3.28.5-9.el8.x86_64
evolution-data-server-debugsource-3.28.5-9.el8.x86_64

evolution-data-server-langpacks-3.28.5-9.el8.noarch
evolution-ews-3.28.5-5.el8.x86_64
evolution-ews-langpacks-3.28.5-5.el8.noarch
evolution-langpacks-3.28.5-6.el8.noarch

glib2-2.56.4-1.el8.x86_64
glib2-debuginfo-2.56.4-1.el8.x86_64
glib2-debugsource-2.56.4-1.el8.x86_64
glib2-devel-2.56.4-1.el8.x86_64

gtk3-3.22.30-3.el8.x86_64
gtk3-debuginfo-3.22.30-3.el8.x86_64
gtk3-debugsource-3.22.30-3.el8.x86_64
gtk3-devel-3.22.30-3.el8.x86_64
Comment 7 Grant Williamson 2019-12-11 12:06:13 UTC 
Created attachment 1643862 [details]
gdb of calendar hang
Comment 8 Grant Williamson 2019-12-11 12:07:52 UTC 
ignore, adding missing package.
Comment 9 Grant Williamson 2019-12-11 12:16:30 UTC 
With this installed.
evolution-3.28.5-6.el8.x86_64
evolution-debuginfo-3.28.5-6.el8.x86_64
evolution-debugsource-3.28.5-6.el8.x86_64

evolution-data-server-3.28.5-9.el8.x86_64
evolution-data-server-debuginfo-3.28.5-9.el8.x86_64
evolution-data-server-debugsource-3.28.5-9.el8.x86_64
evolution-data-server-langpacks-3.28.5-9.el8.noarch

evolution-ews-3.28.5-3.el8.x86_64
evolution-ews-debuginfo-3.28.5-3.el8.x86_64
evolution-ews-debugsource-3.28.5-3.el8.x86_64

evolution-ews-langpacks-3.28.5-3.el8.noarch
evolution-langpacks-3.28.5-6.el8.noarch

glib2-2.56.4-1.el8.x86_64
glib2-debuginfo-2.56.4-1.el8.x86_64
glib2-debugsource-2.56.4-1.el8.x86_64
glib2-devel-2.56.4-1.el8.x86_64

gtk3-3.22.30-3.el8.x86_64
gtk3-debuginfo-3.22.30-3.el8.x86_64
gtk3-debugsource-3.22.30-3.el8.x86_64
gtk3-devel-3.22.30-3.el8.x86_64

running
gdb --batch --ex "t a a bt" -pid=`pidof evolution` &>bt.txt

See bt2-el8.txt
Comment 10 Grant Williamson 2019-12-11 12:16:59 UTC 
Created attachment 1643865 [details]
gdb of calendar hang # 3
Comment 11 Grant Williamson 2019-12-11 12:33:31 UTC 
Created attachment 1643869 [details]
Evolution Calendar Factory Subprocess
Comment 12 Milan Crha 2019-12-11 12:40:09 UTC 
(In reply to Grant Williamson from comment #5)
> I am testing this on EL8. 

Oh, do you? Why? The initial bug was RHEL7 and this one RHEL8? That's confusing and it might be also hard to work with, I'd say. Nonetheless, I'm moving this back to RHEL8 then.
Comment 13 Grant Williamson 2019-12-11 12:42:43 UTC 
The previous issue was on EL7. 
As you already stated they are 2 separated issues, unrelated.
Comment 14 Milan Crha 2019-12-11 12:56:32 UTC 
You are right with the debuginfo packages, they should match version with their binary package counterparts, otherwise it doesn't work. The last backtrace shows debug symbols properly, thanks for it. It also shows that the factory's main thread is not blocked, the thread which runs the operation is blocked by the calendar content refresh/download. It'll make it harder to fix, the operations which blocks the evolution process is piled in a queue and waiting to be served. It's slightly different than having blocked the main thread. I'm not sure how to fix this yet.
Comment 15 Grant Williamson 2019-12-11 13:00:28 UTC 
The interesting thing being. When we use the flatpak from flathub we do not see this issue.
I guess running in a sandbox prevents the blocking we are seeing.

See if you can find some kind of workaround.
Comment 16 Milan Crha 2019-12-11 13:08:13 UTC 
The flatpak version is most like 3.34.x. There had been done a lot of work between 3.28.5 and 3.34.x. Maybe something can be backported. I'll check it out and let you know if/when I find anything (might be early next year, because I'm starting vacation today).
Comment 17 Grant Williamson 2019-12-11 13:32:26 UTC 
Enjoy your vacation. 

I will test on Fedora 31 also.
Comment 18 Milan Crha 2020-02-12 15:19:36 UTC 
I searched the code and as far as I can tell the related part of processing the refresh request is quite the same both in the current stable upstream version and in RHEL 8. That makes me think that the actual issue is somewhere else, maybe it's just a coincidence that it misbehaves in RHEL for you, but works flawlessly in Flatpak.

The backtrace shows that the code is waiting for a response from the server, which blocks the processing queue, because all requests are processed serially. I can change the code in evolution-data-server to not use the processing queue as it is used today, but that won't help much, because libsoup's session is currently configured to use one connection only, thus things could still starve, only in a different queue. Such change could help to local operations only and even that is not sure.

I think, and only think, that there was something similar with libsoup, maybe lost requests or such, but I can be also wrong. It would be weird to see different server response times in a host machine and in Flatpak, especially when Flatpak version would be quicker.

If it's really about latest stable versions working properly, and those shipped in RHEL 8 not that well, then I'd rather wait for newer version in RHEL, than to try to pick only certain changes, because these things depend on multiple libraries, like libsoup, glib-networking, gnutls and all these depend on, thus to get whole (most of the) stack at once. I even do not know how to find a related change for such issue.
Note You need to log in before you can comment on or make changes to this bug.