Bug 1187650 - Correct locking in e_book_backend_ews_set_locale()
Summary: Correct locking in e_book_backend_ews_set_locale()
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: evolution-ews
Version: 21
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Milan Crha
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-01-30 15:08 UTC by Adam D.
Modified: 2015-05-27 16:20 UTC (History)
2 users (show)

Fixed In Version: evolution-ews-3.12.11-2.fc21
Clone Of:
Environment:
Last Closed: 2015-05-27 16:20:55 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Log from addressbook-factory post-kill. (537.61 KB, text/plain)
2015-04-24 12:44 UTC, Adam D.
no flags Details
ESR_DEBUG - Pre-kill (65 bytes, text/plain)
2015-05-12 13:22 UTC, Adam D.
no flags Details
ESR_DEBUG - Post-kill (116.04 KB, text/plain)
2015-05-12 13:22 UTC, Adam D.
no flags Details
Evolution Backtrace (11.09 KB, text/plain)
2015-05-12 13:22 UTC, Adam D.
no flags Details
evolution-addressbook-factory backtrace (8.59 KB, text/plain)
2015-05-14 18:19 UTC, Adam D.
no flags Details

Description Adam D. 2015-01-30 15:08:09 UTC
Description of problem:
evolution-addressbook-factory is running on startup/login. When launching Evolution (tested with both a 2007 and now 2013 Exchange connection), autocomplete does not work on a new message until evolution-addressbook-factory has been killed. Once killed, starting a new message causes it to re-instantiate and it then autocompletes.


Version-Release number of selected component (if applicable):
evolution-3.12.10-1.fc21.x86_64
evolution-ews-3.12.10-1.fc21.x86_64
evolution-data-server-3.12.10-2.fc21.x86_64
evolution-help-3.12.10-1.fc21.noarch

How reproducible:
Every time.

Steps to Reproduce:
1. Start machine, log in.
2. Launch evolution, start a new message, no autocomplete.
3. Kill evolution-addressbook-factory, start new message, completes fine.

Actual results:
No autocomplete on initial launch.

Expected results:
Autocompletion of email addresses on every launch.

Additional info:
This is confirmed internally by two separate F21 clean installations.

Comment 1 Milan Crha 2015-02-04 15:33:20 UTC
Thanks for a bug report. Do you configure the GAL as an offline (Edit->Preferences->Mail Accounts-><ews account>->Edit->Receiving Options tab->Cache offline address book option)? Is your exchange server accessible directly, or any secured/company connection is needed, like a vpn connection which is not available right after start? These are the things which might influence the behaviour.

Comment 2 Adam D. 2015-02-04 18:56:36 UTC
Yes, GAL is ticked as cached for offline. This is internal, with the Exchange server directly accessible (it's directly accessible even externally, no VPN needed).

I can also give it a go without a cached GAL to see if it also affects this.

Comment 3 Milan Crha 2015-02-05 08:38:30 UTC
Thanks for the update. It truly depends on the GAL type (whether offline or online version will be used). The online version doesn't allow browsing and has only a minimal information included, as provided by the server. The offline GAL can be browsed and gives many detailed information about each contact (according to what is filled on the server for it).

I tried to reproduce it here by closing evolution-addressbook-factory and running it on my own, by a command:
   $ /usr/libexec/evolution-addressbook-factory -w
in a terminal. Then I run evolution and begun to compose a new mail message. Right after the composer window had been opened the console with the addressbook factory showed a debugging information from the GAL book, it was updating the local cache. It begun like this:

   Ewsgal: Fetching oal full details file 
   Ewsgal: Downloading full gal 
   OAL file downloaded /home/mcrha/.cache/evolution/addressbook/13957...a-5.lzx
   OAL file decompressed /home/mcrha/.cache/evolution/addressbook/13..ist-5.oab 
   Ewsgal: Removing old gal 
   Ewsgal: Replacing old gal with new gal contents in db 
   Total records is 9 
   GAL processing contacts, 11% complete (1 added, 0 changed, 0 unchanged
   ...
   GAL removing 0 contacts
   GAL update completed successfully in 126733 µs. Added: 9, Changed: 0,
      Unchanged 0, Removed: 0
   Ews gal: sync successful complete 

I guess the reason why your GAL searches fail can be that it is not fully updated yet. My GAL book is pretty short, it has only 9 contacts, but any real-life large GAL books can have thousands of contacts.

There had been done some improvements in this upstream too, but you have them already, as they were done for Evolution-EWS 3.12.7.

Running the addressbook factory with more detailed debugging on, like:
   $ EWS_DEBUG=2 /usr/libexec/evolution-addressbook-factory -w &>log.txt
may give a clue whether the EWS books are doing anything which blocks the GAL search or not.

Comment 4 Adam D. 2015-02-05 16:04:08 UTC
Since killing and starting the address book is what's effectively working around the issue, is there a place I can modify the startup command to capture this debug output on startup to try and figure out why its not working from the outset? 

This seems like a regression, maybe, as it used to work fine on the same servers back in F19/20. GAL size hasn't changed much since then, and we only have 350ish entries total. Once the kill/restart is complete, searches are pretty quick.

Comment 5 Adam D. 2015-02-05 17:39:24 UTC
As an update, it still fails to do an auto-complete from a fresh boot even with a non-cached GAL.

Comment 6 Milan Crha 2015-02-09 16:39:56 UTC
You can rename the /usr/libexec/evolution-addressbook-factory to some other file, then replace it with a script which will run the new file with the debugging on. I wrote detailed steps at bug #807911 comment #14, which was interestingly also about GAL. The interesting part from that comment is about the script content and how to create it. The rest is less relevant here.

Comment 7 Adam D. 2015-04-24 12:44:15 UTC
Big delay in getting back to you, sorry about that. As of:

evolution-3.12.11-1.fc21.x86_64
evolution-data-server-3.12.11-1.fc21.x86_64
evolution-help-3.12.11-1.fc21.noarch
evolution-ews-3.12.11-1.fc21.x86_64

...this issue is still present.

I am attaching the requested (sanitized) log. One thing to note is that on startup, and even launching Evolution, the log was empty until I killed addressbook-factory.orig, at which point it started populating only after doing a lookup.

Comment 8 Adam D. 2015-04-24 12:44:58 UTC
Created attachment 1018465 [details]
Log from addressbook-factory post-kill.

Comment 9 Milan Crha 2015-04-27 09:05:00 UTC
(In reply to Adam DiFrischia from comment #7)
> I am attaching the requested (sanitized) log. One thing to note is that on
> startup, and even launching Evolution, the log was empty until I killed
> addressbook-factory.orig, at which point it started populating only after
> doing a lookup.

Thanks for the update. The log shows that the factory can talk to the server without issues. You mentioned that the problem happens only after login. When you kill the "running" factory, then its next start the lookup works. Do I understand it correctly that it's exactly what happened here, you killed the "running" factory (which didn't produce any output) and then everything started to work properly?

Comment 10 Adam D. 2015-04-27 18:08:28 UTC
That is correct; I have not killed it yet today, and the log file under /tmp is 0 bytes. Evolution-addressbook-factory.orig has a memory footprint of 8.8MB.

After killing it, the log file has shot to 358K on a single lookup, with a memory footprint of 13.1MB.

Comment 11 Milan Crha 2015-04-28 07:02:34 UTC
(In reply to Adam DiFrischia from comment #10)
> That is correct; I have not killed it yet today, and the log file under /tmp
> is 0 bytes. Evolution-addressbook-factory.orig has a memory footprint of
> 8.8MB.

I see. The zero activity is truly suspicious. There is one debugging environment variable which can be used, it's
   ESR_DEBUG=1

It'll not tell much about the EWS GAL itself, but it might produce some side debug prints, like when the D-Bus name was acquired or lost.

Maybe a bit more interesting would be to get backtraces of the running evolution-source-registry and this evolution-addressbook-factory.orig executables, to see what they do. Please install debuginfo packages for evolution-data-server and evolution-ews and make sure their version will match the binary package version.

I would then reboot, run evolution, initiate the lookup (which will not work). Then use this command to get the backtrace:
   $ gdb --batch --ex "t a a bt" -pid=PID &>bt.txt
where PID is a process ID of the respective processes. You can use for example `ps ax | grep evolution` to get list of running evolution processes.

Please check the bt.txt for any private information, like passwords, email address, server addresses,... I usually search for "pass" at least (quotes for clarity only).

Comment 12 Adam D. 2015-05-12 13:21:33 UTC
Having added ESR_DEBUG to the addressbook-factory script, on startup the only output in the log is:

Bus name 'org.gnome.evolution.dataserver.AddressBook6' acquired.

It stays that way on lookup until I kill it and it restarts itself. I appended timestamps to the logs just to make sure I wasn't clobbering anything when it restarts. I will attach both of those logs, as well as the backtrace.

Comment 13 Adam D. 2015-05-12 13:22:04 UTC
Created attachment 1024589 [details]
ESR_DEBUG - Pre-kill

Comment 14 Adam D. 2015-05-12 13:22:29 UTC
Created attachment 1024590 [details]
ESR_DEBUG - Post-kill

Comment 15 Adam D. 2015-05-12 13:22:58 UTC
Created attachment 1024591 [details]
Evolution Backtrace

Comment 16 Milan Crha 2015-05-14 14:27:29 UTC
Thanks for the update. That means that the addressbook factory is stuck in some state. Please get a backtrace of the factory process (not the factory script), to see what it tries to do. Thanks in advance.

I'm thinking of https://bugzilla.gnome.org/show_bug.cgi?id=674885 , but without the backtrace really hard to tell whether it's it.

Comment 17 Adam D. 2015-05-14 18:19:44 UTC
Created attachment 1025553 [details]
evolution-addressbook-factory backtrace

That's my bad, I thought you meant a backtrace for evolution itself. Attached is the backtrace from e-a-f.

Comment 18 Milan Crha 2015-05-15 06:20:55 UTC
I see six threads waiting for a lock in e_book_backend_ews_open_sync(), but no other thread looking as holding it. I'm going to investigate it further.

Comment 19 Milan Crha 2015-05-15 06:45:57 UTC
I fixed this upstream with commit 3538031 in git master of evolution-ews (3.17.2+) [1] and commit e174d0e in ews gnome-3-16 (3.16.3+).

I'm currently building an update for Fedora 21 with the backported fix for it. It'll be available shortly.

[1] https://git.gnome.org/browse/evolution-ews/commit/?id=3538031

Comment 20 Fedora Update System 2015-05-15 07:03:14 UTC
evolution-ews-3.12.11-2.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/evolution-ews-3.12.11-2.fc21

Comment 21 Adam D. 2015-05-15 12:28:11 UTC
Looks to have fixed the issue. Positive karma left for the package. Thanks for looking into this!

Comment 22 Fedora Update System 2015-05-17 06:44:21 UTC
Package evolution-ews-3.12.11-2.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing evolution-ews-3.12.11-2.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-8404/evolution-ews-3.12.11-2.fc21
then log in and leave karma (feedback).

Comment 23 Fedora Update System 2015-05-27 16:20:55 UTC
evolution-ews-3.12.11-2.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.