Bug 710085 - Evolution 3.0 Migration does not migrate mail folders or contacts
Summary: Evolution 3.0 Migration does not migrate mail folders or contacts
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: evolution
Version: 15
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Matthew Barnes
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-06-02 12:29 UTC by Herbert Rutledge
Modified: 2011-06-10 05:33 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-06-10 05:33:47 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
evolution backtrace (6.07 KB, text/plain)
2011-06-03 17:01 UTC, Herbert Rutledge
no flags Details
Evolution 06.08.11 backtrace (5.70 KB, application/octet-stream)
2011-06-08 09:19 UTC, Herbert Rutledge
no flags Details

Description Herbert Rutledge 2011-06-02 12:29:56 UTC
Description of problem:

Evolution migration feature doesn't migrate mail/contacts

Version-Release number of selected component (if applicable):

3.0.1-1

How reproducible:

Every time.

Steps to Reproduce:
1.  Clean local evolution folders, copy settings from backup.
2.  Start evolution, select Migrate option.
3.
  
Actual results:

Mail account settings are preserved, but older mail, mail folders and
contacts do not appear.

Expected results:

Older Evolution data are preserved during migration.


Additional info:

Just as a note, I've been using Evolution as my main e-mail client ever
since version 0.1 (Developer Preview), so that's a lot of mail and 
contacts over the years.  I've seen all kinds of migration failures in
that time, but this one has me stumped.

Running Evolution from the console and selecting Import shows all the 
existing mail folders being successfully imported, but they don't 
show up on the sidebar.  Aside from an initial message that reads 
Gtk-Message: Failed to load module "pk-gtk-module" everything works
without any additional error reporting.  The Package Kit gtk module 
package is installed.


If I send an e-mail to myself, I get an error message to the effect 
that the Sent folder does not exist and the e-mail is appended to 
the displayed Sent folder.  So, it seems that it's all there, just 
invisible.

In order to get the Migration to succeed, I need to copy everything
from the backup except .config/evolution; otherwise, it will hang
after 45 seconds of processing time and the process has to be
manually killed.  I also kill addressbook-factory before stating
again.  

I haven't found any similar problem filed in bugzilla, so I'm
reporting it here.  I'll check this entry periodically since I
don't have a working e-mail client at the moment.

Comment 1 Milan Crha 2011-06-03 06:53:17 UTC
Thanks for a bug report. I guess there is something special about your installation, because this is the first bug report of this type I noticed since 3.0.0 release, and I believe there are many users to whom the migration worked.

I suggest to update to 3.0.2 with evolution-3.0.2-2 from updates-testing, to get the latest code. Just to use most recent stable release.

> 1.  Clean local evolution folders, copy settings from backup.
> 2.  Start evolution, select Migrate option.

What do you mean with "copy settings from backup", please? Is it by the startup wizard which offers you to restore from an older backup, or is it done yourself?

> Running Evolution from the console and selecting Import shows all the 
> existing mail folders being successfully imported, but they don't 
> show up on the sidebar.

After migration is done there is created a new account which keeps your old local data as is. This account is visible in Preferences and can be enabled. Where did you let it import? Was it directly under On This Computer or under On This Computer/Inbox? Though the import might not be needed in this case.

Also, what is your usual folder structure under On This Computer (apart of those system folders), please? I'm wondering whether there isn't some issue with folders added on the same level as the On This Computer/Inbox, instead of placing them under this Inbox. Like in the example:
   On This Computer
     |--- Inbox
     |  +--- folder 1
     |
     +--- folder 2

whether the folder 1 would be migrated/imported successfully, but folder 2 not.

> If I send an e-mail to myself, I get an error message to the effect 
> that the Sent folder does not exist and the e-mail is appended to 
> the displayed Sent folder.  So, it seems that it's all there, just 
> invisible.

That's a bug in the migration code, it didn't change your Sent (and Drafts) folder location after migration. The workaround to fix it is to edit your account and reselect the folders on the Default tab.

Comment 2 Herbert Rutledge 2011-06-03 13:38:53 UTC
Thanks for your timely response.  I couldn't find anyone else reporting 
anything like this either.  To answer your questions:

1.  Copy settings from backup:
    
From a backup of the home directory, copy the following:

  ~/.config/evolution
  ~/.evolution
  ~/.local/share/evolution

This should restore the system to the state it was in after upgrading from F14 to F15, shouldn't it?  Is there something else I'm overlooking?

With the 3.0.2 release, if ~/.config/evolution is copied, selecting Migration
will hang the process after 45 seconds, the same as for 3.0.1.

2.  I used the Import Assistant under both On This Compute and Inbox.  Nothing
    was imported, although the console output reported success for each folder.

3.  With 3.0.2, I see the new account; it was not there with 3.0.1.  
    Using the Import Assistant on the new account again reports success 
    on the console output, but the folders do not show up after the 
    process completes.

4.  Folder structure places folders as subfolders of Inbox so that they may
    be collapsed when not needed so as not to clutter up the side panel.

5.  The contacts are still not imported.  Contacts under Personal is still
    empty.

Let me know what additional information is needed.  I've run a backtrace, but nothing indicating failure shows up.  However, on the console output, I note the following:

If ~/.config/evolution is copied from the ~/ backup, the following message appears multiple times (apparently, once for each folder):

(evolution:3947): camel-WARNING **: Could not find key entry for word '00000000f0000000': No such file or directory

If ~/.config/evolution is not copied, the same message appears, but with 
Success in place of No such file or directory.

Comment 3 Milan Crha 2011-06-03 14:08:59 UTC
(In reply to comment #2)
> 1.  Copy settings from backup:
> ...
> This should restore the system to the state it was in after upgrading from F14
> to F15, shouldn't it?  Is there something else I'm overlooking?

The correct and preferable way is to use File->Backup Settings and then File->Restore Settings (or the startup wizard offer to restore from a backup), but I agree that there are cases where this is not doable. For that is an entry in Evolution FAQ page:
http://live.gnome.org/Evolution/FAQ#How_can_I_transfer_all_my_Evolution_data_from_an_old_home_directory_to_a_new_home_directory.3F

> With the 3.0.2 release, if ~/.config/evolution is copied, selecting Migration
> will hang the process after 45 seconds, the same as for 3.0.1.

Please install debug info packages for evolution-data-server and evolution and get a backtrace of frozen evolution, which you can do like this:
   $ gdb --batch --ex "t a a bt" -pid=PID &>bt.txt
where PID is a process ID of a running process, in this case evolution. Note that it can still do some stuff, it only doesn't update its UI. The 'top' command may also show whether it's not busy.

The rest can be due to incomplete restore, like the contacts, which are stored in GConf, and if the GConf key doesn't match the disk data, then the address book is empty.

> If ~/.config/evolution is copied from the ~/ backup, the following message
> appears multiple times (apparently, once for each folder):
> 
> (evolution:3947): camel-WARNING **: Could not find key entry for word
> '00000000f0000000': No such file or directory
> 
> If ~/.config/evolution is not copied, the same message appears, but with 
> Success in place of No such file or directory.

It's a bug, I agree, but it doens't influence anything from the above, as far as I know.

Comment 4 Herbert Rutledge 2011-06-03 17:01:44 UTC
Created attachment 502863 [details]
evolution backtrace

Comment 5 Herbert Rutledge 2011-06-03 17:03:34 UTC
Backtrace run.

I tried the solution in the Evolution FAQ with no success.

If I run strace on Evolution, I get a similar result to what is in the
backtrace:  nonosleep spews for quite a while until the process hangs.

Thanks again for all your help.

Comment 6 Milan Crha 2011-06-06 07:25:23 UTC
Thanks for the update. Could you install debug info packages for sqlite and evolution-data-server, please? I would like to ask SQLite developer/maintainer for an opinion, but for that is required the debug information available for it.

To workaround your issue, I guess it may help to remove
   ~/.evolution/mail/local/folder.db
file, but please keep it backed up for later use, if any. If there is 
   ~/.local/share/evolution/local_mbox
folder already, then move it away too.

Comment 7 Herbert Rutledge 2011-06-07 18:43:42 UTC
OK, I had to work this weekend, and didn't have much time to get back to this, but I'm off work today.

What I did, before seeing your last comment, was to start with a clean install of Evolution, and then copy the contents of the backup of ~/.local/share/evolution to ~/.local/share.  I then started evolution.  Nothing was showing.  However, after a reboot to get gconf reset, all of the contacts were there, and the contacts work correctly.  I did File--> Back up Evolution data so that I'd at least have a backup of Evolution in this state.

I then ran Evolution from the console and selected Import.  All of the existing mail folders were shown to be imported under their folder names on the console output.  There is now a new folder called #evolution.  This folder has a subfolder called sbd.  However, you get No such file or directory when trying to open #evolution, although sdb is visible, but empty.  

I tried your suggested workaround, but it did not help.  Removing /mail/folders.db plus local_mbox and then running Evolution, it prompts for Migrate, but still hangs after 45 seconds showing the same error messages.

When I originally ran the backtrace, the only -debuginfo package I could find was for the Rawhide version of evolution-data-server.  I finally found the F15 version and have it installed.  However, I can't find a -debuginfo package for sqlite.  I could download the source and build my own.  Although the checksums wouldn't match and would generate a warning, it should still work, if imperfectly.

Would this be helpful?

You know, this is such a corner case that if I could get the contents of #evolution displayed and it turned out to be the old mbox folders, I would consider this a success.  There just can't be that many people who have been using Evolution continuously ever since version 0.1.

Comment 8 Milan Crha 2011-06-08 06:58:59 UTC
Thanks for the update. With respect of sqlite debug info package, the below command works for me just fine:
   $ yum install --enablerepo=fedora-debuginfo sqlite-debuginfo

The #evolution is a system directory and may not be shown. I suppose the migration process makes sure the folder is moved to the right place.

Let's start with the migration hang and then continue with the other issue, if it'll be still present.

Comment 9 Herbert Rutledge 2011-06-08 09:19:07 UTC
Created attachment 503652 [details]
Evolution 06.08.11 backtrace

Comment 10 Herbert Rutledge 2011-06-08 09:24:31 UTC
Odd.  The debuginfo package wasn't there yesterday, but it's there now and installed.  Backtrace enclosed.  Thanks again for all your help.

Comment 11 Milan Crha 2011-06-09 04:54:29 UTC
Thanks for the update. The backtrace shows it's stuck in sqlite function, which I guess is caused by some change in the folders.db file, or the old file is corrupted. I do not expect this being a bug in sqlite.

I do not have any other idea apart of the one mentioned in the comment #6.

Comment 12 Herbert Rutledge 2011-06-09 11:21:35 UTC
I figured as much from looking at the backtrace.

I just cleaned out everything evolution related in the current home directory, and copied everything from the backup of the home directory, but removed only the folders.db file under ~/.local/share/evolution/mail and started evolution from the console.

This time, it worked.  No unusual error messages were reported on the console, and the original mail folders appeared in the sidebar panel under the default account name.  I had never been unable to get the default account name to display there before.

Contacts work, and all the old mail is accounted for.

You can go ahead and close this bug at this point.

Thanks again for all your help.  It has been invaluable.

Comment 13 Milan Crha 2011-06-10 05:33:47 UTC
(In reply to comment #12)
> This time, it worked.  No unusual error messages were reported on the console,
> and the original mail folders appeared in the sidebar panel under the default
> account name.  I had never been unable to get the default account name to
> display there before.
> 
> Contacts work, and all the old mail is accounted for.

Good to hear that. I suspect the folders.db file being corrupted in a way which confused sqlite, but I've no exact clues.

> You can go ahead and close this bug at this point.

OK, thanks, I'm closing this.


Note You need to log in before you can comment on or make changes to this bug.