Bug 166545 - Evolution Import of Archive ... Emails Missing
Evolution Import of Archive ... Emails Missing
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: evolution (Show other bugs)
6
i686 Linux
low Severity medium
: ---
: ---
Assigned To: Matthew Barnes
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-08-23 03:56 EDT by TonyCrouch
Modified: 2007-11-30 17:11 EST (History)
0 users

See Also:
Fixed In Version: Fedora 7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-10-05 00:45:32 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Emails in Saved Folder (200.13 KB, image/png)
2005-08-23 17:32 EDT, TonyCrouch
no flags Details
Highlighting all the Saved Emails (167.35 KB, image/png)
2005-08-23 17:34 EDT, TonyCrouch
no flags Details
Giving the external saved emails file a name (520.21 KB, image/png)
2005-08-23 17:36 EDT, TonyCrouch
no flags Details
Saving emails process being completed (171.44 KB, image/png)
2005-08-23 17:38 EDT, TonyCrouch
no flags Details
Importation Process (423.41 KB, image/png)
2005-08-23 17:39 EDT, TonyCrouch
no flags Details
The importation folder figures (164.01 KB, image/png)
2005-08-23 17:41 EDT, TonyCrouch
no flags Details
File that is created through saving the emails from the saved folder (12.72 MB, application/octet-stream)
2005-09-05 00:03 EDT, TonyCrouch
no flags Details
saved_emails --v-- reimport_test (22.06 KB, application/octet-stream)
2005-09-05 00:05 EDT, TonyCrouch
no flags Details

  None (edit)
Description TonyCrouch 2005-08-23 03:56:47 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050524 Fedora/1.0.4-4 Firefox/1.0.4

Description of problem:
When importing emails that were saved from a folder within Evolution the number of emails that are imported is different from the original number of emails that were saved.

For example, I saved a folder of emails that had 1200 emails in it. When I re-import this into Evolution it shows only 476 of the 1200 emails.

Version-Release number of selected component (if applicable):
evolution-2.2.2-5

How reproducible:
Always

Steps to Reproduce:
1. Save a folder of emails within Evolution.
2. Re-import this saved file into Evolution ("File" --> "Import").
3. Notice the number of files imported is different from the number saved.
  

Actual Results:  The number of emails imported is different from the original number that was saved.

Expected Results:  The number of emails re-imported should be identical to the number that was saved.

Additional info:
Comment 1 TonyCrouch 2005-08-23 08:04:46 EDT
I have run "diff" on these two files and a dramatic difference is noted. In fact
374144 lines are different.

Hope this helps.

Cheers, 
Tony Crouch
Comment 2 Dave Malcolm 2005-08-23 15:13:28 EDT
Thanks for this report.

How exactly did you save the folder (in step 1 of the recipe you supplied)? 
What kind of account was the folder from?
Are you importing the mail from an old home directory into a new home directory?
 If so, what has already happened to that new home directory's ~/.evolution
folder?  (e.g. have you set up accounts yet, if so, of what kind, etc).   Thanks.
Comment 3 TonyCrouch 2005-08-23 15:41:01 EDT
Hi Dave, 
Thanks for your reply in regards to this bug.

I saved the emails as follows:
1) Highlighted the folder in which I wanted to save its contained emails.
2) This brought up the contained emails in the upper right-hand side preview 
pane.
3) Right-clicked an arbitary email in the upper-right hand preview pane.
4) Selected "Save As".
5) Inputted the desired output name for the ascii file that will contain the 
saved emails.
6) Clicked "Ok"
7) Status bar at bottom of Evolution indicates the process is saving the 
correct number of emails. A 'thermometer' bar appears indicating the saving 
process graphically.

The account that the emails are from is a standard pop3 account, nothing 
special.

No, I am not importing the emails across home directories. I simply save the 
emails in the aforementioned process, close Evolution (have even rebooted for 
completeness), and then import the saved emails back into the same linux user 
account and into the same pop3 account the emails were saved from.

If there is any further information you require in order to assist you with 
this bug, please do not hesitate to ask. Am only too happy to help.

Hope this helps.

Thanks for your time.

Cheers, 
Tony Crouch
Comment 4 Dave Malcolm 2005-08-23 15:56:36 EDT
Thanks for the clarification.  Is this mail that's stored on the server, or is
it mail that's downloaded locally?

The recipe in comment #3 appears to be missing these stages:
2.5) Clear any saved searches in the folder
2.6) "Select All" emails in the folder
If this isn't done, you won't get all of the emails in the folder.

However, from 7) it appears that you had all of the emails - this is the case,
right?
What happened to the old contents of that folder?  Are the emails still there,
or did they get trashed somehow? (i.e. why are you restoring?)  Thanks.
Comment 5 TonyCrouch 2005-08-23 16:13:04 EDT
Hi Dave, 
The emails are downloaded locally onto my machine.

Yes, well spotted. I did mis-input your mentioned steps in my saving algorithm. 
Thank you for the correction. In comment to the following:

2.5) Yes, I have ensured that there are no saved searches. This field is left 
blank when saving the emails.

2.6) This was an honest typographical error on my behalf. I meant to say the 
following:

a) Press "Ctrl+A" to highlight all the email messages within the relevant 
folder.
b) Right-click on an arbitary email and choose "Save As".

Yes from what I can see in the saving process that Evolution performs it at 
least "claims" to be saving the correct number of emails. 

The old emails are still present in the original folder. As the process of 
saving them does not delete them.

My reasoning for importing of emails. Is merely for (at the present stage at 
least) ensuring that the process of archiving and importing emails is 
successful under Evolution, in case (touch wood) I ever have to reformat with 
limited or abrupt notice.

I hope this clears up some of your queries. Do not hesitate to get in touch 
with future queries or concens.

Thanks again.

Tony Crouch
Comment 6 Dave Malcolm 2005-08-23 16:25:37 EDT
(BTW, sorry if my last comment came across as a bit condescending; I was being
(perhaps overly) pedantic so as to rule out one possible source of the problem)

I'm afraid I want further clarification... in step 3 of your original report,
where are you getting the number imported from?  (I had a look at the email
import code and couldn't see it outputting any such results; though it was only
a brief look through).  Thanks.
Comment 7 TonyCrouch 2005-08-23 16:52:27 EDT
Hi Dave, 
No problem about the pedanticness. It is far better to be pedantic about such 
matters and ensure that all probable solutions are covered and accounted for 
than to waste ages on a fix that was, had more careful attention been paid 
originally, not necessary.

I am getting the number of emails imported from the same place I am getting the 
number of emails exported. From the upper-right-hand preview pane of Evolution, 
I am not in Linux at the moment so I cannot be 100% sure, but situated near 
the "message search bar" is the "total number of emails" in the folder and 
the "total number of emails unread". It is the "total number of emails" I am 
gaining both these numbers from.

If it would help I can obtain a screenshot and attach it to this report 
illustrating where the total numbers are found and their differing values.

If you require anything else Dave, just ask :-)

Thanks again.

Cheers, 
Tony
Comment 8 Dave Malcolm 2005-08-23 16:58:45 EDT
That area reads something like "N Total, M Unread" for N and M.

A screenshot of "before" and "after" might help - thanks.
Comment 9 TonyCrouch 2005-08-23 17:04:54 EDT
Hi Dave, 
Yes, the totals do read as you have described.

Will attach screenshots in a few minutes ... have to reboot.

Thanks again.

Cheers, 
Tony Crouch
Comment 10 TonyCrouch 2005-08-23 17:30:00 EDT
Hi Dave, 
The screenshots I have included below will show graphically the steps I go
through saving and importing the emails into Evolution. Hopefully this will
clear up the process along with showing before and after figures.

There are six .png files in total.

Hope they help.

Cheers, 
Tony Crouch
Comment 11 TonyCrouch 2005-08-23 17:32:36 EDT
Created attachment 118025 [details]
Emails in Saved Folder

This screenshot shows the Saved folder ("UNE ITD Requests") from which I am
saving the emails. Note in the upper-left hand corner it says, "1189 total, 318
unread".
Comment 12 TonyCrouch 2005-08-23 17:34:43 EDT
Created attachment 118026 [details]
Highlighting all the Saved Emails

Highlighting all the saved emails for saving into an external file to be
imported at a later date.
Comment 13 TonyCrouch 2005-08-23 17:36:51 EDT
Created attachment 118027 [details]
Giving the external saved emails file a name

The text box in which you input the desired external saved emails file name.
Comment 14 TonyCrouch 2005-08-23 17:38:08 EDT
Created attachment 118028 [details]
Saving emails process being completed

This is the process of the emails being saved to the external file. Note that
the number of emails to be saved in the status bar at the bottom of the screen
is identical to the numeber of emails in the folder itself.
Comment 15 TonyCrouch 2005-08-23 17:39:00 EDT
Created attachment 118029 [details]
Importation Process

This is a screenshot of the importation process being completed
Comment 16 TonyCrouch 2005-08-23 17:41:18 EDT
Created attachment 118030 [details]
The importation folder figures

This screenshot shows that the number of total emails imported into Evolution
through the saved file is different to the number in the original folder we
saved from.

This folder has only 503 emails in comparison to the 1189 emails we began with.


Hope these screenshots help.

Cheers, 
Tony Crouch
Comment 17 TonyCrouch 2005-08-25 12:58:40 EDT
Hi Dave, 
Unfortunately, due to a hardware failure in recent days I have had to ship my
notebook away for repair. My machine has taken to spontanteously rebooting due
to an overheating issue. I am assuming that this issue would be unrelated to the
Evolution problem being experienced here.

I am unsure as to how long it will be away for. I will still be online, albeit
less frequently, and be able to provide information but specific information
(i.e. screenshots) will not be able to be provided until my machine comes back
from the repairers.

Thanks again for your concern and effort in regards to fixing this bug.

Cheers, 
Tony Crouch
Comment 18 Dave Malcolm 2005-08-26 16:11:22 EDT
Thanks for all the screenshots.  I see the problem now, though I don't yet know
what's causing it.

Evolution stores each local mail folder in a file beneath ~/.evolution/mail/local

I'd expect there to be a Saved.sbd directory, containing a reimport_test file,
and a similar file in the original location.  You'll also see various extra
files for each folder, with .index, .cmeta (etc) suffixes; this contains
metadata and saved textual content data (to speed up searches).

It might be useful to run the "diff" tool to compare the contents of the saved
file with (a) Evolution's file for the original mail folder and (b) with its
file for the fresh import.

I'd expect the contents to be the same, apart from X-Evolution: headers for each
mail in Evolution's "internal" files.  This should let you check (a) that all
the emails are getting saved correctly, without subtle corruptions and (b) which
mails aren't getting imported.  Does this show up any patterns?  Which emails
don't get imported?  Is there anything unexpected happening?

Hope this helps
Comment 19 TonyCrouch 2005-08-26 16:16:53 EDT
Hi Dave, 
Thanks for the suggestions. I will try them when I get my notebook back from 
the repairers. Which should not be anymore than a few weeks.

Will keep you informed as to what I find when I run the above tests.

Cheers, 
Tony
Comment 20 Dave Malcolm 2005-08-26 16:21:37 EDT
Good luck with the notebook.

I'm putting this back to NEEDINFO_REPORTER since I don't plan to investigate
this further at this stage until I've heard back from you with info from the
tests (or if other people run into the problem); I can't reproduce it at this
end.  Cheers.
Comment 21 TonyCrouch 2005-09-04 23:58:57 EDT
Hi Dave, 
I got my notebook back this morning and have performed the diff tests you
requested. Please find them attached in the following posts.

To help you to understand which saves the file names are:

saved_emails: This file is the saved emails initially saved from my saved emails
folder.

UNE_ITD_req: The emails that are stored in my saved_emails folder.
Theoretically, this file and saved_emails file should be identical. As this is
the file saved_emails is saving from.

reimport_test: This is the file that is created when the saved_emails file is
reimported into Evolution.

Also, I recently performed a yum upgrade and am unsure as to whether or not
Evolution was also updated in this process. On the chance an update fixed this
problem I repeated the steps again and the same discrepancy in emails numbers
has occured. However, if an update has occured I will change the corresponding
Evolution version number.

Hope this helps.

Thanks again.

Cheers, 
Tony
Comment 22 TonyCrouch 2005-09-05 00:03:27 EDT
Created attachment 118442 [details]
File that is created through saving the emails from the saved folder
Comment 23 TonyCrouch 2005-09-05 00:05:18 EDT
Created attachment 118443 [details]
saved_emails --v-- reimport_test

This is the file created after running "diff" on the saved_emails file and the
reimport_test file. The file created after the saved_emails file is reimported
into Evolution.
Comment 24 TonyCrouch 2005-09-05 00:20:13 EDT
I tried to attach the saved_emails --v-- UNE_ITD_req file but I was unable
because the diff file produced is greater than 20Mb (23Mb in fact). Which leads
me to believe that since there is so little difference between the saved_emails
file and the file that is obtained during re-importation then there is a problem
when the emails are saved, even though the status clearly says the correct
number of emails are being saved.

I will upload the text file produced from running diff on saved_emails --v--
UNE_ITD_req for you to use if required. I will provide the link shortly.

Thanks again.

Cheers, 
Tony
Comment 25 TonyCrouch 2005-09-05 01:06:30 EDT
Hi Dave, 
I have managed to upload the diff file from the saved_emails and the 
UNE_ITD_req folders to my uni's FTP server.

This extremely large file (23Mb) can be found at the following address. Please 
contact me should you have any troubles viewing it.

http://www-personal.une.edu.au/~acrouch3/saved_emails_v_UNE_ITD_req

Hope this helps.

Cheers, 
Tony
Comment 26 TonyCrouch 2005-09-05 05:05:32 EDT
Hi Dave, 
Just as clarification I mentioned earlier that I recently upgraded my version of
Evolution. This bug was originally filed when using:

evolution-2.2.2-5

I have since upgraded to:

evolution-2.2.3-2.fc4

I have performed the saving / re-importation process again under this new
version and the same problem holds true.

Just thought you should know.

Cheers, 
Tony
Comment 27 TonyCrouch 2005-09-13 09:57:00 EDT
Hi Dave,
Just wondering how you are going with this issue as i have not heard from you
since posting the requested 'diff' files.

Cheers, 
Tony
Comment 29 Matthew Barnes 2006-07-31 11:19:08 EDT
Hi Tony, I have taken over Evolution maintenance since your last posting.

If you are using Fedora Core 5 or Rawhide, can you tell me whether you're still
seeing this problem?  I was unable to reproduce this using Rawhide, which means
it may have been fixed by the upstream developers since evolution-2.2.3.

evolution-2.6.2-1.fc5.5  (FC5)
evolution-2.7.4-3  (Rawhide)
Comment 30 TonyCrouch 2006-07-31 23:28:27 EDT
Hi Matthew, 
Yes, I have since upgraded to FC5. But the problem is still present plus with
the addition of another problem.

When I first noticed this problem I filed an appropriate bugzilla report.

The link to the bug is: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=189126

I have not retested the problem since I have upgraded my evolution package.

I will attempt the process again over the coming days and get back to you
through the relevant FC5 page.

Thanks for your time.

All the best.

Cheers, 
Tony Crouch
Comment 31 Matthew Barnes 2007-01-02 12:53:12 EST
Is this problem still present in Fedora Core 6 or later?
Comment 32 TonyCrouch 2007-01-20 17:50:24 EST
Hi Matt, 
Yeah, this is still causing problems in FC6. I just tried to update my evolution
software to ensure I have the latest release but am having trouble connecting.

My current version is:
evolution-2.8.0-7.fc6

In the tests I performed this morning the importation / re-importation process
seems to be handled much better by evolution as it states only 1 (one) email
seems to be 'lost' during the entire process.

However, running 'diff' on these two files (the 'saved' and 'imported' files)
yields a very different story, clearly showing there are multiple emails missing.

I ran the entire process again (on the same data set) and achieved different
emails missing (i.e. different emails are being lost each time).

I will attach the files to this bug over the next few minutes.

Thanks for your time.

All the best.

Cheers, 
Tony Crouch

Comment 33 TonyCrouch 2007-02-10 20:34:32 EST
Hi Matt, 
My apologies for not keeping on top of this bug. I tried uploading the 'diff'
files I mentioned in the previous comment but they are really, really large.
Well over 30Mb.

I am about to update my version of Evolution and try the entire procedure again.
I will keep you informed as to my progress.

Thanks for your time.

All the best.

Cheers, 
Tony Crouch
Comment 34 TonyCrouch 2007-05-24 20:32:15 EDT
Matt, 
Am just wanting to let you know, I did eventually got the chance to perform 
the procedure again. I most probably don't have the latest version of 
Evolution at the moment because my 'yum' command is broken.

I will see whether this bug is still present once I do a fresh-install of F7 
later next week.

I will keep you updated as to my progress.

Thanks for your assistance with this bug.

All the best.

Cheers, 
Tony Crouch
Comment 35 Matthew Barnes 2007-10-04 23:44:07 EDT
Hi Tony, my apologies for letting this bug languish for so long.

Are you still able to reproduce this problem on Fedora 7 or later?
Comment 36 TonyCrouch 2007-10-05 00:42:49 EDT
Hi Matt, 
Am glad to say I just tested out this bug problem again now I have done a fresh
install of F7. I am currently in the process of doing a yum update so I am bound
not to be dusing the most recent version.

Nevertheless, this problem seems to have been fixed.

Thanks for all your assistance.

Cheers, 
TC
Comment 37 Matthew Barnes 2007-10-05 00:45:32 EDT
Awesome, thanks for checking.  Closing this as CURRENTRELEASE (Fedora 7) but
feel free to re-open this bug if you encounter the problem again.

Note You need to log in before you can comment on or make changes to this bug.