I have run "diff" on these two files and a dramatic difference is noted. In fact
374144 lines are different.

Hope this helps.

Cheers, 
Tony Crouch

Thanks for this report.

How exactly did you save the folder (in step 1 of the recipe you supplied)? 
What kind of account was the folder from?
Are you importing the mail from an old home directory into a new home directory?
 If so, what has already happened to that new home directory's ~/.evolution
folder?  (e.g. have you set up accounts yet, if so, of what kind, etc).   Thanks.

Hi Dave, 
Thanks for your reply in regards to this bug.

I saved the emails as follows:
1) Highlighted the folder in which I wanted to save its contained emails.
2) This brought up the contained emails in the upper right-hand side preview 
pane.
3) Right-clicked an arbitary email in the upper-right hand preview pane.
4) Selected "Save As".
5) Inputted the desired output name for the ascii file that will contain the 
saved emails.
6) Clicked "Ok"
7) Status bar at bottom of Evolution indicates the process is saving the 
correct number of emails. A 'thermometer' bar appears indicating the saving 
process graphically.

The account that the emails are from is a standard pop3 account, nothing 
special.

No, I am not importing the emails across home directories. I simply save the 
emails in the aforementioned process, close Evolution (have even rebooted for 
completeness), and then import the saved emails back into the same linux user 
account and into the same pop3 account the emails were saved from.

If there is any further information you require in order to assist you with 
this bug, please do not hesitate to ask. Am only too happy to help.

Hope this helps.

Thanks for your time.

Cheers, 
Tony Crouch

Thanks for the clarification.  Is this mail that's stored on the server, or is
it mail that's downloaded locally?

The recipe in comment #3 appears to be missing these stages:
2.5) Clear any saved searches in the folder
2.6) "Select All" emails in the folder
If this isn't done, you won't get all of the emails in the folder.

However, from 7) it appears that you had all of the emails - this is the case,
right?
What happened to the old contents of that folder?  Are the emails still there,
or did they get trashed somehow? (i.e. why are you restoring?)  Thanks.

Hi Dave, 
The emails are downloaded locally onto my machine.

Yes, well spotted. I did mis-input your mentioned steps in my saving algorithm. 
Thank you for the correction. In comment to the following:

2.5) Yes, I have ensured that there are no saved searches. This field is left 
blank when saving the emails.

2.6) This was an honest typographical error on my behalf. I meant to say the 
following:

a) Press "Ctrl+A" to highlight all the email messages within the relevant 
folder.
b) Right-click on an arbitary email and choose "Save As".

Yes from what I can see in the saving process that Evolution performs it at 
least "claims" to be saving the correct number of emails. 

The old emails are still present in the original folder. As the process of 
saving them does not delete them.

My reasoning for importing of emails. Is merely for (at the present stage at 
least) ensuring that the process of archiving and importing emails is 
successful under Evolution, in case (touch wood) I ever have to reformat with 
limited or abrupt notice.

I hope this clears up some of your queries. Do not hesitate to get in touch 
with future queries or concens.

Thanks again.

Tony Crouch

(BTW, sorry if my last comment came across as a bit condescending; I was being
(perhaps overly) pedantic so as to rule out one possible source of the problem)

I'm afraid I want further clarification... in step 3 of your original report,
where are you getting the number imported from?  (I had a look at the email
import code and couldn't see it outputting any such results; though it was only
a brief look through).  Thanks.

Hi Dave, 
No problem about the pedanticness. It is far better to be pedantic about such 
matters and ensure that all probable solutions are covered and accounted for 
than to waste ages on a fix that was, had more careful attention been paid 
originally, not necessary.

I am getting the number of emails imported from the same place I am getting the 
number of emails exported. From the upper-right-hand preview pane of Evolution, 
I am not in Linux at the moment so I cannot be 100% sure, but situated near 
the "message search bar" is the "total number of emails" in the folder and 
the "total number of emails unread". It is the "total number of emails" I am 
gaining both these numbers from.

If it would help I can obtain a screenshot and attach it to this report 
illustrating where the total numbers are found and their differing values.

If you require anything else Dave, just ask :-)

Thanks again.

Cheers, 
Tony

That area reads something like "N Total, M Unread" for N and M.

A screenshot of "before" and "after" might help - thanks.

Hi Dave, 
Yes, the totals do read as you have described.

Will attach screenshots in a few minutes ... have to reboot.

Thanks again.

Cheers, 
Tony Crouch

Hi Dave, 
The screenshots I have included below will show graphically the steps I go
through saving and importing the emails into Evolution. Hopefully this will
clear up the process along with showing before and after figures.

There are six .png files in total.

Hope they help.

Cheers, 
Tony Crouch

Created attachment 118025 [details]
Emails in Saved Folder

This screenshot shows the Saved folder ("UNE ITD Requests") from which I am
saving the emails. Note in the upper-left hand corner it says, "1189 total, 318
unread".

Created attachment 118026 [details]
Highlighting all the Saved Emails

Highlighting all the saved emails for saving into an external file to be
imported at a later date.

Created attachment 118027 [details]
Giving the external saved emails file a name

The text box in which you input the desired external saved emails file name.

Created attachment 118028 [details]
Saving emails process being completed

This is the process of the emails being saved to the external file. Note that
the number of emails to be saved in the status bar at the bottom of the screen
is identical to the numeber of emails in the folder itself.

Created attachment 118029 [details]
Importation Process

This is a screenshot of the importation process being completed

Created attachment 118030 [details]
The importation folder figures

This screenshot shows that the number of total emails imported into Evolution
through the saved file is different to the number in the original folder we
saved from.

This folder has only 503 emails in comparison to the 1189 emails we began with.


Hope these screenshots help.

Cheers, 
Tony Crouch

Hi Dave, 
Unfortunately, due to a hardware failure in recent days I have had to ship my
notebook away for repair. My machine has taken to spontanteously rebooting due
to an overheating issue. I am assuming that this issue would be unrelated to the
Evolution problem being experienced here.

I am unsure as to how long it will be away for. I will still be online, albeit
less frequently, and be able to provide information but specific information
(i.e. screenshots) will not be able to be provided until my machine comes back
from the repairers.

Thanks again for your concern and effort in regards to fixing this bug.

Cheers, 
Tony Crouch

Thanks for all the screenshots.  I see the problem now, though I don't yet know
what's causing it.

Evolution stores each local mail folder in a file beneath ~/.evolution/mail/local

I'd expect there to be a Saved.sbd directory, containing a reimport_test file,
and a similar file in the original location.  You'll also see various extra
files for each folder, with .index, .cmeta (etc) suffixes; this contains
metadata and saved textual content data (to speed up searches).

It might be useful to run the "diff" tool to compare the contents of the saved
file with (a) Evolution's file for the original mail folder and (b) with its
file for the fresh import.

I'd expect the contents to be the same, apart from X-Evolution: headers for each
mail in Evolution's "internal" files.  This should let you check (a) that all
the emails are getting saved correctly, without subtle corruptions and (b) which
mails aren't getting imported.  Does this show up any patterns?  Which emails
don't get imported?  Is there anything unexpected happening?

Hope this helps

Hi Dave, 
Thanks for the suggestions. I will try them when I get my notebook back from 
the repairers. Which should not be anymore than a few weeks.

Will keep you informed as to what I find when I run the above tests.

Cheers, 
Tony

Good luck with the notebook.

I'm putting this back to NEEDINFO_REPORTER since I don't plan to investigate
this further at this stage until I've heard back from you with info from the
tests (or if other people run into the problem); I can't reproduce it at this
end.  Cheers.

Hi Dave, 
I got my notebook back this morning and have performed the diff tests you
requested. Please find them attached in the following posts.

To help you to understand which saves the file names are:

saved_emails: This file is the saved emails initially saved from my saved emails
folder.

UNE_ITD_req: The emails that are stored in my saved_emails folder.
Theoretically, this file and saved_emails file should be identical. As this is
the file saved_emails is saving from.

reimport_test: This is the file that is created when the saved_emails file is
reimported into Evolution.

Also, I recently performed a yum upgrade and am unsure as to whether or not
Evolution was also updated in this process. On the chance an update fixed this
problem I repeated the steps again and the same discrepancy in emails numbers
has occured. However, if an update has occured I will change the corresponding
Evolution version number.

Hope this helps.

Thanks again.

Cheers, 
Tony

Created attachment 118442 [details]
File that is created through saving the emails from the saved folder

Created attachment 118443 [details]
saved_emails --v-- reimport_test

This is the file created after running "diff" on the saved_emails file and the
reimport_test file. The file created after the saved_emails file is reimported
into Evolution.

I tried to attach the saved_emails --v-- UNE_ITD_req file but I was unable
because the diff file produced is greater than 20Mb (23Mb in fact). Which leads
me to believe that since there is so little difference between the saved_emails
file and the file that is obtained during re-importation then there is a problem
when the emails are saved, even though the status clearly says the correct
number of emails are being saved.

I will upload the text file produced from running diff on saved_emails --v--
UNE_ITD_req for you to use if required. I will provide the link shortly.

Thanks again.

Cheers, 
Tony

Hi Dave, 
I have managed to upload the diff file from the saved_emails and the 
UNE_ITD_req folders to my uni's FTP server.

This extremely large file (23Mb) can be found at the following address. Please 
contact me should you have any troubles viewing it.

http://www-personal.une.edu.au/~acrouch3/saved_emails_v_UNE_ITD_req

Hope this helps.

Cheers, 
Tony

Hi Dave, 
Just as clarification I mentioned earlier that I recently upgraded my version of
Evolution. This bug was originally filed when using:

evolution-2.2.2-5

I have since upgraded to:

evolution-2.2.3-2.fc4

I have performed the saving / re-importation process again under this new
version and the same problem holds true.

Just thought you should know.

Cheers, 
Tony

Hi Dave,
Just wondering how you are going with this issue as i have not heard from you
since posting the requested 'diff' files.

Cheers, 
Tony

Hi Tony, I have taken over Evolution maintenance since your last posting.

If you are using Fedora Core 5 or Rawhide, can you tell me whether you're still
seeing this problem?  I was unable to reproduce this using Rawhide, which means
it may have been fixed by the upstream developers since evolution-2.2.3.

evolution-2.6.2-1.fc5.5  (FC5)
evolution-2.7.4-3  (Rawhide)

Hi Matthew, 
Yes, I have since upgraded to FC5. But the problem is still present plus with
the addition of another problem.

When I first noticed this problem I filed an appropriate bugzilla report.

The link to the bug is: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=189126

I have not retested the problem since I have upgraded my evolution package.

I will attempt the process again over the coming days and get back to you
through the relevant FC5 page.

Thanks for your time.

All the best.

Cheers, 
Tony Crouch

Is this problem still present in Fedora Core 6 or later?

Hi Matt, 
Yeah, this is still causing problems in FC6. I just tried to update my evolution
software to ensure I have the latest release but am having trouble connecting.

My current version is:
evolution-2.8.0-7.fc6

In the tests I performed this morning the importation / re-importation process
seems to be handled much better by evolution as it states only 1 (one) email
seems to be 'lost' during the entire process.

However, running 'diff' on these two files (the 'saved' and 'imported' files)
yields a very different story, clearly showing there are multiple emails missing.

I ran the entire process again (on the same data set) and achieved different
emails missing (i.e. different emails are being lost each time).

I will attach the files to this bug over the next few minutes.

Thanks for your time.

All the best.

Cheers, 
Tony Crouch

Hi Matt, 
My apologies for not keeping on top of this bug. I tried uploading the 'diff'
files I mentioned in the previous comment but they are really, really large.
Well over 30Mb.

I am about to update my version of Evolution and try the entire procedure again.
I will keep you informed as to my progress.

Thanks for your time.

All the best.

Cheers, 
Tony Crouch

Matt, 
Am just wanting to let you know, I did eventually got the chance to perform 
the procedure again. I most probably don't have the latest version of 
Evolution at the moment because my 'yum' command is broken.

I will see whether this bug is still present once I do a fresh-install of F7 
later next week.

I will keep you updated as to my progress.

Thanks for your assistance with this bug.

All the best.

Cheers, 
Tony Crouch

Hi Tony, my apologies for letting this bug languish for so long.

Are you still able to reproduce this problem on Fedora 7 or later?

Hi Matt, 
Am glad to say I just tested out this bug problem again now I have done a fresh
install of F7. I am currently in the process of doing a yum update so I am bound
not to be dusing the most recent version.

Nevertheless, this problem seems to have been fixed.

Thanks for all your assistance.

Cheers, 
TC

Awesome, thanks for checking.  Closing this as CURRENTRELEASE (Fedora 7) but
feel free to re-open this bug if you encounter the problem again.