Re: [Evolution] Duplicate message

[guenther <guenther rudersport de>]

> > You did not tell us, where the mails are (ISP, local?) and which
> > protocol you use to access the mails (POP3, IMAP, local...). If you tell
> > that, maybe I can come up with some solution.
>
> They are local, in Evolution.  I have imap and pop3 servers locally,
> though.  I hope I won't have to do this again; I've set fetchmail to use
> message IDs to prevent duplicates, now.
>
> Thanks for your help; I look forward to hearing more.


First of all: Make a backup of ALL mail !!
A tarball of the entire ~/evolution folder would be good. I warned you.

Sure, I will not be responsible for any harm done. Run at your own risk.
Disclaimer... I tested that script though, and it worked as expected.


I played around a bit with procmail and here possibly is a solution for
your problem. As you said, you only have local Evo mail. I suppose they
are in mbox format.

Before messing around with the mbox files, you should exit Evo and maybe
run 'killev'. Remove all indexing files as needed (mbox.ibex.index,
mbox.ibex.index.data, mbox.ev-summary should be them) -- even in
subfolders.

Go to the dir, where your mails are stored. Let's assume, they are in
% cd ~/evolution/local/Inbox

% find . -name 'mbox' -exec ./dedupe.sh {} \;

That command executes dedupe.sh (in the local dir Inbox) and does the
job.

--- dedupe.sh ---
#!/bin/sh
[ -e dedupe.mbox ] && echo 'panic: dedupe.mbox exists, exiting' && exit

cat $1 | formail -D 8192 .msgid -s >> dedupe.mbox

mv -f dedupe.mbox $1
--- sipp ---

Save that file and don't forget executable rights. That script gets feed
the name of every mbox file by the find command.

The entire mbox file is piped into formail, which splits it up into the
seperate mails, checks the message-id and (if new id) appends it to the
dedupe.mbox file.

If done with a mbox, the debupe.mbox overwrites the old one, now without
all duplicate mails.


You have to change the value 8192. It is the maximum size of the
message-id cache. 100 byte per id should really be enough, calculate it
by your estimated message count.

If that value is too small, you may end up with not-deleted dupes. You
want to filter all dupes, so you have to store all not duplicated
message-ids. Make that value big.


Beware: That find command will execute dedupe.sh for every 'mbox' file
recursive(!) from your current dir. If you execute it from
~/evolution/local it will proceed even on your Outbox.


Hope, that explanation was clear enough. If you have questions, feel
free to ask me. Good luck...

...guenther


-- 
char *t="\10pse\0r\0dtu\0  ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}