Uploading mail to an IMAP server with Thunderbird (2.0.0.14) sucks – perl script to the rescue

Today I was trying to upload a lot of mail (something like 160000 mails stored in a deeply nested folder structure of about 13000 folders) to an IMAP server. Those mails were converted from the Eudora mailbox format to a more sane Eudora mailbox format using “Eudora rescue”, and then imported into Thunderbird’s “Local folders”. From there, so we thought, it would only be a matter of minutes to put those mails onto the newly setup dovecot IMAP server, with one drag-and-drop action.

But…

The upload horribly failed after the 3rd folder or so, with no error message whatsoever. No matter what I tried, it would just fail silently. The debug logging (see Debugging/Thunderbird article in the Dovecot wiki) didn’t help, and neither did dovecot’s log on the server. The problem seems to be connected with creating folders on the server somehow – because uploading a big amount (400 for a small test) mails within one folder wasn’t a problem. I couldn’t find out if Thunderbird or dovecot is to blame here…

I eventually managed to get this big pile of mail to the server by using a perl script that I found on perlmonks.org. Thanks to Davis for providing this! I downloaded it, installed the Mail::MboxParser and Mail::IMAPClient dependencies (with two ‘cpan install …’ calls), changed the configuration in the script to our site’s data and ran it. ~30 minutes later, the mails were on the server. Only thing that could have been better: All mails were ‘marked as unread’ – but that’s easy to fix.

Update:
After looking into the perl code, I found a way to ‘mark as read’ during the transfer. I posted it under the original perlmonks.org script as a comment.

My instant messaging communications platform

I’m using this setup for quite a while now. It’s great in many ways – here some of the reasons:

  • I can access it from multiple hosts at once
  • I can use one unified user interface for all instant messenger protocols and IRC
  • I can access the platform even through my mobile phone via GPRS / WLAN
  • I have logs of all communications in one central place

This is how a few free software components work together to make it possible:

Overview (OpenDocument format)

Or as PNG file (rendered with GIMP):
Overview (PNG)

NX gives you “session test failed” and “Failure negotiating the session in stage ‘7’”?

… then check your iptables settings on the server. Perhaps you made the same mistake as I did: Restricting local traffic (on the ‘lo’ interface).

Add the following rules to your iptables ruleset, and make sure it’s at the beginning (before the restrictions):

iptables -A INPUT -i lo -j ACCEPT