IMAP Scraping

Update: For those of you coming here via search engines, a year or so later most of this was re-visited independently by Doug Hellmann on an article on Python magazine, which included a comprehensive solution - a copy of which I include here for archival purposes. His solution does not use SINCE to do incremental updates, but if I ever need to re-visit this, I will try to merge his code and mine.

I am currently in the process of coding a couple of Python classes to handle IMAP accounts, and am in awe at both the arcane complexity of RFC:3501 (which has a great many number of wonderful and useful things to learn) and the utter stupidity of whoever designed the Exchange renderer for the Calendar folder.

Yes, I am trying to read my appointments via IMAP. Don't ask why. Honestly.

All you need to know is that the asinine way in which the Calendar IMAP view was implemented has no way whatsoever to permit you so much as guess at item's actual appointment date without slurping and parsing the whole of the fake RFC:2822 body and the internal vCalendar - the date you get on the headers is the date an item was added to the folder.

Which, incidentally, is driving me up the wall - I know how good MAPI is, how Exchange works internally (at least until version 2000) and can't for the life of me figure out why they didn't add a simple X-Appointment-Date: header - would save me a world of bother.

Fortunately, Exchange honors SINCE, which means I can do a full Calendar scan once and then check back for any updated items (I hope - I still haven't checked what happens if I for some reason update old items).

Obviously, there isn't any way to get at the actual event data beyond the stingy sample you get on the vCalendar (which, incidentally, seems to be mis-encoded if your location or names include accented characters).

I sure as hell hope they change this in Exchange 12. Exchange is still the best corporate e-mail server as far as I'm concerned (provided you know what you're doing with it - which includes being absolutely positive you need it), but it has some amazingly limited and short-sighted "features".

Or, should I say, "incentives" for you to use Microsoft clients.