Forgot your password?
typodupeerror
Microsoft Software

Microsoft Opening Outlook's PST Format 319

Posted by ScuttleMonkey
from the still-a-long-way-to-go dept.
protosage writes to tell us that Microsoft Interoperability is working towards opening up Outlook's .pst format under their Open Specification Promise. This should "allow anyone to implement the .pst file format on any platform and in any tool, without concerns about patents, and without the need to contact Microsoft in any way." "In order to facilitate interoperability and enable customers and vendors to access the data in .pst files on a variety of platforms, we will be releasing documentation for the .pst file format. This will allow developers to read, create, and interoperate with the data in .pst files in server and client scenarios using the programming language and platform of their choice. The technical documentation will detail how the data is stored, along with guidance for accessing that data from other software applications. It also will highlight the structure of the .pst file, provide details like how to navigate the folder hierarchy, and explain how to access the individual data objects and properties."
This discussion has been archived. No new comments can be posted.

Microsoft Opening Outlook's PST Format

Comments Filter:
  • Oh no... (Score:4, Funny)

    by Mr. Roadkill (731328) on Monday October 26, 2009 @06:37PM (#29878525)

    Another sign of the Apocalypse - and it's a doozy. I always figured hell would freeze over before Microsoft opened up something like the .pst specs.

    • Re:Oh no... (Score:5, Interesting)

      by caffeinemessiah (918089) on Monday October 26, 2009 @06:42PM (#29878567) Journal
      This is incredibly brave of Microsoft, given that Outlook is so ubiquitous. I can see a number of good and not-so-good reasons for doing this:

      (1) They feel that Outlook is genuinely capable of withstanding competition from the likes of TBird and other competitors, and to be fair, the quality of Outlook has improved a lot.
      (2) They feel that opening Outlook's specs will give them access to iPhone app-store like ingenuity from the "crowd" (throw in your favorite buzzword here). Basically, let the hackers go at it and come up with neat little means to improve Outlook usability. If more products carry a "Works with MS Outlook" sticker, that can only be good for outlook (in one line of reasoning).
      (3) All the old, seasoned outlook engineers have retired or died, and they're hoping that someone can figure out the .pst specs.

      • Re:Oh no... (Score:5, Interesting)

        by Mr. Roadkill (731328) on Monday October 26, 2009 @06:55PM (#29878701)

        Expanding on point 2, Microsoft may want to open up the MAPI specs a little more for the benefit of iPhones and the like. At $DAYJOB, we have Exchange 2003 and a number of users with iPhones and we've seen some bizarre things happen on occasion with calendar entries (weirdness when one of a number of repeating appointments is changed or cancelled and not showing up as changed or removed on the iPhone, that kind of thing). While I'm prepared to believe that it's partially to do with Apple testing more thoroughly with and developing against Exchange 2K7, I can't help but feel that a better understanding of how Outlook communicates with Exchange and a better understanding of how Outlook represents the data internally would help other developers produce something that works better with Exchange.

        And that could well be Microsoft's strategy...domination at mail-and-collaboration server end. If they open up the client specs a little more, and that makes Exchange 2010 and beyond more attractive, they've won.

      • Re: (Score:3, Funny)

        by davester666 (731373)

        In other news, Microsoft disables the ability for all of it's software to import and/or export PST files...

      • Re:Oh no... (Score:5, Insightful)

        by Technician (215283) on Monday October 26, 2009 @06:59PM (#29878739)

        I think it is much more likely the reason is (4).

        (4) As standards committees and governments adopt open formats, Outlook is at risk of being rejected for the closed format. Opening the format ensures the benefits of the Outlook/exchange server will remain the industry standard in software and support purchases. Like IE, expect some features to simply work better on an Exchange Server with Outlook on Windows while unsupported applications on a foreign OS may have random errors and glitches.

        • Re:Oh no... (Score:5, Insightful)

          by diegocgteleline.es (653730) on Monday October 26, 2009 @07:47PM (#29879171)

          Let me add another reason:

          (5) They don't care about the outlook format because Sharepoint is the new closed format. They don't care if your outlook mailboxes (or .doc or anything else) is in an open format because you put it all in sharepoint. You still can read your mailbox with another program, but because the "metadata" of your IT infrastructure (which isn't a single file, but a lot of files with owners and relationships between all them) is stored in sharepoint you're tied to it for the eternity. This is a brilliant move - Microsoft can convice governments that their outlook and office and all their apps are using open formats, but no government will ask about the openness of sharepoint because it's not an application that reads some kind of document.

          • Re:Oh no... (Score:5, Funny)

            by Abreu (173023) on Monday October 26, 2009 @08:44PM (#29879623)

            (6) It's a trap!

          • Re: (Score:3, Interesting)

            (-5) Sharepoint is stored in a SQL server database. The structure is vaguely nightmarish because of the desire for obfuscation, but it is perfectly possible to get the files back out with a bit of work. It is less of a lock-in than a .pst file would be, even with the release of these specs.

            I'll bet that Alfresco or Knowledge Tree's commercial products can come up with modules to migrate from a Sharepoint if they haven't already.

          • Re:Oh no... (Score:5, Informative)

            by onenil (624773) on Tuesday October 27, 2009 @03:42AM (#29881433) Homepage
            *sigh*

            SharePoint was more open that the PST format was prior to this announcement. The (well documented) SharePoint API [microsoft.com] enables access to all content - it would be relatively trivial to write software that could walk your entire SharePoint content dbs and indeed farm to extract all data out in a way that could easily be implemented in alternative products. I'm sure its been done. Hell, there's software that does the reverse (and I know this being a SharePoint guy) - that use the very same API to insert data into a SharePoint environment from say a Lotus Notes environment. And trust me, you have as much access to write as you do to read data.

            Repeat after me - SharePoint does not lock your data up. It implements a reasonably good document management, content management, workflow, "intranet in a box" site - it aint no drupal when looking specifically at CMS, but that's one of the many tools on this swiss army knife. Sure, corporations will be 'locked in' to SharePoint, but that is because the alternatives that come close to doing what it does are woeful (*cough* Lotus Notes). They're locked in to its functionality, which - correct me if I'm wrong - is ultimately what you choose one software product over another on.
        • Re: (Score:2, Interesting)

          I think the most likely explanation is that PST files are deprecated in the next version of Exchange... they are pushing for people to move to server-integrated archiving instead. That will make PSTs somewhat redundant so why not open up the spec if it gets you warm fuzzies from the industry.
          A comment from an Exchange developer on the EHLO blog:

          "To put it simply you need to move away from PSTs. Larger mailboxes are the answer here. In addition you can leverage, single item recovery, and our messagin
      • Re: (Score:3, Interesting)

        by gerf (532474)

        and to be fair, the quality of Outlook has improved a lot.

        I love how Outlook uses almost 300MB of virtual memory at work. Seriously, wtf.

        • Re: (Score:3, Funny)

          by tsm_sf (545316)
          Well to be fair you do need to have templates like 'balloon party' ready to go at a moments notice.
      • by roc97007 (608802)

        (4) They're about to change the pst spec to a different closed standard with some backwards compatibility.

      • Re:Oh no... (Score:5, Informative)

        by BitZtream (692029) on Monday October 26, 2009 @07:50PM (#29879203)

        Reality check:

        The PST format is rather useless. You can already access all the data on a Windows machine (which you have already to create it anyway) using Outlook plugins, either a COM Outlook Object Model plugin or a Exchange client plugin, depending on what you need.

        So okay, now things like Thunderbird can import the mail from Outlook, which is good for people who use POP3 I guess, IMAP and Exchange store the mail on the server so theirs no real need.

        Products won't carry a 'Works with Outlook' sticker because of this, the file is locked when Outlook is open, you you have to use an Outlook plugin if you want to do anything useful with it for normal people who use Outlook.

        As someone who writes Outlook plugins for a job, this is rather useless for much other than exporting data from a backup without reinstalling Outlook after a crash of your system.

        I.E. useful only in a limited set of circumstances that are really a corner case.

        This doesn't do anything for communicating with Exchange, which is really what you want.

        • Re: (Score:3, Interesting)

          This doesn't do anything for communicating with Exchange, which is really what you want.

          Well it isn't what I want. I want TBird (with its calendaring extensions) to be able to read the Outlook address book and calendar from the .pst files.

          Then I want TBird to be able to sync to my phone but that's another story and doesn't seem likely to happen so I may as well get an android and run TBird on it.

        • Re: (Score:3, Interesting)

          by jschrod (172610)
          Well, (1) I want to access PST file content on non-Windows systems. E.g., for a search engine. (2) I want to access PST file content on systems where COM-embedding has been turned off for "security reasons". (One of our bank customers has that.) If I think more about it, I might even find more use cases.

          I.e., not everybody in the world has your limited use cases. I welcome the opening of the PST specs.

      • Re:Oh no... (Score:4, Insightful)

        by CastrTroy (595695) on Monday October 26, 2009 @08:06PM (#29879325) Homepage
        I think a much better idea would be to rewrite Outlook to use a real database as a backend. They already have SQL Server. Why not just store all your mail in a SQL Server database? You wouldn't have problems with maximum file sizes. You would have much better scalability for those with gigabytes of email, and you could have a common interface working with the data in the terms of running SQL queries. I don't know why no other email client like thunderbird wouldn't do the same. Make it easy to access your email store, and you could easily write tons of applications to access your email.
      • Re: (Score:3, Interesting)

        by StormReaver (59959)

        > This is incredibly brave of Microsoft, given that Outlook is so ubiquitous.

        Hardly. This is the result of Microsoft having to abide by the results of a court case that they fought against tooth and nail, that they ignored for months, then finally, begrudgingly, realized they had lost. This is Microsoft doing something because they have absolutely no other choice. Everything else has failed, so Microsoft is finally, years later, complying with court orders trying to remedy Microsoft's illegal abuses o

    • Re: (Score:3, Funny)

      by R2.0 (532027)

      I'm thinking more "The Ring" for software - thousands of software developers open the specifications file and all die horrible deaths within a week.

    • Re: (Score:3, Insightful)

      Now if only Thunderird gets a .pst import function for the calendar and address book it will be almost perfect for my use... at least v2 - god knows what has been done to v3.
    • Quote from the link you gave:

      How many other fast-tracked ISO standards have no conforming implementations.

      Answer: at least one. ODF.

      Did you have a point?

  • by bomanbot (980297) on Monday October 26, 2009 @06:52PM (#29878677)
    Its good to see Microfsoft open up the Outlook PST format, if only to improve importing into other mail clients like Thunderbird etc.

    But honestly, using the PST format in other applications sounds like a terrible idea to me: Those monolithic PST files, which Outlook uses to store mail data get corrupted easily (at least in my experience) and storing all your email data in one gigantic file always struck me as a really bad design choice anyway.
  • by spectre_240sx (720999) on Monday October 26, 2009 @06:58PM (#29878725) Homepage

    I'd wager that Microsoft is willing to do this because the .pst format is becoming irrelevant. Medium and large businesses already want nothing to do with them due to issues with performance and management. That leaves small businesses and a small number of home users. With hosted exchange options becoming more common among small businesses, the need for .pst files is going away very quickly.

    • by shutdown -p now (807394) on Monday October 26, 2009 @07:25PM (#29878965) Journal

      .pst is an Outlook message database, not Exchange message database. It doesn't matter where your Exchange is hosted, if you use Outlook to connect to it, it caches local copies [wikipedia.org] of all data you worked with in a .pst file on your machine.

    • by Bazman (4849) on Monday October 26, 2009 @07:27PM (#29878991) Journal

      Our central IT dept gives us something like 100MB of quota on the Exchange server. Running out of quota? The official advice is 'save your stuff in a PST file'.

      Of course you can't save your PST on the IT dept-supplied backed-up network drive because MS say "don't do that". So people end up with PST files on unbacked-up local storage on a particular machine...

      • by sarhjinian (94086)

        I think I work for the same people you do.

      • Re: (Score:3, Insightful)

        by MarcQuadra (129430)

        It's more than MS saying 'don't do that'!

        The PST format requires a lot of small direct I/O, and when you mount one over CIFS/SMB you run the serious chance of filling up the queues on the client or even the server. I've brought down a fully-loaded and patched Server 2003 box with a PST -> PST transfer over the wire, and by 'down' I mean really down, not responsive, not accepting new connections, and needing a reboot.

        I've restored so many corrupt PST files from backup that I'm considering setting up a Dov

        • Re: (Score:3, Interesting)

          by Bazman (4849)

          You're lucky if your PSTs get backed up nicely. Lots of people keep Outlook running all the time, and that means it has an exclusive lock on the PSTs. Then the backup process fails to copy the PST.

          So we have files that aren't on a backed-up server, can't live on a backed-up network share, and often fail to backup from local starage via local back-up systems.

          No wonder people like to print their emails out.

    • by Monoman (8745) on Monday October 26, 2009 @08:03PM (#29879293) Homepage

      Bingo! I believe MS has already banned PSTs in house. The writing is on the wall where I work. Too many times PST get corrupted which turns into support nightmares for the VIP customers. Once the VIPs (they sign the checks) are sold on getting rid of PSTs and expanding the mailbox sizes they will pay the bill.

  • by MoFoQ (584566) on Monday October 26, 2009 @07:00PM (#29878749)

    what happen to the obligatory tag that gets added on Slashdot to a post about Microsoft "opening up" something, the "itsatrap" tag.

    here are some prime examples:
    Microsoft Partially Opens Proprietary XML Format [slashdot.org]
    (mainly because this happened: Microsoft Open Document Standard Not So Open [slashdot.org])

    Microsoft Releases Linux Device Drivers As GPL [slashdot.org]

    in fact, there are plenty of other examples in the " itsatrap [slashdot.org] " tag-egory

    • Maybe nobody is yet sure that the current action is really a trap... Altough, comming from Microsoft, it is quite likely, that is why people are trying to figure the plot here.
    • by _Sprocket_ (42527)

      We can also tag the sky "blue" if that helps you.

  • So where's the link to the RFC or other plain text document describing the .PST file?

  • by zhilla2 (1586095) on Monday October 26, 2009 @07:20PM (#29878925)

    People who program different migration utilities benefit from this, and of course users of such tools. Even wild ideas like Fuse filesystem that mounts it as Maildir.
    So, converters, importers, exporters, indexing tools, repair/forensics, optimize/defragment/find duplicates tools, sort, grep.
    Also, if its a standard than it needs to be STANDARDIZED, so no special treatment for own products.

  • by RichMan (8097) on Monday October 26, 2009 @07:29PM (#29879011)

    Make your named socket a .pst file and outlook can access your real email database through the defined interface.
    Nice and spiffy and you don't end up tied to the Microsoft format.

    • by cortana (588495)

      Good luck getting Outlook to open \\pipe\whatever :)

      Hm, can you seek on a file descriptor opened from a pipe?

  • Embrace
    Extend
    Extin... oh wait

  • I Don't Have a .PST (Score:3, Interesting)

    by Kozar_The_Malignant (738483) on Monday October 26, 2009 @08:08PM (#29879331)
    I have an .ost file on my laptop you insensitive MS clods. Does this great revelation include them?
    • Re: (Score:3, Funny)

      by CoolGopher (142933)

      I have an .ost file on my laptop you insensitive MS clods.

      In Swedish, a .ost file would imply that you have a .cheese file. Maybe it's a really cheesy format?

  • Thank you RMS (Score:2, Insightful)

    by Dysphoric1 (1641793)

    And the iconoclastic tree of RMS bears another fruit. You can bet that without the pressure exerted by free and/or open source software and its advocates this would never have happened...

    (I now await moderation punishment for having mentioned the name of him is not to be named...)

  • So now we can write open source tools to fix corrupt PST files!

    Don't even think about doing anything open source with PST files, until you have a tool to fix the files when they go corrupt.

<< WAIT >>

Working...