Document Wars

IDABC ODEF Workshop 2007 in Berlin

As information exchange in and with public administrations is very often bound to documents, editing, archiving and exchange possibilities for documents are crucial for the optimum function of administrations, both in terms of practicality and cost. Initiatives such as the PEGSCO Recommendations on Open Document Formats published by the IDABC Management Committee, demonstrate public administrations preference for "open" document exchange and storage formats that are subject to formal standardisation via international standardisation procedures.   The primary objectives of the Berlin event, held at the German Federal Ministry of the Interior (BMI), were to: compile further input from Member State public administrations on their experiences and strategies on ODEF gather industry viewpoints on the initiatives relating to ODEF standardization and information on future standardisation developments provide a platform for exchange between stakeholders in public administrations and main industry players The program of the workshop included, among other: ODEF Strategies: Examples from European Administration Practical Experiences with the implementation of ODEF Report on ODEF-Standardisation activities  4 parallel sessions with participants   A panel discussion with stakeholders 

When such new features that enhance the interoperability require enhancements to the Open Document file format we will propose the necessary changes to the OASIS Open Document TC. This way not only OpenOffice.org but also Open Document benefits from our efforts. Florian Reuter, who now works for Novell, lists some of the changes we have in mind in his blog . So there are a lot of common ideas how we can improve the interoperability between OpenOffice.org and Microsoft Word documents and I hope we can work together with Florian here.

Massachusetts last week officially confirmed that its executive agencies for now will continue using Microsoft Office instead of switching to alternative desktop applications. But by Jan. 1, in keeping with a controversial policy announced last year, the state plans to start adding plug-in software that will let its Office users create and save files in the industry-standard OpenDocument format.

he OpenDocument Format (ODF) is an open XML-based document file format for office applications to be used for documents containing text, spreadsheets, charts, and graphical elements. The file format makes transformations to other formats simple by leveraging and reusing existing standards wherever possible. As an open standard under the stewardship of OASIS, ODF also creates the possibility for new types of applications and solutions to be developed other than traditional office productivity applications.

A week or so ago, I published a podcast at IT Conversation with Scott Mace interviewing Gary Edwards about OpenDocument. Edwards is the president of the OpenDocument Foundation. OpenDocument Foundation is a non-profit that works to promote the OpenDocument file format–an XML file format for office documents. There’s no question that businesses want an XML-based file format for office data. The question, naturally, is which XML-based file format. Microsoft has it’s own XML-based file format called OpenXML.

June 08, 2007 (Computerworld) -- Only a tiny fraction of the PCs at Massachusetts government agencies are able to use the Open Document Format (ODF) for Office Applications, despite an initial deadline of this month for making sure that all state agencies could handle the file format.

You might think the steady defeat of bills in several U.S. states to mandate the use of free interoperable file formats might dampen the spirits of IBM, one of the prime supporters of ODF (OpenDocument Format). Far from it, said IBM's Bob Sutor, who sees the recent news as par for the course in the evolution of any open standard.

the general idea is for there to be a layer of data on the Internet that he calls the “data bus” and the way the data bus works is not too different from how we’ve heard Microsoft’s WinFS filesystem described where connectivity between related data items is organic rather than synthesized. For example, whereas today, a mashup developer may have to call upon two APIs to show where a specific Starbucks is on a map, the Semantic Web approach might involve little more than a simple query of that data bus using a query technology called SparQL.

Only a tiny fraction of the PCs at Massachusetts government agencies are able to use the Open Document Format (ODF) for Office Applications, despite an initial deadline of this month for making sure that all state agencies could handle the file format.

So I'm disappointed. And not just on behalf of open documents, but on behalf of the CIOs of this country, who are now caught between a rock and a hard place, without a paddle to defend themselves with if they won't to do anything new, innovative and necessary, if a major vendor's ox might be gored in consequence. After the impressive lobbying assault mounted over the past six months against open document format legislation, I expect you won't be hearing of many state IT departments taking the baton back from their legislators.    And who can blame them? If they tried, it wouldn't be likely to be anything as harmless as an open document format that would bite them in the butt.

Subject: Suggested ODF1.2 items From: "Florian Reuter" <freuter@novell.com> To: <office@lists.oasis-open.org> Date: Mon, 20 Nov 2006 17:03:24 +0100 Suggested enhancement for OpenDocument V1.2

Microsoft trounces pro-ODF forces in state battles over open document formats Eric Lai and Gregg Keizer   document.write(''); if (navigator.userAgent.indexOf("Gecko")==-1) { document.write(''); } document.write(''); if (document.getElementById('dclk999')) { document.getElementById('dclk999').src = 'http://ad.doubleclick.net/adi/idg.us.cpw.desktopapplications/index;pos=imu;tile=3;sz=336x280;ord=' + ord + '?'; } document.write(''); if ((!document.images && navigator.userAgent.indexOf('Mozilla/2.') >= 0)|| navigator.userAgent.indexOf("WebTV") >= 0) {document.write('');} June 03, 2007 (Computerworld)

Keeping it private The other problem, Mathers said, was the jargon-laden disinformation that committee members felt they were being fed by lobbyists for both IBM and Microsoft. Although lobbyists would tell the committee one thing in private, they got cold feet when asked to verify the information publicly, under oath. "Suddenly, nobody wanted to sign witness affirmation forms and testify," he said. That undermined the credibility of each side, but it particularly damaged the position of ODF proponents. After Wyne testified publicly that in Massachusetts, only a handful of computers had thus far been converted over to using ODF, IBM declined to dispute her claims, Mathers said -- despite having earlier given "gleaming" reports on the progress of ODF in Massachusetts. "That's when I really started to question the whole bill," he said.

here was strong consensus among Member State administrations on the necessity to use ODEF on "openness" being the basic criteria of ODEF and resulting requirements towards industry players / consequences for public administrations There is a general dissatisfaction with the perspective of having competing standards; One format for one purpose: Administrations should be able to standardize (internally) on a minimal set of formats; No incomplete implementations, no proprietary extensions; Products should support all relevant standards and standards used should be supported by multiple products; Conformance testing and document validation possibilities are needed -> in order to facilitate mapping / conversion; Handle the legacy / safeguard accessibility

I have an email apparently originating from Microsoft asking people to support their opposition to California A.B. 1668 - Open Document Format, Open Source. by writing to the California Assemblymen involved in this bill. This email has contact information for the Assemblymen involved, and a lot of information about their position regarding ODF.

Rep. Homan and his son Doug tried to add their little open standards boost to SB 1974 as quietly as possible. They wanted the modified bill to at least get through its first committee approval before anyone spotted what they had done. But Microsoft's Florida lobbyists were on the ball and spotted it almost immediately. "It was like the movie 'Men in Black,'" says Rep. Homan. "Three Microsoft lobbyists, all wearing black suits." Another lobbyist (unaffiliated with Microsoft) who would speak only "on background" laughed at the "Men in Black" description. "I know those guys," he said. "They even wear sunglasses like in that movie. They are the 'Men in Black' of Florida lobbying, for sure." A legislative staff employee who would lose his job if he were quoted here by name said, "By the time those lobbyists were done talking, it sounded like ODF (Open Document Format, the free and open format used by OpenOffice.org and other free software) was proprietary and the Microsoft format was the open and free one." Two other legislative employees (who must also remain anonymous) told Linux.com that the Microsoft lobbyists implied that elected representatives who voted against Microsoft's interests might have a little more trouble raising campaign funds than they would if they helped the IT giant achieve its Florida goals.

EIOffice includes the features found in Microsoft Office, plus a number of productivity features found nowhere else.

A fatal flaw in your analysis By Marbux Posted Saturday 21st April 2007 08:15 GMT Your analysis contains a fatal flaw, Martin. That is your belief that adequate Microsoft XML <> OpenDocument translators will be available. In fact, all of the translators suck mightily and there is no prospect at all of them being perfected. The major problems are: (i) that Microsoft's XML formats seem deliberately designed to thwart their parsing with XPath, which is essential to XML transformations; (ii) that Microsoft's "XML" file formats include binary blobs, bitmasks, and multiple Windows and Microsoft dependendencies, all of which defy XML transformations; and (iii) OpenDocument assumes a richer page layout engine than Microsoft Word provides, so while DOCX can be completely mapped to ODT it is impossible to fully map in the other direction without declaring an MS Office interoperability subset of OpenDocument and ODF applications implementing a compatibility mode with reduced features. (That is more than somewhat ironic, given Microsoft's spin that it couldn't implement all of its features in OpenDocument. In fact, the exact opposite is true.) In fact, Steve Ballmer is on record as saying that the developers of the Novell-Microsoft-Clever Age plug-ins will not even attempt to achieve full fidelity file translations between the two formats. http://www.eweek.com/article2/0,1895,2050848,00.asp?kc=EWEWEMNL103006EP17A Those translators achieve at best far less conversion fidelity than existing file conversion filters between OpenDocument and Microsoft binary file formats such as the OpenOffice.org conversion filters, which achieve only about 80 per cent fidelity. The file format cognescenti know this. See e.g., the paper by Gary Edwards and Sam Hiser included in this edition of the European Journal for the Informatics Professional. http://www.upgrade-cepis.org/issues/2006/6/up7-6Hiser.pdf (PDF). (Note that I contributed to that paper.) And as also detailed in that paper, what works well enough for some of us does not necessarily work well enough for all. Anything less than full fidelity data conversions is absolutely unacceptable in the context of wholly automated business processes and is in fact illegal in various contexts, including government records. So your thesis doesn't fly. In fact, I'd go so far as to bet that you have been suckered by the Microsoft spin doctors. Another indication is your depiction of the file format wars as being waged primarily between IBM and Microsoft, a recent theme of Microsoft's public relations machine. While it is seductive to believe that the controversy is just another chapter in the war between major competitors, the pro-ODF camp is far broader than IBM. For example, nearly 20 governments recently opposed fast track processing of Microsoft's draft standard at ISO. Do you believe they were all carrying water for IBM? Government bodies in more than 50 nations have chosen to adopt ODF. http://opendocumentfellowship.org/government/precedent And dozens of developers now support the OpenDocument standard in their applications. http://opendocumentfellowship.org/applications While IBM has had a noteworthy role in proliferating the OpenDocument formats, there is a movement without a recognizable leader in the industry. When it comes to vendor influence on things relevant to ODF, Sun Microsystem's far outshines IBM. But in fact, a core group of open standards and free and open source developers and advocates -- inside and outside government -- have played a far larger role. This is a customer-driven phenomenon, not a vendor-driven effort as you portray. So I will respectfully suggest that you reexamine your position on these issues. Reasonable minds can differ, but not on the grounds you advocate.

What’s distinctive about the goals of OOXML? Primarily, to have full fidelity with pre-existing binary documents created in Microsoft Office. “What people want is to make sure that their billions of important documents can be saved in a format where they don’t lose any information. As a design goal, we said that those formats have to represent all the information that enables high-fidelity migration from the binary formats”, says Paoli. He mentions work with institutions including the British Library and the US Library of Congress, concerned to preserve the information in their electronic archive. I asked Paoli if such users could get equally good fidelity by converting their documents to ODF. “Absolutely not,” he says. “I am very clear on that. Those two formats are done for different reasons.” What can go wrong? Paoli gives as an example the myriad ways borders can be drawn round tables in Microsoft Office and all its legacy versions. “There are 100 ways to draw the lines around a table,” he says. “The Open XML format has them all, but ODF which has not been designed for backward compatibility, does not have them. It’s really the tip of the iceberg. So if someone translates a binary document with a table to ODF, you will lose the framing details. That is just a very small example.”

“Open Document Format and Office Open XML have very different goals”, says Paoli, responding to the claim that the world needs only one standard XML format for office documents. “Both of them are formats for documents … both are good.”

I put it to Paoli that OOXML is hard to implement because of all its legacy support, some of which is currently not well documented. “I don’t believe that at all. It’s actually the opposite,” he says. He make the point that third parties like Corel, which have previously implemented support for binary formats like .doc and .xls, should find it easy to transition to OOXML. “We believe Open XML adoption by vendors like Corel will be very easy because they have already been doing 90% of the work, doing the binary formats. The features are already there.”

Where do I stand? Well, at the risk of getting a lot of hate mail saying I am in Microsoft's pocket, if what Robertson and Paoli say about OpenXML and ODF is correct, then I think OpenXML is needed, at least until ODF becomes backward-compatible with older Office file formats and offers the capabilities large organizations require in their productivity solutions to run their business.