Well well well. We knew that IBM had access to the secret binary blueprints back in 2006. Now we know that Sun ALSO had access!

And why is this important? In June of 2006, Massachusetts CIO Louis Gutierrez asked the OpenDocument Foundation's da Vinci Group to work with IBM on developing the da Vinci ODF plug-in clone of Microsoft's OOXML Compatibility Pack plug-in. When we met with IBM they were insistent that the only way OASIS ODF could establish sufficient compatibility with MSOffice and the billions of binary documents would be to have the secret blueprints open.

Even after we explained to IBM that da Vinci uses the same internal conversion process that the OOXML plug-in used to convert binaries, IBM continued to insist that opening up the secret binaries was a primary objective of the OASIS ODF community.

For sure this was important to IBM and Sun, but the secret binaries were of no use to us. da Vinci didn't need them. What da Vinci needed instead was a subset of ODF designed for the conversion of those billions of binary documents! A need opposed by Sun.

Sun of course would spend the next year developing their own ODF plug-in for MSOffice. But here's the thing: it turns out that Sun had complete access to the secret binary blueprints dating back to 2006!!!!!!

So even though IBM and Sun have had access to the blueprints since 2006, they have been unable to provide effective conversions to ODF!

This validates a point the da Vinci group has been trying to make since June of 2006: the problem of perfecting a high fidelity conversion between the billions of binaries and ODF has nothing to do with access to the secret binary blueprints. The real issue is that ODF was NOT designed for the conversion of those binary documents.

It is true that one could eXtend ODF to achieve the needed compatibility. But one has to be very careful before taking this route. The Sun - ODF covenant not to sue specifically exempts eXtensions to ODF not involving Sun! Meaning, if the interoperable subset of ODF was designed and implemented without Sun-OASIS participation and approval, the covenant not to sue does not apply. Developers beware! You cannot safely eXtend ODF without Sun's permission.

Here is the relevant text from Sun's covenant: "Sun irrevocably covenants that, subject solely to the reciprocity requirement described below, it will not seek to enforce any of its enforceable U.S. or foreign patents against any implementation of the Open Document Format for Office Applications (OpenDocument) v1.0 Specification, or of any subsequent version thereof ("OpenDocument Implementation") in which development Sun participates to the point of incurring an obligation, as defined by the rules of OASIS, to grant (or commit to grant) patent licenses or make equivalent non-assertion covenants."

The obscurity of intent is masked in clever legalese. Which means, bring your legal team if you want to eXtend ODF, and prepare to argue.

My point is that this covenant could have been written clear and direct to say that Sun will not sue anyone for any reason related to ODF. But they didn't do that.

People will of course wonder why ODF is so bad that it might a as well be ZERO interop? The answer to this question is complicated, but a good place to start is to observe that, just as OOXML is an XML encoded dump of MSOffice in-memory-binary-representation, ODF is an XML encoded dump of OpenOffice/StarOffice in-memory-binary-representation.

The interop problem truly kicks in at the level of specifying this encoding. The Ecma and OASIS technical committees are responsible for fully specifying the OOXML and ODF. This means a complete syntax and semantic description needed to properly implement the specs. ODF and OOXML share one very big fault; the presentation-layout layer (or styles) is not fully specified! We have the syntax but not the semantics describing how layout works. This is particularly problematic in that both ODF and OOXML are application specific dumps. While they each do a good job separating content from presentation, neither fully specifies the presentation layer. Nor is the presentation layer portable in the sense that a CDF XHTML + CSS separation is portable.

And it is the presentation layer that binds the formats to their originating applications. MSOffice has one way of implementing basic document structures like lists, fields, tables, sections and page dynamics, and, OpenOffice has another. That these application differences are embodied in the formats creates an enormous interoperability problem. Applications can exchange content, but break when trying to interpret another applications presentation-layout layer. Especially when that presentation layer is under specified!

There were three aspects of ODF 1.0 that were under specified: numbered lists, formulas, and styles (presentation-layout). ODF 1.2 attempts to fix the formula problem, but does nothing for styles. The numbered lists "interop" problem was not fixed, but exacerbated.

So even though the binary blueprints were released two years ago to Sun and IBM, we have yet to see any improvement in conversion fidelity able to crack the lock MSOffice workgroup-workflow business processes have in the marketplace. Writing a subset of ODF enabling us to achieve that high fidelity conversion has a legal cloud hanging over the process. And all of these concerns are shadowed by the fact that neither OOXML or ODF have fully specified their presentation layers!

No wonder the W3C's formats are attracting so much attention.

~ge~

I don't agree with this statement from Microsoft's Oliver Bell. As someone who served on the OASIS ODF Technical Committee from it's inception in November of 2002 through the next five years, i have to disagree.

It's not that Microsoft wasn't welcome. They were. It's that the "welcome" came with some serious strings. Fo rMicrosoft to join OASIS would have meant strolling into the camp of their most erstwhile and determined competitors, and having to ammend an existing standard to accomodate the implementation needs of MSOffice.

There is simply no way for the layout differences between OpenOffice and MSOffice to be negotiated short of putting both methodologies into the spec. Meaning, the spec would provide two ways of implementing lists, tables, fields, sections and page dynamics.

A true welcome would have been for ODF to have been written to accomodate these diferences. Rather than writing ODF to meet the implementation model used by OepnOffice, it would have been infinitely better to wrtite ODF as a totally application independent file format using generic docuemnt structures tha tcould be adapted by any application.

It turns out that this is exactly the way the W3C goes about the business of writing their fiel format specifications (HTML, XHTML, CSS, XFORMS, and CDF). The results are highly interoperable formats that any applciation can implement.

You can harmonize an application specific format with a generic, applicaiton independent format. But you can't harmonize two application specific formats!!!!

The easy way to solve the document exchange problem is to leave the legacy applications alone, and work on the conversion of OOXML and ODF docuemnts to a single, application independent generic format. The best candidate for this role is that of the W3C's CDF.

CDF is a desription of how to combine existing W3C format standards into a single container. It is meant to succeed HTML on the Web, but has been designed as a universal file format.

The most exciting combination is that of XHTML 2.0 and CSS in that it is capable of handling the complete range of desktop productivity office suite documents. Even though it's slightly outside the W3C reach, the most popular CDF compound is that of XHTML, CSS and JavaScript. A combination otherwise known as "AJAX".

Dana
Blankenhorn posted this article back in March of 2007.  It was
right at the time when the OASIS ODF TC and Metadata XML/RDF SC (Sub
Committee) were going at it hammer and tong concerning three very
important file format characteristics needed to fulfill a real world
interoperability expectation:

....
Compatibility
- file format
level interop -
:::  backwards compatibility / compatibility
with existing file formats, including the legacy of billions of
binary Microsoft documents

.......
Interoperability
- application
level interop-
::::::  application interoperability including
interop with all Microsoft applications

..........
Convergence -
cross platform interop
:::::::::  the portable XML document
promise of being able to move content/data/media rich document
packages across desktop - server - device and Web information
systems.  Another way of expressing this would be the exchange
of portable XML documents with data bindings across desktop
productivity environments, enterprise publication-content-archive
management systems, SaaS, SOA and Web 2.0

Organizations the
world over are at an important inflection point.  They need to
move existing documents, applications and processes to XML. 
Once there, the explosively rich digital civilization emerging around
Web based collaborative computing is within reach.  The big
enchilada being SaaS, SOA and the Web 2.0 AJAX-REST mashup interop
that promises to shatter all the restrictions of big vendor
controlled "interop".

The problem is that they have
to answer this question, "Which
XML?  OOXML, or ODF?"

Microsoft
has provided a free OOXML plugin for most of the installed base of
500 million MSOffice bound desktops.  The key aspect of the
OOXML plugin is that it effectively answers the three criteria above;
the issues of compatibility, interoperability and convergence.

The
problem is that the OOXML Plugin, (otherwise known as the Microsoft
Compatibility Pack), is entirely bound to Microsoft legacy Windows
platform dependencies and, newly emerging Vista Stack dependencies
(as in desktop-server-device-web MS platform mashup).  There is
simply nothing open or application - platform independent about the
Microsoft application specific implementation of OOXML. 

So
the question is, can one implement ODF meeting the same
criteria?

Massachusetts, California and Denmark have all
answered that question with a resounding NO.  And Lord knows,
Massachusetts tried!  Tried but failed.

Massachusetts
even conducted a year long "Pilot Study" regarding the
issues and barriers they would need to overcome to successfully
implement ODF.  The result of the Pilot Study was the conclusion
that the only way any government could successfully implement ODF was
through the use of ODF Plugins for MSOffice.  Plugins that would
effectively clone the OOXML Plugin for MSOffice, except that they
would map the applications internal in-memory-binary-representation
of a document to ODF instead of to OOXML.

The reason for this
need is that there are two barriers to every ODF implementation. 


The first barrier is that of high fidelity document
conversion : the application “imbr”
binary <> ODF conversion.

The second barrier is the
"impossible" one.  This is the barrier presented by
years of MSOffice bound workgroup-workflow business process
development.  Because of the second barrier, any ODF
implementation must be able to integrate into existing business
processes. 

The Pilot Study discovered and exposed the
grip these day to day MSOffice bound business processes have on
governments.  The Pilot Study also revealed the incredible cost
of trying to rip out and replace MSOffice; having to re engineer
these bound business processes, line of business integrated apps and
complicated assistive technology add-ons for ODF alternative
solutions like OpenOffice.

In fact, there was no proof that
one could even recreate / re engineer the bound business processes
for OpenOffice at any cost, let alone a reasonable cost.

The
Pilot Study determined that it is too costly and way too disruptive
to go the "rip out and replace" route all of the major ODF
vendors were insisting upon.  This would be Sun, IBM, Novell and
Red Hat!

Meanwhile, the big ODF vendors all back ODF mandate
legislation that would force government IT to do the impossible -
implement ODF no matter what the cost, disruption or technical
difficulty.

Undaunted, Massachusetts waded into the
difficulties the big ODF vendors had with an ODF Plugin clone of the
OOXML MSOffice plugin.  The results were a disaster, with IBM
and Oracle fully supporting the da Vinci - InfoSet plugin proposal. 
And Sun in total opposition. 

Red Hat never got invited
to the IBM-Oracle da Vinci dance.  Novell was invited, but ended
up pursuing and hiring the OpenDocument Foundation's da Vinci
architect, Florian Reuter, as a key player of critial importance to
their secret deal with Microsoft!  With the failure of the Louis
Gutierrez da Vinci - InfoSet initiative, Louis resigned.  And
Novell hired Florian. 

On his first day at work for
Novell, the deal with Microsoft was announced, and Florian was
immediately tasked with the challenge of writing the Novell OOXML
Translator Plugin for OpenOffice. 

As it turns out, this
was a critical obligation for Novell as their part of the Microsoft
deal.  Most importantly, the OpenOffice plugin provides the
first non Microsoft implementation of OOXML.  Furthermore, the
Novell OOXML enabled version of OpenOffice is the default on 90% of
all LiNUX distributions, and has become the cornerstone of subsequent
Microsoft "interoperability - patent protection" deals with
Linspire, Xandros and TurboLinux.

So, what does this all
mean?  And what does it have to do with the March OASIS ODF
issues running in the background of Dana's three card Monte
article?

Faced with the Pilot Study results, and seeing up
close and personal the challenge of the two barriers to ODF
implementations, the OpenDocument Foundation had to re write our da
Vinci plugin so that it could match exactly the round trip fidelity
of the OOXML plugin for MSOffice.  Exactly!

Interestingly,
this can be accomplished through the advanced use of the ODF 1.0
"Conformance Clause".  This is Section 1.5 of the ODF
specification, and was written into the spec by the legendary reverse
engineer expert, Stellent's Phil Boutros.  The purpose of the
conformance clause is exactly to provide a means for converting the
billions of binary documents to ODF.  Using the Conformance
Clause, da Vinci could even perfect the round trip of these documents
throughout a MSOffice workgroup-workflow.

The problem is that
use of the Conformance Clause breaks interoperability with
OpenOffice!  OOo only partially implements and supports the
Conformance Clause; pathetically limiting support to paragraphs and
text spans.  None of the important structural issues of lists,
fields, tables, sections or page dynamics are supported.  And
that totally breaks any hope of interop with da Vinci MSOffice.

Note
that OpenOffice ODF also breaks interop with near every other
application implementation of ODF.  The difference is,
Massachusetts needed OpenOffice to be fully interoperable with da
Vinci MSOffice if they were to pull off an effective migration to
ODF.  Meaning, enabling Massachusetts to freely choose, without
business process compromise or disruption, their future purchase of
applications.

So an important consideration for Massachusetts
was Sun and OpenOffice support for what was then called the "ODF
1.2 iX "Interoperability Enhancement" Proposals.

If
there was ever to be interoperability with the kind of round trip
fidelity demanded by MSOffice bound business processes, the ODF
community had to rally around the ODF iX proposals, and support for
internal ODF Plugins that could match the OOXML plugin conversion
process.

On July 3rd, 2006, the first version of the ODF 1.2
iX proposal was submitted to Louis Gutierrez and his ITD staff. 
A version was circulated to OASIS ODF TC and Metadata XML/RDF
interested members.  On August 3rd, 2006, a second "composite"
version of ODF 1.2 iX was submitted.  And, on November 20th,
2006 a third "simplified composite" version was provided to
the OASIS ODF TC (the failure of ODf in Massachusetts having taken
place in October of 2006).

The first item on the November 20th
ODF 1.2 iX proposal list to be submitted as a formal proposal to the
OASIS ODF TC is known as the "List Override" Proposal. 
It was submitted by Florian Reuter, in his new role as the Novell
developer working on the OOXML Translator Plugin for OpenOffice!

Did you catch that? Florian Reuter,
the primary architect of the da Vinci plugin and now Novell developer
tasked with the Microsoft-Cleverage-Novell (the deal!) OOXML-ODF
Translator Project to perfect the OOXML Translator Plugin for
OpenOffice; posted and then submitted the first of the ODf 1.2 iX
“Interoperability Enhancement” Proposals as needed by the
MS-Cleverage Translator Project!

What happened next is truly tragic but
very revealing. Sun went after Florian and the iX proposals with an
unprecedented viciousness and personal vehemence that went way beyond
the total breakdown of the traditional ODf consensus process.
Interestingly though, Florian was able to present his “List
Enhancement Proposal” arguments in terms of a comparative
requirements table that argued the importance of compatibility,
interoperability and convergence (see above :).

Sun triumphed. The Novell proposal
went down to bitter defeat, and with it any illusion that the OASIS
ODf TC would ever support the compatibility, interoperability, and
convergence needs of ODf with respect to Microsoft documents,
applications and bound business processes. And, the OpenDocument
Foundation ended up getting booted out of OASIS.

Needless to say, the critically
important ODf 1.2 iX “Interoperability Enhancement” Proposals are
dead. And so is any hope of migrating existing MS documents,
applications and processes to ODf XML. The OASIS ODf TC has answered
the world's questions about which XML file format they should migrate
to, “OOXML or ODf?” The only option for those who are bound to
MSOffice is to use the OOXML plugin, and migrate to the highly
proprietary, application and platform dependent, even stack
dependent, OOXML.

Throw in the data binding Smart
Documents model, and OOXML-Smart Docs becomes the foundation of a 15
year lock in for desktop bound business processes that will no doubt
migrate to the Vista Stack center, the Exchange/SharePoint Hub. What
other choice is there if there is no way to migrate those same
MSOffice bound business processes to ODf Hubs? Thanks a pant load
Sun!

Which brings me to this finish. John
Bosak, founder of both the W3C XML and OASIS ODf, recently stated the
official
Sun position as that of supporting ISO approval of OOXML as an
international standard (DIS 29500). Sun's reason? Only OOXML is
compatible-interoperable
with the legacy of billions of MS binary documents and Microsoft
applications:

“We
wish to make it completely clear that we support DIS 29500 becoming
an ISO Standard and are in complete agreement with its stated
purposes of enabling interoperability among different implementations
and providing interoperable access to the legacy of Microsoft Office
documents.”

Read it and weep. Sun has opposed
every single compatibility-interoperability initiative proposed to
the OASIS ODf TC, beginning with the very
first meeting in December of 2002! This opposition continued
through the battle to implement ODf in Massachusetts, and on into the
Microsoft-Novell-Cleverage OOXML-ODF Translator project.

And now Sun claims that they must
support OOXML as an international standard after nearly five years of
blocking any and every effort to improve ODf interoperability with
Microsoft documents, applications and processes?

The world seriously needs to revisit
the 2004 “$2 Billion” deal between Sun and Microsoft because
we've been had. I believe there is evidence aplenty that Sun
traded ODf universal interoperability and the monopoly busting
threat of an OpenOffice lead community charge based on that
interoperability for a sweet sweet hardware deal. A hardware deal
guaranteeing Sun machines in Microsoft data centers coupled with
Redmond support anywhere enterprise and governments needed superior
high performance integration with Microsoft applications.

It's a deal that saved Sun. It's also
a deal that demanded Sun cheat everyone else in the ODf Community out
of their sincere efforts to create a truly universal file format.

Yeah, we've been had. But this is
hardly the end of it.

~ge~