Skip to main content

Home/ Document Wars/ Group items tagged subset

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
Gary Edwards

XML Production Workflows? Start with the Web and XHTML - 0 views

quod.lib.umich.edu/...n-workflows-start-with-the-web

OpenWeb XML XHTML-CSS HTML5-CSS3-JavaScript IDML

shared by Gary Edwards on 07 Nov 13 - No Cached
  • Challenges: Some Ugly Truths The challenges of building—and living with—an XML workflow are clear enough. The return on investment is a long-term proposition. Regardless of the benefits XML may provide, the starting reality is that it represents a very different way of doing things than the one we are familiar with. The Word Processing and Desktop Publishing paradigm, based on the promise of onscreen, WYSIWYG layout, is so dominant as to be practically inescapable. It has proven really hard to get from here to there, no matter how attractive XML might be on paper. A considerable amount of organizational effort and labour must be expended up front in order to realize the benefits. This is why XML is often referred to as an “investment”: you sink a bunch of time and money up front, and realize the benefits—greater flexibility, multiple output options, searching and indexing, and general futureproofing—later, over the long haul. It is not a short-term return proposition. And, of course, the returns you are able to realize from your XML investment are commensurate with what you put in up front: fine-grained, semantically rich tagging is going to give you more potential for searchability and recombination than a looser, more general-purpose approach, but it sure costs more. For instance, the Text Encoding Initiative (TEI) is the grand example of pouring enormous amounts of energy into the up-front tagging, with a very open-ended set of possibilities down the line. TEI helpfully defines a level to which most of us do not have to aspire.[5] But understanding this on a theoretical level is only part of the challenge. There are many practical issues that must be addressed. Software and labour are two of the most critical. How do you get the content into XML in the first place? Unfortunately, despite two decades of people doing SGML and XML, this remains an ugly question.
  • Practical Challenges In 2009, there is still no truly likeable—let alone standard—editing and authoring software for XML. For many (myself included), the high-water mark here was Adobe’s FrameMaker, substantially developed by the late 1990s. With no substantial market for it, it is relegated today mostly to the tech writing industry, unavailable for the Mac, and just far enough afield from the kinds of tools we use today that its adoption represents a significant hurdle. And FrameMaker was the best of the breed; most of the other software in decent circulation are programmers’ tools—the sort of things that, as Michael Tamblyn pointed out, encourage editors to drink at their desks. The labour question represents a stumbling block as well. The skill-sets and mind-sets that effective XML editors need have limited overlap with those needed by literary and more traditional production editors. The need to think of documents as machine-readable databases is not something that comes naturally to folks steeped in literary culture. In combination with the sheer time and effort that rich tagging requires, many publishers simply outsource the tagging to India, drawing a division of labour that spans oceans, to put it mildly. Once you have XML content, then what do you do with it? How do you produce books from it? Presumably, you need to be able to produce print output as well as digital formats. But while the latter are new enough to be generally XML-friendly (e-book formats being largely XML based, for instance), there aren’t any straightforward, standard ways of moving XML content into the kind of print production environments we are used to seeing. This isn’t to say that there aren’t ways of getting print—even very high-quality print—output from XML, just that most of them involve replacing your prepress staff with Java programmers.
  • Why does this have to be so hard? It’s not that XML is new, or immature, or untested. Remember that the basics have been around, and in production, since the early 1980s at least. But we have to take account of a substantial and long-running cultural disconnect between traditional editorial and production processes (the ones most of us know intimately) and the ways computing people have approached things. Interestingly, this cultural divide looked rather different in the 1970s, when publishers were looking at how to move to digital typesetting. Back then, printers and software developers could speak the same language. But that was before the ascendancy of the Desktop Publishing paradigm, which computerized the publishing industry while at the same time isolating it culturally. Those of us who learned how to do things the Quark way or the Adobe way had little in common with people who programmed databases or document-management systems. Desktop publishing technology isolated us in a smooth, self-contained universe of toolbars, grid lines, and laser proofs. So, now that the reasons to get with this program, XML, loom large, how can we bridge this long-standing divide?
  • ...44 more annotations...
  • Using the Web as a Production Platform The answer, I think, is right in front of you. The bridge is the Web, a technology and platform that is fundamentally based on XML, and which many publishers are by now comfortably familiar with. Perhaps not entirely comfortably, but at least most publishers are already working with the Web; they already either know or have on staff people who understand it and can work with it. The foundation of our argument is this: rather than looking at jumping to XML in its full, industrial complexity, which seems to be what the O'Reilly-backed StartWithXML initiative[6] is suggesting, publishers instead leverage existing tools and technologies—starting with the Web—as a means of getting XML workflows in place. This means making small investments and working with known tools rather than spending tens of thousands of dollars on XML software and rarefied consultants. It means re-thinking how the existing pieces of the production toolchain fit together; re-thinking the existing roles of software components already in use. It means, fundamentally, taking the Web seriously as a content platform, rather than thinking of it as something you need to get content out to, somehow. If nothing else, the Web represents an opportunity to think about editorial and production from outside the shrink-wrapped Desktop Publishing paradigm.
  • Is the Web made of Real XML? At this point some predictable objections can be heard: wait a moment, the Web isn’t really made out of XML; the HTML that makes up most of the Web is at best the bastard child of SGML, and it is far too flaky/unstructured/underpowered to be taken seriously. We counter by arguing that although HTML on the Web exists in a staggering array of different incarnations, and that the majority of it is indeed an unstructured mess, this does not undermine the general principle that basic, ubiquitous Web technologies can make a solid platform for content management, editorial process, and production workflow.
  • With the advent of a published XML standard in the late 1990s came the W3C’s adoption of XHTML: the realization of the Web’s native content markup as a proper XML document type. Today, its acceptance is almost ubiquitous, even while the majority of actual content out there may not be strictly conforming. The more important point is that most contemporary Web software, from browsers to authoring tools to content management systems (from blogs to enterprise systems), are capable of working with clean, valid XHTML. Or, to put the argument the other way around, clean, valid XHTML content plays absolutely seamlessly with everything else on the Web.[7]
  • The objection which follows, then, will be that even if we grant that XHTML is a real XML document type, that it is underpowered for “serious” content because it is almost entirely presentation (formatting) oriented; it lacks any semantic depth. In XHTML, a paragraph is a paragraph is a paragraph, as opposed to a section or an epigraph or a summary.
  • n contrast, more “serious” XML document types like DocBook[8] or DITA-derived schemas[9] are capable of making semantic distinctions about content chunks at a fine level of granularity and with a high degree of specificity.
  • So there is an argument for recalling the 80:20 rule here. If XHTML can provide 80% of the value with just 20% of the investment, then what exactly is the business case for spending the other 80% to achieve that last 20% of value? We suspect the ratio is actually quite a bit steeper than 80:20 for most publishers.
  • IDML is a well thought-out XML standard that achieves two very different goals simultaneously: it preserves all of the information that InDesign needs to do what it does; and it is broken up in a way that makes it possible for mere mortals (or at least our Master of Publishing students) to work with it.
  • XHTML, on the other hand, is supported by a vast array of quotidian software, starting with the ubiquitous Web browser. For this very reason, XHTML is in fact employed as a component part of several more specialized document types (ONIX and ePub among them).
  • Why re-invent a general-purpose prose representation when XHTML already does the job?
  • It is worth pausing for a moment to consider the role of XHTML in the ePub standard for ebook content. An ePub file is, anatomically, a simply disguised zip archive. Inside the zip archive are a few standard component parts: there are specialized files that declare metadata about the book, and about the format of the book. And then there is the book’s content, represented in XHTML. An ePub book is a Web page in a wrapper.
  • To sum up the general argument: the Web as it already exists presents incredible value to publishers, as a platform for doing XML content management with existing (and often free) tools, and without having to go blindly into the unknown. At this point, we can offer a few design guidelines: prefer existing and/or ubiquitous tools over specialized ones wherever possible; prefer free software over proprietary systems where possible; prefer simple tools controlled and coordinated by human beings over fully automated (and therefore complex) systems; play to our strengths: use Web software for storing and managing content, use layout software for layout, and keep editors and production people in charge of their own domains.
  • Putting the Pieces Together: A Prototype
  • At the SFU Master of Publishing Program, we have been chipping away at this general line of thinking for a few years. Over that time, Web content management systems have been getting more and more sophisticated, all the while getting more streamlined and easier to use. (NB: if you have a blog, you have a Web content management system.) The Web is beginning to be recognized as a writing and editing environment used by millions of people. And the ways in which content is represented, stored, and exchanged online have become increasingly robust and standardized.
  • The missing piece of the puzzle has been print production: how can we move content from its malleable, fluid form on line into the kind of high-quality print production environments we’ve come to expect after two decades of Desktop Publishing?
  • Anyone who has tried to print Web content knows that the existing methods leave much to be desired (hyphenation and justification, for starters). In the absence of decent tools for this, most publishers quite naturally think of producing the print content first, and then think about how to get material onto the Web for various purposes. So we tend to export from Word, or from Adobe, as something of an afterthought.
  • While this sort of works, it isn’t elegant, and it completely ignores the considerable advantages of Web-based content management.
  • Content managed online is stored in one central location, accessible simultaneously to everyone in your firm, available anywhere you have an Internet connection, and usually exists in a much more fluid format than Word files. If only we could manage the editorial flow online, and then go to print formats at the end, instead of the other way around. At SFU, we made several attempts to make this work by way of the supposed “XML import” capabilities of various Desktop Publishing tools, without much success.[12]
  • In the winter of 2009, Adobe solved this part of the problem for us with the introduction of its Creative Suite 4. What CS4 offers is the option of a complete XML representation of an InDesign document: what Adobe calls IDML (InDesign Markup Language).
  • The IDML file format is—like ePub—a simply disguised zip archive that, when unpacked, reveals a cluster of XML files that represent all the different facets of an InDesign document: layout spreads, master pages, defined styles, colours, and of course, the content.
  • What this represented to us in concrete terms was the ability to take Web-based content and move it into InDesign in a straightforward way, thus bridging Web and print production environments using existing tools and skillsets, with a little added help from free software.
  • Such a workflow—beginning with the Web and exporting to print—is surely more in line with the way we will do business in the 21st century, where the Web is the default platform for reaching audiences, developing content, and putting the pieces together. It is time, we suggest, for publishers to re-orient their operations and start with the Web.
  • We would take clean XHTML content, transform it to IDML-marked content, and merge that with nicely designed templates in InDesign.
  • The result is an almost push-button publication workflow, which results in a nice, familiar InDesign document that fits straight into the way publishers actually do production.
  • Tracing the steps To begin with, we worked backwards, moving the book content back to clean XHTML.
  • The simplest method for this conversion—and if you want to create Web content, this is an excellent route—was to use Adobe’s “Export to Digital Editions” option, which creates an ePub file.
  • Recall that ePub is just XHTML in a wrapper, so within the ePub file was a relatively clean XHTML document. It was somewhat cleaner (that is, the XHTML tagging was simpler and less cluttered) than InDesign’s other Web-oriented exports, possibly because Digital Editions is a well understood target, compared with somebody’s website.
  • In order to achieve our target of clean XHTML, we needed to do some editing; the XHTML produced by InDesign’s “Digital Editions” export was presentation-oriented. For instance, bulleted list items were tagged as paragraphs, with a class attribute identifying them as list items. Using the search-and-replace function, we converted such structures to proper XHTML list and list-item elements. Our guiding principle was to make the XHTML as straightforward as possible, not dependent on any particular software to interpret it.
  • We broke the book’s content into individual chapter files; each chapter could then carry its own basic metadata, and the pages conveniently fit our Web content management system (which is actually just a wiki). We assembled a dynamically generated table of contents for the 12 chapters, and created a cover page. Essentially, the book was entirely Web-based at this point.
  • When the book chapters are viewed online, they are formatted via a CSS2 stylesheet that defines a main column for content as well as dedicating screen real estate for navigational elements. We then created a second template to render the content for exporting; this was essentially a bare-bones version of the book with no navigation and minimal styling. Pages (or even the entire book) can be exported (via the “Save As...” function in a Web browser) for use in either print production or ebook conversion. At this point, we required no skills beyond those of any decent Web designer.
  • Integrating with CS4 for Print Adobe’s IDML language defines elements specific to InDesign; there is nothing in the language that looks remotely like XHTML. So a mechanical transformation step is needed to convert the XHTML content into something InDesign can use. This is not as hard as it might seem.
  • Both XHTML and IDML are composed of straightforward, well-documented structures, and so transformation from one to the other is, as they say, “trivial.” We chose to use XSLT (Extensible Stylesheet Language Transforms) to do the work. XSLT is part of the overall XML specification, and thus is very well supported in a wide variety of tools. Our prototype used a scripting engine called xsltproc, a nearly ubiquitous piece of software that we found already installed as part of Mac OS X (contemporary Linux distributions also have this as a standard tool), though any XSLT processor would work.
  • In other words, we don’t need to buy InCopy, because we just replaced it with the Web. Our wiki is now plugged directly into our InDesign layout. It even automatically updates the InDesign document when the content changes. Credit is due at this point to Adobe: this integration is possible because of the open file format in the Creative Suite 4.
  • We wrote an XSLT transformation script[18] that converted the XHTML content from the Web into an InCopy ICML file. The script itself is less than 500 lines long, and was written and debugged over a period of about a week by amateurs (again, the people named at the start of this article). The script runs in a couple of seconds, and the resulting .icml file can then be “placed” directly into an InDesign template. The ICML file references an InDesign stylesheet, so the template file can be set up with a house-styled layout, master pages, and stylesheet definitions for paragraphs and character ranges.
  • The result is very simple and easy to use. Our demonstration requires that a production editor run the XSLT transformation script manually, but there is no reason why this couldn’t be built directly into the Web content management system so that exporting the content to print ran the transformation automatically. The resulting file would then be “placed” in InDesign and proofed.
  • It should be noted that the Book Publishing 1 proof-of-concept was artificially complex; we began with a book laid out in InDesign and ended up with a look-alike book laid out in InDesign. But next time—for instance, when we publish Book Publishing 2—we can begin the process with the content on the Web, and keep it there throughout the editorial process. The book’s content could potentially be written and edited entirely online, as Web content, and then automatically poured into an InDesign template at proof time. “Just in time,” as they say. This represents an entirely new way of thinking of book production. With a Web-first orientation, it makes little sense to think of the book as “in print” or “out of print”—the book is simply available, in the first place online; in the second place in derivative digital formats; and third, but really not much more difficult, in print-ready format, via the usual InDesign CS print production system publishers are already familiar with.
  • Creating Ebook Files Creating electronic versions from XHTML source is vastly simpler than trying to generate these out of the existing print process. The ePub version is extremely easy to generate; so is online marketing copy or excerpts for the Web, since the content begins life Web-native.
  • Since an ePub file is essentially XHTML content in a special wrapper, all that is required is that we properly “wrap” our XHTML content. Ideally, the content in an ePub file is broken into chapters (as ours was) and a table of contents file is generated in order to allow easy navigation within an ebook reader. We used Julian Smart’s free tool eCub[19] to simply and automatically generate the ePub wrapper and the table of contents. The only custom development we did was to create a CSS stylesheet for the ebook so that headings and paragraph indents looked the way we wanted. Starting with XHTML content, creating ePub is almost too easy.
  • today, we are able to put the process together using nothing but standard, relatively ubiquitous Web tools: the Web itself as an editing and content management environment, standard Web scripting tools for the conversion process, and the well-documented IDML file format to integrate the layout tool.
  • Our project demonstrates that Web technologies are indeed good enough to use in an XML-oriented workflow; more specialized and expensive options are not necessarily required. For massive-scale enterprise publishing, this approach may not offer enough flexibility, and the challenge of adding and extracting extra semantic richness may prove more trouble than it's worth.
  • But for smaller firms who are looking at the straightforward benefits of XML-based processes—single source publishing, online content and workflow management, open and accessible archive formats, greater online discoverability—here is a way forward.
  • Rather than a public-facing website, our system relies on the Web as a content management platform—of course a public face could easily be added.
  • The final piece of our puzzle, the ability to integrate print production, was made possible by Adobe's release of InDesign with an open XML file format. Since the Web's XHTML is also XML, is can be easily and confidently transformed to the InDesign format.
  • Furthermore, just to get technical for a moment, XHTML is extensible in a fairly straightforward way, through the common “class” attribute on each element. Web developers have long leveraged this kind of extensibility in the elaboration of “microformats” for semantic-web applications.[10] There is no reason why publishers shouldn’t think to use XHTML’s simple extensibility in a similar way for their own ends.
  • Using the Web as a Production Platform
  • Gary Edwards on 07 Nov 13
     
    I was looking for an answer to a problem Marbux had presented, and found this interesting article.  The issue was that of the upcoming conversion of the Note Case Pro (NCP) layout engine to the WebKit layout engine, and what to do about the NCP document format. My initial reaction was to encode the legacy NCP document format in XML, and run an XSLT to a universal pivot format like TEI-XML.  From there, the TEI-XML community would provide all the XSLT transformation routines for conversion to ODF, OOXML, XHTML, ePUB and HTML/CSS. Researching the problems one might encounter with this approach, I found this article.  Fascinating stuff. My take away is that TEI-XML would not be as effective a "universal pivot point" as XHTML.  Or perhaps, if NCP really wants to get aggressive; IDML - InDesign Markup Language. The important point though is that XHTML is a browser specific version of XML, and compatible with the Web Kit layout engine Miro wants to move NCP to. The concept of encoding an existing application-specific format in XML has been around since 1998, when XML was first introduced as a W3C standard, a "structured" subset of SGML. (HTML is also a subset of SGML). The multiplatform StarOffice productivity suite became "OpenOffice" when Sun purchased the company in 1998, and open sourced the code base. The OpenOffice developer team came out with a XML encoding of their existing document formats in 2000. The application specific encoding became an OASIS document format standard proposal in 2002 - also known as ODF. Microsoft followed OpenOffice with a XML encoding of their application-specific binary document formats, known as OOXML. Encoding the existing NCP format in XML, specifically targeting XHTML as a "universal pivot point", would put the NCP Outliner in the Web editor category, without breaking backwards compatibility. The trick is in the XSLT conversion process. But I think that is something much easier to handle then trying to
  • Gary Edwards on 07 Nov 13
     
    I was looking for an answer to a problem Marbux had presented, and found this interesting article.  The issue was that of the upcoming conversion of the Note Case Pro (NCP) layout engine to the WebKit layout engine, and what to do about the NCP document format. My initial reaction was to encode the legacy NCP document format in XML, and run an XSLT to a universal pivot format like TEI-XML.  From there, the TEI-XML community would provide all the XSLT transformation routines for conversion to ODF, OOXML, XHTML, ePUB and HTML/CSS. Researching the problems one might encounter with this approach, I found this article.  Fascinating stuff. My take away is that TEI-XML would not be as effective a "universal pivot point" as XHTML.  Or perhaps, if NCP really wants to get aggressive; IDML - InDesign Markup Language. As an after thought, i was thinking that an alternative title to this article might have been, "Working with Web as the Center of Everything".
  • Gary Edwards on 15 Nov 13
     
    I was looking for an answer to a problem Marbux had presented, and found this interesting article.  The issue was that of the upcoming conversion of the Note Case Pro (NCP) layout engine to the WebKit layout engine, and what to do about the NCP document format. My initial reaction was to encode the legacy NCP document format in XML, and run an XSLT to a universal pivot format like TEI-XML.  From there, the TEI-XML community would provide all the XSLT transformation routines for conversion to ODF, OOXML, XHTML, ePUB and HTML/CSS. Researching the problems one might encounter with this approach, I found this article.  Fascinating stuff. My take away is that TEI-XML would not be as effective a "universal pivot point" as XHTML.  Or perhaps, if NCP really wants to get aggressive; IDML - InDesign Markup Language. The important point though is that XHTML is a browser specific version of XML, and compatible with the Web Kit layout engine Miro wants to move NCP to. The concept of encoding an existing application-specific format in XML has been around since 1998, when XML was first introduced as a W3C standard, a "structured" subset of SGML. (HTML is also a subset of SGML). The multiplatform StarOffice productivity suite became "OpenOffice" when Sun purchased the company in 1998, and open sourced the code base. The OpenOffice developer team came out with a XML encoding of their existing document formats in 2000. The application specific encoding became an OASIS document format standard proposal in 2002 - also known as ODF. Microsoft followed OpenOffice with a XML encoding of their application-specific binary document formats, known as OOXML. Encoding the existing NCP format in XML, specifically targeting XHTML as a "universal pivot point", would put the NCP Outliner in the Web editor category, without breaking backwards compatibility. The trick is in the XSLT conversion process. But I think that is something much easier to handle then trying to
Gary Edwards

Interoperability Enhancement Proposal: Suggested ODF1.2 items - 0 views

lists.oasis-open.org/...msg00048.html

foundation interop ix odf odfix opendocument subset

shared by Gary Edwards on 20 Jan 08 - Cached
  • Subject: Suggested ODF1.2 items From: "Florian Reuter" <freuter@novell.com> To: <office@lists.oasis-open.org> Date: Mon, 20 Nov 2006 17:03:24 +0100
  • Gary Edwards on 20 Jan 08
     
    This is the fifth of the six major iX - interoperability enhancement proposals submitted to the OASIS ODF TC - SC between July 2006 and February of 2007. This particular iX proposal lead to the "List Enhancement Proposal" donnybrook that consumed the OASIS ODF TC for the next six months, ending with the OpenDocument Foundation being booted out of OASIS in May of 2007. The six iX proposals were all different approaches to the same basic problem: ODF was not desinged to be interoperable with MSOffice documents, applications or bound processes. The proposals come out of the OpenDocument' Foundation's efforts to save ODF in Massachusetts. ODF iX repressents a subset of ODF designed to grealty improve compatibility with MS binary and XML formats. With the ODF iX subset, the da Vinci plug-in would be able to convert the billions of MSOffice binary and xml documents with a very high level of fidelity, and do so within the bounds of "round trip" business processes. The most basic iX approach was to add five generic elements to the existing ODF specification. The five generic elements would cover lists, tables, fields, sections, and page dynamics (breaks). It is a well known fact that these five areas of incompatibility between OpenOffice ODF and MSOffice binaries represent 95% of all conversion fidelity problems. MSOffice has one way of implementing lists, and, OpenOffice has another. These application specific implementation models are irreconcilably different. It's also true that the applicaiton specific implementation models are directly reflected in each file format. So applications implementing ODF must also implement the OpenOffice model for lists, fields, tables, sections and page dynamics-page positioning if they are to have any meaningful measure of exchange fidelity. Perhaps the best of the iX approaches was that based on the innovative use of metadata to describe presentation-layout attributes.
Gary Edwards

Brian Jones: Open XML Formats : Mapping documents in the binary format (.doc; .xls; .pp... - 0 views

blogs.msdn.com/...pt-to-the-open-xml-format.aspx

burtongroup cdf css ibm iso microsoft odf okelly ooxml opendocument openxml sun w3c xhtml

shared by Gary Edwards on 20 Jan 08 - Cached
  • The second issue we had feedback on was an interest in the mapping from the binary formats into the Open XML formats. The thought here was that the most effective way to help people with this was to create an open source translation project to allow binary documents (.doc; .xls; .ppt) to be translated into Open XML. So we proposed the creation of a new open source project that would map a document written using the legacy binary formats to the Open XML formats. TC45 liked this suggestion, and here was the TC45 response to the national body comments: We believe that Interoperability between applications conforming to DIS 29500 is established at the Office Open XML-to- Office Open XML file construct level only.
    • Gary Edwards
      Gary Edwards on 20 Jan 08
       
      And here i was betting that the blueprints to the secret binaries would be released the weekend before the September 2nd, 2007 ISO vote on OOXML! Looks like Microsoft saved the move for when they really had to use it; jus tweeks before the February ISO Ballot Resolution Meetings set to resolve the Sept 2nd issues. The truth is that years of reverse engineering have depleted the value of keeping the binary blueprints secret. It's true that interoperability with MSOffice in the past was near entirely dependent on understanding the secret binaries. Today however, with the rapid emergence of the Exchange/SharePoint juggernaught, interop with MSOffice is no longer the core issue. Now we have to compete with E/S, and it is the E/S interfaces, protocols and document API's and dependencies tha tmust be reverse engineered. The E/S juggernaught is now surging to 70% or more of the market. These near monopoly levels of market penetration is game changing. One must reverse engineer or license the .NET libraries to crack the interop problem. And this time it's not just MSOffice. Today one must crack into the MS Stack whose core is tha tof MSOffice <> E/S. So why not release the secret binary blueprints? If that's the cost of getting the application, platform and vendor specific OOXML through ISO, then it's a small price to pay for your own international standard.
  • Gary Edwards on 20 Jan 08
     
    Well well well. We knew that IBM had access to the secret binary blueprints back in 2006. Now we know that Sun ALSO had access!
    And why is this important? In June of 2006, Massachusetts CIO Louis Gutierrez asked the OpenDocument Foundation's da Vinci Group to work with IBM on developing the da Vinci ODF plug-in clone of Microsoft's OOXML Compatibility Pack plug-in. When we met with IBM they were insistent that the only way OASIS ODF could establish sufficient compatibility with MSOffice and the billions of binary documents would be to have the secret blueprints open.
    Even after we explained to IBM that da Vinci uses the same internal conversion process that the OOXML plug-in used to convert binaries, IBM continued to insist that opening up the secret binaries was a primary objective of the OASIS ODF community.
    For sure this was important to IBM and Sun, but the secret binaries were of no use to us. da Vinci didn't need them. What da Vinci needed instead was a subset of ODF designed for the conversion of those billions of binary documents! A need opposed by Sun.
    Sun of course would spend the next year developing their own ODF plug-in for MSOffice. But here's the thing: it turns out that Sun had complete access to the secret binary blueprints dating back to 2006!!!!!!
    So even though IBM and Sun have had access to the blueprints since 2006, they have been unable to provide effective conversions to ODF!
    This validates a point the da Vinci group has been trying to make since June of 2006: the problem of perfecting a high fidelity conversion between the billions of binaries and ODF has nothing to do with access to the secret binary blueprints. The real issue is that ODF was NOT designed for the conversion of those binary documents.
    It is true that one could eXtend ODF to achieve the needed compatibility. But one has to be very careful before taking this ro
Gary Edwards

Independent study advises IT planners to go OOXML | All about Microsoft | ZDNet.com - 0 views

blogs.zdnet.com/microsoft

burtongorup cdf css garyedwards maryjo odf okelly ooxml w3c xhtml

shared by Gary Edwards on 20 Jan 08 - Cached
  • “ODF represents laudable design and standards work. It’s a clean and useful design, but it’s appropriate mostly for relatively unusual scenarios in which full Microsoft Office file format fidelity isn’t a requirement. Overall, ODF addresses only a subset of what most organizations do with productivity applications today.” The report continues: “ODF is insufficient for complex real-world enterprise requirements, and it is indirectly controlled by Sun Microsystems, despite also being an ISO standard. It’s possible that IBM, Novell, and other vendors may be able to put ODF on a more customer-oriented trajectory in the future and more completely integrate it with the W3C content model, but for now ODF should be seen as more of an anti-Microsoft political statement than an objective technology selection.”
    • Gary Edwards
      Gary Edwards on 20 Jan 08
       
      Mary Jo takes on the recently released Burton Group Report comparing OOXML and ODF. Peter O'Kelly, one of the Burton Group authors, once famously said, "ODF is a great format if you live in an alternative universe where MSOffice doesn't exist!" This observation speaks to the core problem facing ODF and those who seek to implement the ODF standard: ODF was not designed for the conversion of MSOffice documents. Nor was ODF designed to work with MSOffice applications. Another way of saying this is to state that ODF was not designed to be interoperable with MSOffice documents, applications and bound processes. The truth is that ODF was designed for OpenOffice/StarOffice. It is an application specific format. Both OOXML and ODF do a good job of separating content from presentation (style). The problem is that the presentation - layout layers of both ODF and OOXML remains bound to specific applications producing it. While the content layers are entirely portable and can be exchanged without information loss, the presentation layers can not. Microsoft makes no bones about the application specific design and purpose of OOXML. It's stated right in the Ecma 376 charter that OOXML was designed to be compatible with MSOffice and the billions of binary documents in MSOffice specific binary formats. The situation however is much more confusing with ODF. ODF is often promoted as being application, platform and vendor independent. After five years of development though, the OASIS ODF TC has been unable to strip ODF of it's OpenOffice/StarOffice specific aspects. ODF 1.0 - ISO 26300 had three areas that were under specified; meaning these areas were described in syntax only, and lacked the full semantics demanded by interoperable implementations. Only OpenOffice and StarOffice code base applications are able to exchange documents with an acceptable fidelity. The three under specified areas of ODF are: Lists (numbered), F
Gary Edwards

Harmonizing ODF and OOXML using NameSpaces | Tim Bray's Thought Experiment - 0 views

www.tbray.org/...Office-XML

bray florian-reuter namespaces odf ooxml opendocument openxml

shared by Gary Edwards on 07 Mar 08 - Cached
  • First, what if Microsoft really is doing the right thing? Second, how can we avoid having two incompatible file formats? [Update: There’s been a lot of reaction to this piece, and I addressed some of those points here.]
  • On the technology side, the two formats are really more alike than they are different. But, there are differences: O12X’s design center, Microsoft has said repeatedly, is capturing the exact semantics of the billions of existing Microsoft Office documents. ODF’s design center is general-purpose reusability, and leveraging existing standards like SVG and MathML and so on.
    • Gary Edwards
      Gary Edwards on 07 Mar 08
       
      OOXML, or to put it more accurately "O12X" as Tim suggests, is designed to capture the exact semantics of MSOffice 12. In fact, OOXML is an XML encoding of the MSOffice 12 in-memory-binary-representation dump. When it comes to representing older versions of MSOffice documents, OOXML must use legacy compatibility settings" to capture the semantics. And it's not an exacting science to say the least. The thing is, OpenOffice ODF uses the same technique resulting in application specific ODF documents with over 150 un docuemnted, unspecified "compatibility settings". After years of requests from the OASIS ODF Technical Committee to document these application specific settings, Sun has yet to provide any kind of response. And this kills ODF interoperability. Especially concerning KOffice. There is also the issue of OASIS ODF high-jacked namespaces. When ODF applications reference a namespace, the actual URL is high-jacked with http://oasis-open.org/???? replacing the proper namespace of http://W3C.org/???? This high-jacking impacts the oDF reuse of important W3C technologies such as XForms, SVG, MathML, and SMiL. So where's the problem you ask? Well, when a developer imports or tries to process an OpenOffice ODF document, they rely on say the W3C XForms specification for their understanding. OpenOffice however seriously constrains the implementation of XForms, SVG, MathML, RDFa and RDF/XML. This should be reflected in the new namespace. However, if you follow the high-jacked URL, you'll find that there is nothing there. There is no specification describing how OpenOffice implements XForms in ODF! This breaks developer libraries, breaks ODF interoperability between ODF applications, and, offends the W3C to no end. So i think it might be fair to say that at this point, neither ODF or OOXML have come close to fulfilling their design objectives.
  • The capabilities of ODF and O12X are essentially identical for all this basic stuff. So why in the flaming hell does the world need two incompatible formats to express it? The answer, obviously, is, “it doesn’t”.
    • Gary Edwards
      Gary Edwards on 07 Mar 08
       
      Exactly!! Except for one thing that Tim misses: the presentation layers of both ODF and OOXML are application specific. It is also the application specific nature of OpenOffice ODF presnetation layer that prevents interoperability with KOffice ODF! There is near zero interop between OpenOffice and KOffice, and KOffice has been a contributing member of the OASIS ODF TC for FIVE YEARS! It's the presentation layer Tim. ODF and OOXML are application specific formats because their presentation layers are woefully applicaiton specific and entirely reflective of each applications layout engine and feature set implementation model. I often imagine what ODF would be like if back in 2001, Sun had chosen to implement CSS as the OpenOffice presentation layer instead of the quirky but innovative, and 100% application specific automatic-styles presentation layer we now see in ODF. Unlike ODF's "automatic-styles", CSS is a totally application independent presentation model prized exactly for it's universal interoperability!
  • ...1 more annotation...
  • The ideal outcome would be a common shared office-XML dialect for the basics—and it should be ODF (or a subset), since that’s been designed and debugged—then another extended vocabulary to support Microsoft features , whether they’re cool new whizzy features or mouldy old legacy features (XML Namespaces are designed to support exactly this kind of thing). That way, if you stayed with the basic stuff you’d never need to worry about software lock-in; the difference between portable and proprietary would be crystal-clear. And, for the basic stuff that everybody uses, there’d be only one set of tags. This outcome is technically feasible. Who could possibly be against it?
    • Gary Edwards
      Gary Edwards on 07 Mar 08
       
      Bingo! ODF and OOXML should strip off the application specific complexities and seek a neutral generic XML representation of basic document structures common to ALL documents. Then, use the XML NameSpace mechanism to extend (with proper descriptions) the generic to include the volumes of application specific features that now fill each format. One thing i disagree with Tim about. And that's that the interop of ODF and OOXML is hopelessly broken. The OpenDocument Foundation tried for over a year to close the compatibility gap between ODF and MSOffice binary - xml documents. The OASIS ODF TC would have none of it. IBM and Sun are set on a harsh course of highly disruptive and costly rip-out-and-replace of MSOffice based on government mandates for ODF. There is no offer of compromise to be had. On the Microsoft side, even if they did want to compromise (a big IF), there is that problem of over 550 million MSOffice workgroup-workflow desktops to contend with. The thing is, the only way to harmonize, merge, convert or translate between two application specific formats is to actually harmonize the applications themselves. While the generic subset is a worthy goal, the process would be fraught with real world concerns that the existing application workflows are not disrupted. My proposal? Demand that ODF and OOXML application vendors provide format options for PDF, and the W3C's family of formats: (X)HTML5, (X)HTML - CSS, and CDF (XHTML-CSS-XForms-SVG-SMil-MathML). That will do it. We might never see the quality of interoperability we had hoped for in a desktop application to application scenario. But we can and should fully expect high quality interop at the higher level of the Web. You can convert an application specific format to a generic like CDF. By setting up conversion channels to the same CDF profile within MSOffice, OpenOffice, KOffice, Symphony, and Google Docs, we can achieve the universal interoperabil
Gary Edwards

ODF - the state of play - The future of ODF under OASIS, now that the... - 1 views

www.it-director.com/...-that-the-standards-war-is-won

ODF msooxml

shared by Gary Edwards on 20 Nov 15 - No Cached
  • Gary Edwards on 20 Nov 15
     
    "ODF - open document format - is an open, XML-based rich document format that has been adopted as the standard for exchanging information in documents (spreadsheets, charts, presentations and word processing documents), by many governments and other organisations (see, for example, here), including the UK Government. This is despite strong opposition by Microsoft; but I have seen Microsoft's proposed "open XML" standard and, frankly, it is huge and horrid (in the word of standards, these go together). If I remember correctly, the early draft I saw even incorporated recognition of early Excel leap-year bugs into the standard. ODF is now a pukka ISO standard, maintained by OASIS, under the proud banner: "The future is interoperability". My personal thoughts, below, are prompted by an ODF session at ApacheCon Core titled "Beyond OpenOffice: The State of the ODF Ecosystem" held by Louis Suárez-Potts (community strategist for Age of Peers, his own consultancy, and the Community Manager for OpenOffice.org, from 2000 to 2011), and attended by very few delegates - perhaps a sign of current level of interest in ODF within the Apache community. Nevertheless, and I am talking about the ODF standard here, not Apache Open Office (which is currently my office software of choice) or its Libre Office fork (which seems to be where the excitement, such as it is, is, for now), the standards battle, or one battle, has been won; we have a useful Open Document Format, standardised by a recognised and mature standards organisation, and even Microsoft Office supports it. That's good. So what could be the problem? Well, I don't care whether I use ODF from Open Office, Libre Office or even Office 365, I just want to be sure that everyone else can read my ODF documents (with a .odt, .ods or .odp extension, for text, presentation or spreadsheet, respectively), with whatever software they like; and that they'll either see exactly the functionality and formatting I see; or a well defined (an
Gary Edwards

Comments on 'On the Office format wars' - 0 views

www.regdeveloper.co.uk/...comments

iso odf officeopenxml ooxml opendocument xml

shared by Gary Edwards on 12 Dec 07 - Cached
  • A fatal flaw in your analysis By Marbux Posted Saturday 21st April 2007 08:15&nbsp;GMT Your analysis contains a fatal flaw, Martin. That is your belief that adequate Microsoft XML &lt;&gt; OpenDocument translators will be available. In fact, all of the translators suck mightily and there is no prospect at all of them being perfected. The major problems are: (i) that Microsoft's XML formats seem deliberately designed to thwart their parsing with XPath, which is essential to XML transformations; (ii) that Microsoft's "XML" file formats include binary blobs, bitmasks, and multiple Windows and Microsoft dependendencies, all of which defy XML transformations; and (iii) OpenDocument assumes a richer page layout engine than Microsoft Word provides, so while DOCX can be completely mapped to ODT it is impossible to fully map in the other direction without declaring an MS Office interoperability subset of OpenDocument and ODF applications implementing a compatibility mode with reduced features. (That is more than somewhat ironic, given Microsoft's spin that it couldn't implement all of its features in OpenDocument. In fact, the exact opposite is true.) In fact, Steve Ballmer is on record as saying that the developers of the Novell-Microsoft-Clever Age plug-ins will not even attempt to achieve full fidelity file translations between the two formats. http://www.eweek.com/article2/0,1895,2050848,00.asp?kc=EWEWEMNL103006EP17A Those translators achieve at best far less conversion fidelity than existing file conversion filters between OpenDocument and Microsoft binary file formats such as the OpenOffice.org conversion filters, which achieve only about 80 per cent fidelity. The file format cognescenti know this. See e.g., the paper by Gary Edwards and Sam Hiser included in this edition of the European Journal for the Informatics Professional. http://www.upgrade-cepis.org/issues/2006/6/up7-6Hiser.pdf (PDF). (Note that I contributed to that paper.) And as also detailed in that paper, what works well enough for some of us does not necessarily work well enough for all. Anything less than full fidelity data conversions is absolutely unacceptable in the context of wholly automated business processes and is in fact illegal in various contexts, including government records. So your thesis doesn't fly. In fact, I'd go so far as to bet that you have been suckered by the Microsoft spin doctors. Another indication is your depiction of the file format wars as being waged primarily between IBM and Microsoft, a recent theme of Microsoft's public relations machine. While it is seductive to believe that the controversy is just another chapter in the war between major competitors, the pro-ODF camp is far broader than IBM. For example, nearly 20 governments recently opposed fast track processing of Microsoft's draft standard at ISO. Do you believe they were all carrying water for IBM? Government bodies in more than 50 nations have chosen to adopt ODF. http://opendocumentfellowship.org/government/precedent And dozens of developers now support the OpenDocument standard in their applications. http://opendocumentfellowship.org/applications While IBM has had a noteworthy role in proliferating the OpenDocument formats, there is a movement without a recognizable leader in the industry. When it comes to vendor influence on things relevant to ODF, Sun Microsystem's far outshines IBM. But in fact, a core group of open standards and free and open source developers and advocates -- inside and outside government -- have played a far larger role. This is a customer-driven phenomenon, not a vendor-driven effort as you portray. So I will respectfully suggest that you reexamine your position on these issues. Reasonable minds can differ, but not on the grounds you advocate.
  • Gary Edwards on 12 Dec 07
     
    Here we go again.  A couple of boot lickin lackies at The Register make some moronic statements about the OpenDocument XML file format, and the portable document cognisceti experts come out of the wood work to set the record straight.  I think it's a scam to get boost hits. 

    Once again Marbux hands out a major bitch splappin to Microsoft shills who have no idea what's coming.  What a great job Marbox does, and does with a kind consideration that certainly isn't warranted given the idiocy of the main article.  Where does the man's patience come from?  I gave up long ago.

    ~ge~

Gary Edwards

PlexNex: Achieving Openness - 0 views

fussnotes.typepad.com/...achieving_openn.html

ecma-376 iso microsoft odf officeopenxml ooxml opendocument xml

shared by Gary Edwards on 12 Dec 07 - Cached
  • "ECMA 376" is a set of file formats subject to ECMA and now to ISO. "Office 2007" is a set of file formats which extend "ECMA 376" file formats. Office 2007 file formats are undocumented per se. ECMA 376 are. ECMA 376 file formats are documented but only at a syntactic level. To realize the true meaning of every single attribute is to realize that the documentation is more like 600,000 pages, not 6,000. Of particular difficulty is to keep some kind of control over the virtually infinite combinations of such attributes. Quick analysis of the underlying schemas reveals that simple concepts such as text formatting is expressed in no less than 6 different and incompatible ways. This leads to thinking that the file formats were only designed to comply with existing legacy formats that themselves are the result of 15 years of inside/outside library aggregation (some of the libraries were bought from non-Microsoft vendors). In fact, the truth is, ask any reverse engineer third-party who worked with legacy formats, they'll tell you Microsoft essentially added angle brackets around the binary serialization in legacy formats. This makes for a very cool XML-based file format, not an international standard.
  • Gary Edwards on 12 Dec 07
     
    Whoa, Stepen Rodriguez knocks this one out of the park.  What an impressive dissembling of MS OfficeOpenXML and it's poor sister subset, Ecma 376.  Incredible. 
Gary Edwards

Between a rock and a hard place: ODF & CIO's - Where's the Love? - 0 views

www.consortiuminfo.org/...article.php

cio government microsoft oasis odf officeopenxml ooxml opendocument openxml

shared by Gary Edwards on 12 Dec 07 - Cached
  • So I'm disappointed.&nbsp;And not just on behalf of open documents, but on behalf of the CIOs of this country, who are now caught between a rock and a hard place, without a paddle to defend themselves with if they won't to do anything new, innovative and necessary, if a major vendor's ox might be gored in consequence. After the impressive lobbying assault mounted over the past six months against open document format legislation, I expect you won't be hearing of many state IT departments taking the baton back from their legislators.&nbsp; &nbsp; And who can blame them?&nbsp;If they tried, it wouldn't be likely to be anything as harmless as an open document format that would bite them in the butt.
  • Gary Edwards on 12 Dec 07
     
    Andy Updegrove weighs in on the wave of ODF legislative failures first decribed by Eric Lai and Gregg Keizer compiled the grim data in a story they posted at ComputerWorld last week titled  Microsoft trounces pro-ODF forces in state battles over open document formats.


    Andy believes that it is the failure of state legislators to do their job that accounts for these failures.  He provides three reasons for this being a a failure of legislative duty.  The most interesting of which is claim that legislators should be protecting CIO's from the ravages of aggressve vendors. 


    The sad truth is that state CIO's are not going to put their careers on the line for a file format after what happened in Massachusetts.


    Andy puts it this way, "
      

    And second, in a situation like this, it is a cop out for legislatures to claim that they should defer to their IT departments to make decisions on open formats.  You don't have to have that good a memory to recall why these bills were introduced in the first place: not because state IT departments aren't a good place to make such decisions, but because successive State CIOs in Massachusetts had been so roughly handled in trying to make these very decisions that no state CIO in his or her right mind was likely to volunteer to be the next sacrificial victim.
    As both Peter Quinn and Louis Gutierrez both found out, trying to make responsible standards-related decisions whe
Gary Edwards

Re: [office-comment] Public Comment - 0 views

lists.oasis-open.org/...msg00016.html

shared by Gary Edwards on 23 Jan 08 - Cached
  • Regarding section 1.5 itself: The Open Office TC decided to use the term MAY rather than MUST (or will) at the mentioned location, because it wanted to ensure that the OpenDocument specification can be used by as many implementations as possible. This means that the format should also be usable by applications that only support a very small subset of the specification, as long as the information that these applications store can be represented using the OpenDocument format. A requirement that all foreign elements and attributes must be preserved actually would mean that some applications may not use the format, although the format itself would be suitable. Therefor, we leave it up to the implementations, which elements and attributes of the specification they support, and whether they preserve foreign element and attributes. Some more information about this can be found in appendix D of the specification.
    • Gary Edwards
      Gary Edwards on 23 Jan 08
       
      This OASIS ODF discussion is about the Compliance - conformance clause of the ODF specification: Section 1.5. A developer has complained that use of MAY instead of MUST in the wording of the clause would enable conforming applications to destroy foreign elements and alien attribute markup at will. This of course would result in ZERO Interoeprability!!!!! The foreign elments and alien attributes were included for the purposes of improved ODF compatibility with the billions of MSOffice binary documents that would need to be converted to ODF. Sadly, the section 1.5 loop hole falls short of the compatibility goal, but that only begins to scratch the surface of the ODF problems. OpenOffice only supports foreign elements and alien attributes for text spans, and paragraphs!!!!!! All other such markup is unrecognized and therefore "destroyed" by OpenOffice. ZERO interop. No roundtripping with MSOffice desktops. Lossy conversion with jagged fidelity. Guaranteed.
Gary Edwards

EOOXML objections - Grokdoc - 0 views

www.grokdoc.net/...EOOXML_objections

ecma eooxml imported:del.icio.us iso jtc-1 microsft mooxml oasis odf

shared by Gary Edwards on 12 Dec 07 - Cached
  • Gary Edwards on 12 Dec 07
     
    Marbux has done some heroic work here, using the GrokDoc Wiki.  The Title is "EOOXML Objections", and it's primary purpose is to help ISO National Body Memebers evaluate the 0ver 6,000 pages of the Microsoft - ECMA Office Open XML Specification for MSOffice. 

    On January 5th, 2007, Microsoft officially submitted EOOXML to ISO under the fast track rules.  Before EOOXML can hit the fast track though, ISO provides members with a 30 day "Contradition Review Phase".   During this brief phase, ISO NB's (national standards body members) muct evaluate the proposal and post their allegations concerning contradictions and inconsistencies with other ISO products - like ODF.

    What Marbux is assembling here is a one stop shop for ISO NB's strugglign to understand the issues at stake.  It's incredible wha the has accomplished in such a short time.  But then, the clock is ticking.  February 5th is a hard and unmovable deadline. 

    The basic contradiction is thatt EOOXML is a subset of ISO existing product, ODF.  Both attempt to do the exact same thing:  provide an XML file format for desktop productivity environments such as MSOffice, OpenOffice, and WordPerfect Office.  What seriously differentiates the two is that ODF was designed expressly to be a universal file format, application and platform independent, able to transition across many different information domains connecting the legacy of desktop productivity to near everything else.  MOOXML on the other hand was designed for MSOffice and the legacy of billions of binary documents that only Micrsoft knows the secrets to converting to XML.  As such, MOOXML is designed to be application and platform bound, with these proprietary dependencies written right into the specification.

    One of the more important elements of the Marbux arguments is that the OpenDocument Foundation's daVinci Plugin and InfoSet Engine - API prove conclusively
1 - 11 of 11
Showing 20▼ items per page