Skip to main content

Diigo Home
Home/ OpenDocument/ Group items tagged quality

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
Paul Merrell

untitled - 0 views

www.dpawson.co.uk/...090612.html

ODF specificity quality testability

shared by Paul Merrell on 18 Jun 09 - Snapshot
  • Most (quality) specifications provide clear instructions using
    those magic words SHALL, SHALL NOT, and MAY where those words have
    a defined meaning for an implementor. Paragraphs are clearly
    identified as either normative or informative. That way an
    implementor knows what they must and may implement to claim
    conformance against a specification. This approach has been well
    established over time as a sensible way for spec writers and
    implementors to work

  • Most (quality) specifications provide clear instructions using
    those magic words SHALL, SHALL NOT, and MAY where those words have
    a defined meaning for an implementor. Paragraphs are clearly
    identified as either normative or informative. That way an
    implementor knows what they must and may implement to claim
    conformance against a specification. This approach has been well
    established over time as a sensible way for spec writers and
    implementors to work




    That is the way quality specifications are written. For
    example, ISO/IEC's JTC 1
    Directives     (link to PDF) requires that international standards
    designed for interoperability "specify clearly and unambiguously
    the conformity requirements that are essential to achieve the
    interoperability."





    With that clarity, conformance is testable and can provide
    confidence of interoperability. A suite of tests may be developed
    and applied to an implementation to determine which tests pass,
    which fail, and hence arrive at an objective pronouncement on
    conformance of an implementation against the entirety of the
    specification.

  • In a quality specification, it should be feasible to select a
    normative paragraph, identify a conformance test for it, and make
    a clear statement that this test proves that an implementation
    meets (or fails to meet) that requirement. Call it a test plan:
    define the tests (test specification), define the expected set of
    results, and define what constitutes a "pass" of each test that
    establishes conformance. The plan then provides the matrix of test
    spec against requirement. Simple.
  • ...4 more annotations...

  • Rob Weir of IBM chaired (apology for the misuse of that last
    word) the formation list and then simply announced what the
    charter would be rather than seeking consensus among the list
    participants. As part of this process before that charter was
    produced and while I still naively believed that consensus was a
    goal, I sat down with ODF 1.1 and did a paragraph-by-paragraph
    review for testability. The numbers were quite revealing. I
    completely reviewed only the first four major sections and found
    very few clear requirements.




    The majority were mere statements with no normative language
    used to identify what was required or optional. Implementors would
    have to make their own interpretation.

  • It's ironic that the chair viewed as good news the fact that
    there were far fewer testable paragraphs than he had predicted. But
    his prediction of 10,000 test cases is probably far closer to how
    many testable paragraphs there should be; my counts were actually
    bad news.
  • All of the above leads to the interesting question of just how
    the chair expects to accomplish much that is useful in regard to ODF
    conformance testing before the specification is amended to tighten
    up the language and add clear requirements. The syntax conformity is
    already handled by validation against the schema, but the semantics
    are woefully under-specified.
  • Summary: ODF 1.1 isn't verifiable as a specification. From a
    fairly cursory review of the latest draft, ODF 1.2 will follow the
    same path. With OASIS now being more demanding regarding conformance
    requirements on every specification and with ISO/IEC taking a closer
    interest in liaison with the ODF TC, I find it hard to see how the
    ODF TC co-chairs can maintain this view toward verification.
1 - 1 of 1
Showing 20▼ items per page