Resolving the Durand Conundrum: some ODD thoughts

The Durand Conundrum is a jokey name for a serious question first raised by David Durand when the current TEI ODD XML format was being finalised at an early working group meeting, though I cannot now remember where. In the original TEI ODD system, that used for the production of all versions before P4, content models and system entities were declared using SGML syntax, which was then wrapped in ODD-defined containers of various kinds. When we moved to XML, the line of least resistance was to re-express those same SGML rules using RelaxNG, thus making ODDXML a hybrid beast of a language, in which everything except element content models was expressed in our own XML vocabulary, while element content models (and datatype definitions) used the RelaxNG XML vocabulary. David pointed out, quite reasonably, that this was a lazy compromise of a solution: we could probably express everything using RelaxNG, not just the content models for elements, and so why not ditch the ODD system entirely and use pure RelaxNG ?

We called this a conundrum because we couldn’t come up with any entirely convincing argument against it, and perhaps there is none. A hybrid system is just not good engineering. It means that parts of the conceptual model which ODD expresses cannot be manipulated or specified using ODD itself – witness the contorsions we go through to facilitate different interpretations of class references, or the recent debate about how to introduce interleaving. But the Durand conundrum can be resolved in two ways: one would indeed be to re-express everything in ODD in RelaxNG; the other would be to expand ODD to enable it to define content models natively, without having recourse to a loosely defined subset of RelaxNG. In this article, I would like to explore the second possibility.

The current set of P5 content models makes use of a (largely undocumented) subset of RelaxNG facilities, mainly as a consequence of the design goal of supporting schema generation in DTD and W3C schema languages as well as RelaxNG, so it isn’t strictly true that we are currently using RelaxNG. Support for DTD schema language in particular imposes many limitations on what would otherwise be possible, while the many additional facilities provided by W3C Schema and Relaxng for content validation are hardly used at all (though equivalent facilities are now provided by the <constraints> element). A few years ago, the demise of DTDs was confidently expected; in 2012 however the patient remains in rude health, and it seems likely that support for DTDs will continue to be an ongoing requirement, even after support for P4 is formally withdrawn at the end of 2012.  We therefore assume that whatever elements we use to specify content models will need to have the following characteristics:

• the model permits alternation, repetition, and sequencing of individual elements,
  element classes, or sub-models (groups of elements)
• only one kind of mixed content model — the classic (#PCDATA | foo | bar)* — is
  permitted
• the SGML ampersand connector — (a & b) as a shortcut for ( (a,b) | (b,a) ) is not
  permitted
• a parser or validator is not required to do look ahead and consequently the model
  must be deterministic, that is, when applying the model to a document instance, there
  must be only one possible matching label in the model for each point in the document

To support repetition, we already have attributes @minOccurs and @maxOccurs, which are defined locally on the <datatype> element. We propose that these should instead be supplied by an attribute class @att.repeatable, members of which would include <datatype>, and also the existing elements <elementRef>, <classRef> and <macroRef>, which we repurpose for use within content model declarations.

We also need to add four new elements: <model> to define the content model; <sequence> to indicate that its children form a sequence within a content model; <alternate> to indicate that its children can be alternated within a content model; and <pcdata> to indicate that the content model permits a text node at this point.

<model> might eventually perhaps replace the current <content> element, since it has the same function. It seems safer however to define a new root element for the new proposals in case everything goes horribly wrong. In the testModel.odd test implementation, this new element is added as an alternative to <content> wherever that occurs in the schema.

<sequence> and <alternate> are used to wrap parts of the content model being defined. <pcdata> is used to indicate presence of a pure text node in the model: making it an explicit XML element makes it possible to write a schematron constraint to implement the required restriction on mixed content models (though I haven’t done that yet)

Using these elements, a content model such as ((a, (b|c)*, d+), e?) would be expressed as follows:

<model>
<sequence>
<sequence>
<elementRef key="a"/>
<alternate minOccurs="0" maxOccurs="unlimited">
<elementRef key="b"/>
<elementRef key="c"/>
</alternate>
<elementRef key="d" maxOccurs="unlimited"/>
</sequence>
<elementRef key="e" minOccurs="0"/>
</sequence>
</model>

Note that the value for both @minOccurs and @maxOccurs is understood to be 1 unless it is explicitly provided on an element, or on its immediate parent.

Classes are handled in the same way. Thus, a content model such as (model.a,
model.b+, (model.c | model.d)*)would be expressed as follows:

<model>
<sequence>
<classRef key="model.a"/>
<classRef key="model.b" maxOccurs="unlimited"/>
<alternate minOccurs="0" maxOccurs="unlimited">
<classRef key="model.c"/>
<classRef key="model.d"/>
</alternate>
</sequence>
</model>

When processing this declaration, an ODD processor has to expand each
<classRef> to a model which will match any single member of the specified class, by default.

This behaviour may be varied by supplying a different value for the attribute @expand which we propose to make available on <classRef>. This attribute takes the same possible values as the existing @generate attribute
on the <classSpec> element. For example, assuming that elements a and b are the members of class model.ab,
<classRef key="model.ab" expand="sequence"/> is interpreted as a,b rather than as (a | b).

A mixed content model such as (#PCDATA | a | model.b)* would be expressed as follows:
<model>
<alternate minOccurs="0" maxOccurs="unlimited">
<pcdata/>
<elementRef key="a"/>
<classRef key="model.a"/>
</alternate></model>

References to existing TEI-defined macros are handled in the same way, using the existing <macroRef> element. Where the body of a macro is a <content> element rather than a <valList>, it will of course be necessary to replace that by an equivalent <model>.

---

---