A List Apart

Issue № 199

More About Custom DTDs

by Published in HTML34 Comments

A note from the editors: At our request, J. David Eisenberg, one of A List Apart’s technical editors, wrote “Validating a Custom DTD” to accompany Peter-Paul Koch’s “JavaScript Triggers,” an article that demonstrated a means of abstracting a web page’s behavioral layer from its structural and presentational ones. In their first article for ALA, the W3C’s Quality Assurance Group breaks down the downside of custom DTDs.

In a previous issue of A List Apart, Peter-Paul Koch discussed the addition to his markup of non-standard attributes to create JavaScript triggers. J. David Eisenberg noted that since the attributes were not part of XHTML, the W3C Markup Validator was likely to reject documents using these attributes, and wrote an article on the topic of Custom DTDs and validation.

This article will follow up on these writings by discussing the need for custom DTDs: why making a custom DTD for the sole purpose of validation is a mistake, and in which cases it does make sense to create and use one. For these cases, this article will also present techniques for creating clean custom DTDs and avoiding hacks.

Of poetry and markup languages

Instead of talking about custom DTDs, let’s just talk poetry for a moment. Poetry and web documents are very similar in their language constraints, and poetry also addresses the topics of love, flowers and starry skies, something DTDs seldom do. So, poetry wins.

An impetuous poet may, now and then, commit a spelling mistake or a grammatical blunder in even the most brilliant ode to love. A dictionary or spell checker can therefore be a useful tool, even if it won’t fix a lame rhyme or make a bland metaphor shine; talent, inspiration, and a few other tools will help with that.

Can a poet decide to invent new words, develop new grammatical constructs? Certainly! The new language would not be English, but that’s what poetic license is all about, isn’t it? And if in the end the poem is using a language that is so remote from English that no one understands it, who cares? We can still call it art….

Sadly, browsers, search engine indexers and other agents on the web have a very limited understanding of art and poetic license — and web content, unlike poetry, benefits from the use of a lingua franca. The interoperability derived from using a common language is, after all, the purpose behind the existence of the “web standards.”

Web standards? That’s precisely why I want to validate!

And you are right, of course! Unless, of course, you treat both validation and web standards as some sort of sacred cow, practically forgetting the meaning behind their importance.

This is probably the worst mistake that the advocates of web standards can ever make: to fight for an abstract, arcane concept of standards and consider validation for the sake of validity a goal in itself.

The power of a common language

Babel. The mythic tower teaches us that there is great social value in a language used widely, and that the power of these shared semantics can be maximized if the common language is used properly and consistently by all parties involved in a given communication.

In the web paradigm, where the communication is seldom one-to-one but mostly one-to-many, the above logic is even more crucial, and translates to: “The proper use of the shared semantics of a standard markup language empowers me as a web author, and since I know that validation is one practical way of getting close to that goal, therefore I validate my web documents.”

It is important to remain conscious, however, that validity alone is not a guarantee of compliance, and even further from being a guarantee of quality.

Many normative principles of the HTML specifications cannot be expressed in terms of DTD constraints. The content of attributes, for instance, are generally defined in the DTD as being CDATA (more or less equivalent to “any kind of text”) even though the specification defines a very clear and strict syntax for them. Nothing in a DTD, for instance, can enforce that the content of the lang attribute must be a language code from RFC1766…

Not only isn’t validity a magic key to quality, there isn’t even such a thing as absolute, general “validity.” A document can claim to be using a specific DTD, and be valid with regards to this DTD. This is what is often summarized as “a valid document.”

The above does not deprecate the process of validation. It does, however, remind us of a very important point: validation only one step in checking the correctness of the language used in markup documents, and should never become the goal in itself. That is why the idea of creating a custom DTD for the sole purpose of having documents pass validation might be well-motivated but it is ultimately a deluded endeavor.

When custom DTDs make sense

There are, of course, real cases in which the creation and use of a custom DTD for a custom markup language makes sense. Taking the existing XHTML and enriching it with new elements can be an interesting and useful practice, for instance, to manage an internal knowledge database in a specific field.

The situations in which custom markup languages are needed characteristically include relatively specific, closed environments. These environments can benefit from extending beyond the semantics provided by the standard languages without worrying about the modified semantics of a global language and the absence of interoperability with generic agents. If done properly, the extended or proprietary language can fairly be transformed into “regular” XHTML.

When such a case arises, downloading and modifying the standard DTD until it fits the requirements is not an elegant method; it is even discouraged by the HTML (4.01) specification:

For reasons of interoperability, authors must not “extend” HTML through the available SGML mechanisms (e.g., extending the DTD, adding a new set of entity definitions, etc.).

A much more elegant and efficient method exists: the Modularization of XHTML provides a clean framework to create semantically rich languages. It is, for instance, the method used to build MathML2 (DTD) and the WAP Forum’s XHTML Mobile Profile (DTD).

Creating a custom markup language with the modularization of XHTML

The creation of a custom DTD using the XHTML Modularization method is not very complicated, but it is not trivial either. A step-by-step guide would require a full article, and we will not go into details here, but here are a few pointers and reminders:

  • The instructions from the Modularization of XHTML specification are a good place to learn the practicalities of Modular DTD creation, but the tutorial written by HTML Working Group participant Shane McCarron is an even better starting point.
  • In most cases, it is not necessary to start from scratch as it can be tedious, in most cases, extending XHTML will be relatively easy, and offers the benefit of building on the already defined semantics of the standard XHTML. If you don’t need the whole XHTML as a starting point, chances are picking a few modules and leaving others out will provide you with a compact, yet useful, basis for the new language.
  • When creating new elements, avoid overloading the html namespace. Creating the new elements in a new XML namespace, as recommended by the specification,  is only marginally more complicated, and ensures that semantics are clearly defined and distinguished.

Transforming our custom language to standard XHTML

We are now able to create documents using our new XHTML Host Language (this is the formal name for a language built using the modularization of XHTML), and its extended semantics. In many cases the new language will be used only in a closed environment, but at some point or another we will want to put our data on the whole, wild world wide web.

As discussed above, it may not be a good idea to put documents authored in our proprietary language directly on the web, since few or no user agent will be able to understand its peculiar semantics. But since both our custom language and XHTML are XML-based languages, we can easily use XSLT to transform a document written in the former to a document in the latter.

As a simple example, consider the case where we would have expanded XHTML with a new <poetryml:author> element, using the href and name attributes, and an empty <poetryml:pause /> element marking speech pauses in a poetry verse.

The poetryml2xhtml.xsl stylesheet could then be used to transform the PoetryML markup <poetryml:author name=“foo” uri=“http://people.example.org/misterfoo”> into XHTML: <address class=“author”>foo

The stylesheet is a little long, but the transformation mentioned above is the matter of a few lines:

<xsl:template match="poetryml:author">
    <address class='author'>
        <a href='{@uri}'><xsl:value-of select="{@name}" /></a>

Is this a loss of semantics? From a PoetryML standpoint, it is, since our poetryml:author element would have been very precisely defined, but viewed from the generic web user agent, address has a much clearer meaning than an unknown element in a foreign namespace.

Last words

Custom languages created with the method above are obviously not global standards, but within a given environment, for instance the knowledge database of a company, the specific semantics of this proprietary language are well understood by the range of tools for which they have been designed.

The HTML standard, unlike these proprietary languages, is made through a process ensuring, among other benefits, the widest possible interoperability across web browsers. Building a proprietary language upon this standard is a perfectly acceptable practice as long as you are aware that the proprietary language loses these benefits. A document written by using such a custom DTD may be validated against this DTD, but it will not be valid (X)HTML1.0 Strict, HTML 4.01 Transitional, or any other version of the HTML standard. It will be valid… something else.

Custom DTDs can be a very useful tool to enrich the existing markup languages or create entirely new ones. One always has to keep in mind that they are tantamount to creating a new language, and that proprietary languages are best kept in closed environments where they can be taught to a limited set of agents and tools, and NOT to make the web a modern version of the Tower of Babel by unleashing them in the wilderness.

About the Author

34 Reader Comments

Load Comments