Standardization and the Open Web

We’re done arguing over the importance of web standards. Accessibility, stability, quality control, and ease-of-use all helped to settle the debate long ago. Advocacy websites created to promote web standards—such as Chris Heilmann’s Web Standards for Business and The Web Standards Project—haven’t needed to change at all since the mid-2000s.

What has changed, however, is the way standards are developed—an issue arguably as important as the standards themselves. The next community debate, then, isn’t about web standards; it’s about how web standards should be standardized.

What’s in a standard?#section2

The idea that standardization is important is reflected in the language we use to describe our projects and communities. For example, the JSON API homepage states that it is a “Standard for building APIs in JSON.” The FAQ page describes JSON API as a specification, and developers are talking about its use in terms of compliance. A competing project, HAL, references the visual language of standardization on its website—the flow of the page reminiscent of a formal Request For Comment, before directing you to the actual Internet Engineering Task Force RFC.

These projects illustrate a conflation of ideas about standards, which, left unaddressed, can lead to confusion. Both the JSON API specification and the HAL specification are de facto standards—an idea for a best practice approach to a common-use problem that is spreading organically through the developer community.

The specifications we tend to think of more commonly, such as those for HTML and JavaScript, are voluntary consensus standards, meaning that international standards-setting bodies and industry consortia have agreed to work on and adopt these specifications, and create incentives for their implementation. But even in a voluntary consensus environment, differences of opinion can split a technology—JSON (not to be confused with JSON API) actually has two competing voluntary consensus specifications: one with the standards group Ecma, the other with IETF.

While the term “standard” is used here in all cases, all specifications are not created equal. We sometimes even see RFCs for technical specifications that will never become standards because they are theoretical ideas for how something might work; ergo, all standards will have specifications, but not all specifications are standards.

Making standards#section3

“Official” standards are specifications that have gone through a process of voluntary consensus. There is potentially a clear path for projects to evolve from a de facto specification to one that is standardized through voluntary consensus:

1. Developer identifies problem and proposes solution to peers;
2. Peer community provides feedback and proposes potential alternate solutions in public channels like GitHub or Google Groups;
3. Peer community reaches mass consensus and hands specification off to a standards body;
4. Developers implement solution while the standards body formalizes and legalizes the standard.

Most developers I know are smart, resourceful, and prefer the path of least resistance; thanks to the all bugs are shallow mentality of the OSS community, they’re inclined to work together to solve issues of mutual concern. This is a fairly straightforward, not-at-all-new idea, and The Extensible Web Manifesto is essentially a call to action to implement more developer feedback on this very process.

It’s exciting to think that the next official web standards might come from the developer community, but in practice this path to official standardization has been obfuscated. The Responsive Images Community Group experienced this firsthand when it proposed a specification for the picture element—noting an issue with the way HTML handled responsive images, the RICG proposed a developer-built, de facto solution to the Web Hypertext Application Technology Working Group, maintainers of the “living” HTML standard. In a well-documented series of events, WHATWG practically dismissed the developer solution in favor of a vaguely specified solution they created over the course of a few days. If it weren’t for the passion and persistence of the community and of RICG leadership, the developer solution would’ve been defeated.

The RICG specification was ultimately accepted by WHATWG and the W3C, but the experience of the standardization process certainly left a bad taste in developers’ mouths. It would be easy enough to focus our attention on improving this process for community groups like RICG, and the web would certainly be a better place for developers if we did so—but wouldn’t it be nice if we could define standardization not as “a process that makes technology,” but as “a process that makes agreements about technology”?

In reality, open standardization is a fundamentally power-laden and political process, and it’s making its way into how we think about Open Source project and community governance. Put in the terms of Eric Raymond’s seminal essay, we’ve built web technologies in the bazaar-style of the open source development ethos, but standardizing those technologies is a cathedral-building activity.

As we seek to standardize technology, we need to recognize the tension inherent in building cathedrals that will later become central authorities for us to reject. Our challenge is to find the balance between capitalizing on the benefits of standardization processes without eroding our community ideals. Thankfully, there are long histories of standardization efforts in other industries and communities that can provide insight for standardizing the web.

Old models for modern standards#section4

Open source communities can learn a lot from the histories and governance models of different standards organizations—indeed, web standards consortia like Ecma International and the W3C already have similar organizational structures, but it’s helpful to understand the prior art upon which we are laying our community standards-setting foundation. After all, the “keep what works” mentality only works in the long run if you understand why it works in the first place.

The ideological origins of web standards bodies come from early efforts to standardize telegraphy and engineering in the 19th century, through committees such as the American Society of Civil Engineers, American Society of Mechanical Engineers, and American Institute of Electrical Engineers. Many hosted regular “congresses”—Victorian-era precursors to today’s web development conferences—that helped to create standards and to further define the identity of the professional engineer.

As engineering disciplines began to overlap, it became clear that cooperation between these industrial societies would be necessary, so in 1918 the American Engineering Standards Committee was formed to encourage cooperation and coordination of standards across groups. The resulting structure was an “organization of organizations” that facilitated consensus-building among multiple engineering organizations, each comprised of a diverse pool of engineers from a diverse set of companies.

Today, the AESC is known as the American National Standards Institute, but its 100-year-old model of governance—rife with community crises and ideals—is reflected in web standards groups. Then, as now, organizational disputes often arose between “shop culture” practitioners and “school culture” academics. A certain amount of tension between groups is healthy for moving ideas forward, and these early groups evolved organizational means for creating and releasing tension as necessary. Today’s default response to disputes in open-source software is to fork the specification in question, producing a network of rival camps who are incentivized to emphasize differences instead of areas of agreement.

When ideals compete#section5

“Openness” is a core ideal in the Open Web community, as well as something of a polluted word. The rhetoric of openness is meant to communicate a favorable set of values, and those values often depend on the speaker and the audience. In his book Open Standards and the Digital Age, Professor Andrew Russell notes that “for individuals, ‘open’ is shorthand for transparent, welcoming, participatory, and entrepreneurial; for society at large, open signifies a vast increase in the flow of goods and information through a global, market-oriented system of exchange.” In the absence of a single definition that suits all parties, we tend to take “open” to mean “inclusive of everything.”

Standardization, on the other hand, is often a process that defines what something is in terms of what it is not. Russell notes that the broader societal goal of standardizing technology is to create a “cohesive and flexible network” that can sustain complex social and economic activity. Thus, the process of making standards means incorporating a wide range of practices and ideas with political, economic, and cultural dimensions—all of which may be of strategic importance to creators, implementors, end users, and the general public. Put this way, standards are technically-oriented instances of diplomacy.

In establishing the ISO subcommittee for developing open working standards in 1977, Charles Bachmann noted that “the adjective ‘open’ means to imply that all participants come to the system as equal partners.” In reality, participants don’t often come to the table as equal partners—the OSI’s own progress was stymied by organizational power plays and the growth of a competing technology, TCP/IP. But equality as an ideal of open standards-making has remained. This ideal is rooted in a deeply held opposition to centralized power, which, according to Russell, is reflected in the histories of many standards-setting organizations. Upholding this vision of equality and achieving successful implementation meant, at times, hiding conflicts and issues from those outside the meeting room—not exactly the transparent behavior one might expect from an “open” system.

If standards really are agreements between equal parties, then the agreement is the controlling authority. And if standards-setting is a rejection of centralized control, then the standardization process becomes one of creative destruction. It’s the ideological circle of open-standards-making life: a group makes a consensus standard on some technology; as the standard circulates, a new party arises to point out a flaw or an unconsidered use case for the existing standard. The original group then has to make room for the new party and rework the standard, or else face rejection of the group and the standard. In rejecting the original group, the offended party forms a competing group and standard, and the cycle begins anew.

It’s complicated#section6

It’s a tangled web we weave standardizing the Open Web—political, economic, and social relationships between people, technologies, companies, and industry groups are difficult to ascertain at a glance. On closer inspection, one can see that these organizations and communities are complex systems forming a complex network—so complex that I was compelled to create this interactive Open Standards network graph to help keep it all straight as I researched.

Before we rush off to create a complex, decentralized network of open source standards groups, it probably warrants mentioning that complex systems fail 100% of the time. A decentralized network may let us fail smaller in most cases, but the key to longevity of the system is failing smart—and if the research has taught me anything, it’s that standardization fails on the human, not the technological, element. For better or worse, complexity is not viral—so to mitigate this, we need to make the complexity of the standardization system consumable without abstracting away meaningful parts of the process.

In the absence of community coordination, methodless enthusiasm will ensue—and caught somewhere in the Bermuda triangle of competing standards bodies, implementers, and OSS maintainers is the developer community. If we want our community-driven projects to become official, internationally recognized standards, we need to understand the impact of our governance processes as well as we understand the technical specifications for our technologies.