I was fortunate enough to give a talk at OSCON 2008 in Portland. I realized on Tuesday (the day before the talk) that my business tutorial slides that I normally use as a level set when consulting were just NOT going to cut it. Here are the OSCON slides:
I've been thinking about this since I published the Standards Primer a month ago. In the next two to five years, Google will be challenged by a technology standards effort that it will need to encourage or manage in its mission to organize the world’s information and make it universally accessible and useful. Such efforts cannot be erected quickly (as demonstrated most recently by Microsoft), and preparations for that day should begin in the near future. Google is also in a unique position to turn the standards development process on its head.
Here are a number of ideas for how Google can plan for the future in the most flexible and cost effective manner, and contribute a unique and valuable re-think of the standards development process. Well organized, a standards function can be an order of magnitude more effective at growing or defending a company’s business.
A Standards Office for Google
There are a set of strategic questions to be answered by a standards function at Google: What standards should be developed or encouraged to reduce the friction of organizing the world’s information and making it universally accessible and useful, or to open up new areas for growth? Is Android a reference implementation? An open source community? Does it require a standard? Are there micro-format standards that would make indexing web sites, libraries, or health records easier? What standards could others be planning that need to be managed because they threaten Google customer engagements or limit growth? Is a search results standard good or bad for business?
Google has a perceived public culture around serving customers, aggressively innovating, doing no evil, and having fun. These attributes should be celebrated equally loudly in their standards function which for most companies is portrayed as staid, conservative and dull.
As with all business functions, success in the standards function is a matter of organization, preparation and execution. To succeed, you need to:
The following is the start of some considerations for a standards office at Google:
The investment need not be great. Google is in a great position to begin. But there are even more interesting ideas in the standards arena and Google is in a unique position to fulfill them.
Turning Standards on their Head
Taking a typical Google viewpoint of what could be done with infinite cycles, storage, and bandwidth when thinking about standards opens up new opportunities in the standards engagement space that come back to Google’s mission to organize the world’s information and make it universally accessible.
Developing interoperability standards is a labour intensive process. A working group meets to debate and discuss the nuts and bolts of the specification, but that act and its attendant work have lots of friction in the system:
While different standards organizations exist to fulfill different missions, and so have different structures, policies, and practices, this lack of underlying consistency leads to a diverging base of information rather than a converging one. Regardless of the differing policies and practices of the standards development organizations, the friction in the system is common and causes lots of problems.
In the end, regardless of policy and procedure tied to specific SDO, they all need the same basic toolset to support the process. Google has the building blocks in http://code.google.com and “Google Apps for your Domain” to develop http://standards.google.com providing the tool infrastructure and best practices to solve many of the friction problems faced by standards developers.
Regardless of an SDO’s policies and procedures, individual working groups can more easily deliver standards that meet the needs of the SDO and its constituencies in ways that would make the standards more discoverable, accessible, and useable. Using this as the opening olive branch, while developing a strategic standards office would put Google in an excellent position quickly.
Google is in an excellent position to begin their standards strategy function. The effort should be undertaken soon to provide the best most effective platform for success in this strategic area going forward.
Photo by Dauvit Alexander
I have recently had several long discussions about the motivations and machinations that surround the development of technology interoperability standards. Over the past few years, I've also captured a lot of ideas and experience on the blog. I pulled it all together into one place in the following paper, "Understanding Technology Standardization Efforts" (PDF 86.2K).
For the record, I was a long term participant in the POSIX and UNIX standardization efforts. I was a working group participant, balloted many pieces of the standards and their amendments, and participated in the management of the standards effort at the IEEE as both an inaugural member of the Project Management Committee and a voting member of the Sponsor Executive Committee. I was an international participant at ISO, as document editor, and participated on behalf of three different national body delegations (Canada, U.S., UK) over a number of years. I began my participation in 1989 as a customer (working for EDS with GM and the U.S. government as their primary POSIX-interested customers), but quickly ended up as a vendor, working for MKS developing a conforming POSIX.2 implementation that formed the basis of implementations from IBM, DEC, HP, UNISYS and Sun. In 1995, I put my money where my mouth was on the importance of applications portability, standards and the coming juggernaut of NT and co-founded Softway Systems, implementing the POSIX and UNIX standards on NT to enable UNIX applications to be directly migrated to the platform. A large amount of free and open source software was incorporated into the product. Softway Systems was acquired by Microsoft in 1999, and I worked there for five years. Over the years I've been in regular contact with people standardizing C#/CLI, the Linux Standards Base, and ODF.
Several friends and colleagues from the standards world have reviewed the paper and provided excellent comments. The paper is much better for it. All mistakes obviously remain my own.
Last week Joe Wilcox (Microsoft Watch) observed that Microsoft Office 2007 apparently doesn't conform to the Open XML standard (ISO/IEC 29500) that Microsoft has rammed through the system. Alex Brown has the full test here. No surprise. I've argued for the past year that the product must have diverged from the standard under construction. It's a normal thing in the standards world as Joe and Alex observe. They each challenge Microsoft to declare itself with respect to the standard and the future of the product.
But here's the problem: Microsoft already has declared itself. Last August Microsoft commissioned a study from IDC on the adoption of document standards. The "study" names Office Open XML as the obvious favourite. "Among the XML-based document standards, Office Open XML seems to be creating the most traction in the market." In the PR push leading up to the September 2007 votes on ISO/IEC 29500, Microsoft was already equating the standard with Microsoft Office 2007. That's what the sales field will be telling customers, with graphs culled from the "report". [srw — If you really want to read the report, follow the link from Mary Jo Foley's editorial. I still refuse to give the paid report link cred, small as it may be.]
Here's more writing on the ISO adoption and next steps:
Microsoft Claims Success with ISO and Open XML Standard
Copyright © 2007 by Kordite
"Another key factor is the fact that people recognize the broad use of Open XML in the market as seen by the hundreds of independent implementations of Ecma 376." [Jason Matusow, Microsoft Director of Standards]
Think of the confusion if we only partially implemented the HTML standard. Okay — bad example. What if we only partially implemented a railroad standard? The track gauge would be correct, but the rail width was incorrect, or there was only one rail? Or maybe the track stopped before reaching its destination. Microsoft continues to maintain the Rovian perspective that a standard with "support" (their language is improving to "implementations") rather than complete conformance is good news for the industry. In this particular case it even ignores the very conformance statement in their own standard. It's only good news for Microsoft. It means lots of people are encouraged to do partial things around documents produced by Microsoft Office 2008. The economics is in the vendor's favour, not the consumer's. It defeats the actual purpose of de jure standardization. [In the industry, we call it a vendor specification regardless of standards body imprimatur.]
We now enter the next phase of the dance. Customers will discover they don't get the benefits that they thought they bought. A customer of note [likely government] or a consortia will put together a conformance certification program around the standards in the space. Brands and certifications will be the rule of the day. Microsoft will discover it needs to actually ensure their own products adhere [formally] to the standards they produced. The Microsoft Office team will discover conformance testing to a specification is (i.) hard work, (ii.) different than normal product testing, and (iii) that their product is drifting off the very standard they launched. (The .NET runtime team learned this a few years ago and I'm betting there are still conformance bugs logged against the product as "won't fix".) Implementation conformance will become important.
"You keep using that word. I do not think it means what you think it means." — Inigo Montoya, in the Princess Bride
Other writing I've done in this space:
[Update (2008-2-25 13:45): There's an excellent press release from ISO that outlines exact history and next steps and requirements for this ballot.]
I have long maintained that technology standardization is commercial diplomacy and the purpose of individual participants (as with all diplomats) is to expand one's area of economic influence while defending sovereign territory. This week a lot of people are gathering in Geneva for the ISO ballot resolution meeting for Office Open XML (OOXML), Microsoft's Office product specification. The debate no doubt will be contentious.
Microsoft had a perfect opportunity to participate in the Open Document Format (ODF) standard's development at OASIS. They ignored that opportunity. The best time for technology standardization arises when a problem space is well understood, with sufficient real implementation knowledge to discern what works and what doesn't. Microsoft had arguably the best experience to contribute. They chose not to participate. Standardized document formats with multiple product implementations posed a threat to their Office business.
That threat became real when the Commonwealth of Massachusetts chose ODF as a basis for product procurement to best serve its citizens. Microsoft's response was not to adopt the ODF standard that already existed with multiple implementations (and continues to act as a hub for alignment with other international work like China's UOF standard), but to rush their own product specification into the standardization process.
They have over the two year process done a remarkable amount of work to bring the specification through ECMA to ISO, and have made great gestures to enable others to support the Microsoft specification.
But there's a problem.
Microsoft is an adjudicated monopoly in the United States. The EU continues to investigate possible abuse of their market dominance. (Market leadership and innovation are not what's being punished, but rather the abuse of a dominant position.) Microsoft can complain all they want, but the practices that enabled their success continue to plague them. We cannot collectively rewrite history. Microsoft is indeed held to a different measure. They have forfeited some of the freedoms that other companies enjoy. In many ways, they have lost our trust.
One can not judge Microsoft's newly declared preference for "openness" against the work they've done promoting their own product specification, but against their continued refusal to adopt ODF. In the end, OOXML as an ISO standard (with its attendant market confusion) will best serve the needs of Microsoft over its customers, and that's a shame.
Andy Updegrove has an excellent essay on his blog as we go into this week's ballot resolution deliberations. He takes a different approach. In it he argues that a particular class of standards should be held to a higher bar for acceptance, because they enable fundamental technology access in the world going forward. He makes an compelling case for why OOXML should be flunked out of the ISO process.
This promises to be a fascinating week.
Andy updegrove posted great news this morning on his standards blog. The US Federal Trade Commission (FTC) announced its resolution that a patent licensing promise made by a patent holder in a standards setting process is binding on a future holder of the patent.
National Semiconductor participated in an IEEE standards effort to develop the 100 Mbps "Fast Ethernet" specification in 1994. Two key (pending) patents were under their control, and they licensed them clearly, cleanly, and cheaply for US$1000 flat one-time fee to all takers. The patents changed hands, first to a group (2002) that wanted to change the licensing deal, then to N-Data (2003), a patent troll that was aggressively pursuing a changed expensive license.
Andy sums it up best:
"[T]he reliance upon promises made with respect to patents is of concern not only in the standard setting context, but with respect to open source software as well. The details of the settlement will provide significant guidance as to how the regulators would view similar conduct in an open source setting. Moreover, in the case of N-Data, the FTC has acted aggressively while acknowledging that the actions at issue might not rise to the level of violating relevant antitrust laws. In doing so, the Commissioners provide strong assurance to participants in standard setting that the FTC recognizes the importance of standards in the modern world. Finally, the details of the actual settlement demonstrate a willingness on the part of the FTC to craft a detailed and savvy set of requirements that addresses the realities of actual licensor-licensee conduct in the marketplace."
This is great news in the context of patent promises made to open source developers from the likes of IBM and Sun Microsystems, and through mechanisms like the Open Invention Network and the Linux Foundation's Patent Commons Project. It removes FUD slung around with respect to patents and intellectual property in both the standards arena and open source project communities. Each is a collaborative effort with significant economic importance and impact. Each will hopefully see the intellectual property landscape a little more clearly now.
Full details on Andy's blog.Deb Bryant is blogging, which is great news. Deb is of course the creator and executive director of the Government Open Source Conference (GOSCON) that is held each year in Portland, OR.
The closing session of last Fall's conference was an executive panel on open document formats that included representatives from Sun, Microsoft, IBM, and Adobe. Deb's latest post points to the video of the panel, as well as the ongoing GOSCON forum discussion between the panellists. If you're interested in either the open document standards debate or government involvement in free and open source software, I would encourage you to have a read.
It's official — IBM has joined the OpenOffice.org project. [There's good reporting and analysis from Andy Updegrove and Redmonk's Stephen O'Grady. Update (12 Sep): Here's Andy's interview with IBM's Doug Heintzman, Director of Strategy for the Lotus division.]
Here's the back of the envelop analysis.
From the OpenOffice.org community perspective, I'm guessing Louis Suarez-Potts (OO.o Community Manager) is feeling good to get a new injection of code/energy. This is great for the community. The OpenOffice suite keeps getting better and better, but new blood with new code could provide a much needed boost.
Overall Sun Microsystems is probably [very] happy IBM is supporting OpenOffice.org directly. This is a much better situation than IBM building some form of ODF development platform inside Eclipse.org to enable ODF over OOXML, with OpenOffice.org hit as collateral damage. [This would be sort of ironic since Eclipse helped to pull the Java centre-of-gravity away from Sun, and Visual Studio was collateral damage (or icing depending upon one's perspective).] Collaboration is the much stronger market play here for Sun and IBM, and most importantly OO.o users and customers.
From the IBM perspective, this is brilliant business as usual. ODF is the global leverage they need to crack open the Microsoft Office marketplace. (I've written ad nauseam that ODF and Microsoft Office is just another example of Christensen economics in motion. Microsoft has over-delivered on Office. They mistakenly think more innovation faster is the answer. Let the chips fall where they may.) IBM will likely use OpenOffice to front-end Lotus and the Domino server product lines, and anchor their business messages to their customers's needs around standards and open source software, much the same as they do with Eclipse and the Websphere developer world. Their claims are that much stronger with this announcement.
Sun gave Gnome a huge leg up about four years ago when they contributed a wealth of their accessibility technology R+D. IBM will now contribute the same into OpenOffice.org. It means they can easily manage their way through U.S. government procurement regulation in this space. Once again brilliant IP management from IBM, and good for OO.o users and customers. [For those that have heard me present, this is exactly what I mean about having a mature intellectual asset strategy, and being generous exactly in order to play to win.]
A strengthened OpenOffice.org will help Novell immeasurably to keep their distance with Microsoft on the desktop. Novell has done a lot of work with OO.o in the past. They have a great desktop Linux product. They can simply take a ride on this one and eat the benefits. There's really nothing Microsoft can say here. Regardless of any agreements around OOXML that Novell may have with Microsoft, Novell comes out clean on the ODF front as customers demand it.
I noticed the press release includes a quote from Beijing's Redflag Chinese 2000 Software Co., Ltd., the makers of Redflag Linux and RedOffice. This is significant. Apparently last November I was one of the first people to blog about the document format work in China that led to a Chinese national standard (UOF). Redflag Chinese 2000 was implementing UOF in Red Office (the Chinese packaging of OO.o). There is work afoot to harmonize ODF and UOF. And clearly Redflag Chinese 2000 remains committed to the OO.o effort.
So despite the bluff and bluster, the OOXML camp inside Microsoft should not be sleeping well at this point.
"Don't blink. Blink and you're dead. Don't turn your back. Don't look away. And don't blink. Good luck!" — the Doctor
So I'm a naive user of technology. (No. Really I am. Ask anyone that's worked with me.) I am definitely not an expert in modern XML document standards. (I have actually hacked troff escapes in a document production chain to insert commands in the PostScript output stream that would be recognized by the PDF generator to produce a hyper-linked document, so I know a little bit about the concepts involved, but that was also 10 years ago now.)
I am a [marvelously happy] Mac user for the past two years. That means I already have iWork 2008 loaded with the new improved Pages '08 (the Apple word processor). On the Apple web site, if I search for "office open xml" then I end up on this page (31 Aug, 2007), which tells me all about Pages '08:
Widely compatible.
Pages ‘08 supports industry-standard formats, so you can easily open documents created in other word processing applications and share documents with others. Whether they’re using a Mac or a PC.
Open for business.
Import your Microsoft Word documents into Pages ’08 with ease. Whether they’re Microsoft Office 2007 (Office Open XML) or earlier Word files, Pages will open them. Pages imports not only the text, but also the styles, tables, inline and floating objects, charts, footnotes, endnotes, bookmarks, hyperlinks, lists, sections, change tracking, and other elements of your original Word document.
COOL! I'm in! This is awesome. I want to see how well I can read interesting docx files. As it happens, ECMA International makes the Office Open XML standard available as both PDF and as docx files. Clever — it's a document format standard see, and so they've provided it in its own format. Perfect.
So I download the .docx version of ECMA-376. All 5 parts of it. And I open "Part 1 - Fundamentals" and immediately get told some warnings occurred:
I choose to review and get:
The file mostly looks good, but not quite as clean as the PDF image with the other font (Consolas?). And clicking on the first warning (about the unsupported field) gives me NO additional information to understand what/where the error might be. Now this is what we in the standards industry call "a quality of implementation issue". Clearly Apple has not done a good job. Get used to hearing this phrase a lot in the press — I'm predicting Microsoft will be forced to apply it liberally to their partners that helped them win votes and helped with the marketing message.
Then I notice the paging problem. I have no idea why, but there seems to be page drift between the PDF and .docx versions. [More on THAT little problem in a minute.] The paging problem does NOT mean there's necessarily a problem with the standard itself but rather the document production machine ECMA was using — we don't know what the definitive source and tool chain was that produced the PDF. (Serious document production is the same as serious software production, something most word processor users fortunately don't get to experience.) Oh, and there are line numbers in the PDF that don't appear in the .docx as opened by Pages '08.
Ignoring the document version skew problem, I decide to see what happens when I throw an even bigger docx file at Pages '08. So I open "Part 3 - Primer" and ...
A few more "warnings" to deal with here. More missing font problems. Things were "removed". No helpful information as to what or how.
I asked a friend with Office 2007 to download and open the two .docx files. You guessed it — no warnings. So we're now on the slippery slope. Apparently I can create files in Office 2007 that Microsoft marketing claims are "standard" Office Open XML that may (or may not) use proprietary extensions. Or maybe Apple did a really bad job. How would a government customer interested in preserving documents know? But it gets worse. The Office 2007 pagination perfectly matched the PDF version. And there are line numbers in the Office 2007 version just like the PDF version.
Ooops.
I'm betting the average business or government office person saving a file won't think twice about it. You see Office 2007 gives you no way to save something as "strict" Office Open XML. Not even not by default, but not at all. Microsoft's definition of "Office Open XML" appears to be .docx itself.
Indeed, even Apple's Pages '08 will only EXPORT to old Microsoft Office format (.doc) and not the standard Office Open XML (.docx) format. So I appear to have no way to generate a OOXML file from Pages '08. [Yes, yes, yes — Microsoft will again point out it's a quality of implementation problem. Or they'll point out that Pages '08 is a "consumer" of OOXML only, which is allowed by the standard. I get it. It's not Microsoft's fault. I'm beginning, however, to wonder at the quality of implementation on the Novell platform. There's a business partnership under duress.]
So as an adjudicated monopoly of desktop operating systems, supplying an office productivity suite with 95+% market share, they will be able to claim instant victory for the adoption of their international standard because .docx files equal Office Open XML standard files. Oh, wait — that's what was essentially done in the IDC study published this week that was "sponsored by Microsoft".
[Now we're about to get a wee bit tedious and exact as standards wonks are prone to be. I'm going to try to explain the conformance game. It can be subtle. Apologies in advance for perhaps getting too ... well boring. If you're not interested in standards mechanics, you can safely stop reading.]
So OOXML defines a couple of types of conformance. There is Document Conformance, and Application Conformance. And conforming applications can be producers (i.e. OOXML document writers) or consumers (i.e. OOXML document readers) or both. Here's the text from the standard [Part 1, PDF edition, p. 3, lines 8-30]:
2.3 What this Standard Specifies
To address the issues listed above, this Standard constrains both syntax and semantics, but it is not intended to predefine application behavior. Therefore, it includes, among others, the following three types of information:
- Schemas and an associated validation procedure for validating document syntax against those schemas. (The validation procedure includes un-zipping, locating files, processing the extensibility elements and attributes, and XML Schema validation.)
- Additional syntax constraints in written form, wherever these constraints cannot feasibly be expressed in the schema language.
- Descriptions of element semantics. The semantics of an element refers to its intended interpretation by a human being.
2.4 Document Conformance
Document conformance is purely syntactic; it involves only Items 1 and 2 in §2.3 above.
- A conforming document shall conform to the schema (Item 1) and any additional syntax constraints (Item 2).
- The document character set shall conform to the Unicode Standard and ISO/IEC 10646-1, with either the UTF-8 or UTF-16 encoding form, as required by the XML 1.0 standard.
- Any XML element or attribute not explicitly included in this Standard shall use the extensibility mechanisms described by Parts 4 and 5 of this Standard.
2.5 Application Conformance
Application conformance is purely syntactic; it also involves only Items 1 and 2 in §2.3 above.
- A conforming consumer shall not reject any conforming documents of the document type (§4) expected by that application.
- A conforming producer shall be able to produce conforming documents.
This is the traditional way things are done with programming languages standards as well. The concept of a strictly conforming C-language program is defined in the ISO/ANSI C standard so as then to define conformance of an actual implementation (i.e. C-language compilers). In the OOXML standard, document conformance exists to be able to talk about implementation conformance, i.e. what readers/writers need to produce or accept if they conform to the standard.
For completeness sake, the "document type" reference in 2.5 above is described in section §4 as [Part 1, PDF Edition, p. 6, lines 16-26]:
document type — One of the three types of Office Open XML documents: Wordprocessing, Spreadsheet, and Presentation, defined as follows:
- A document whose package-relationship item contains a relationship to a Main Document part (§11.3.10) is a document of type Wordprocessing.
- A document whose package-relationship item contains a relationship to a Workbook part (§12.3.23) is a document of type Spreadsheet.
- A document whose package-relationship item contains a relationship to a Presentation part (§13.3.6) is a document of type Presentation.
An Office Open XML document can contain one or more embedded Office Open XML packages (§15.2.10) with each embedded package having any of the three document types. However, the presence of these embedded packages does not change the type of the document.
Now there is no statement of conformance to Office Open XML on the Apple web site beyond the above statement of "support". A search in Pages '08 Help for "office open xml" finds no reference at all. So Apple appears not to actually claim conformance to the OOXML standard anywhere. They simply "support" it. So they're not really guilty of not reading a conforming OOXML document.
But the Microsoft standards and marketing machines are claiming "support" for their standard with the assured tones that "support" = "conformance". Aside from the successful "adoption" claims in the aforementioned IDC report (where Office 2007 market share apparently equates to Office Open XML adoption) we have Tom Robertson (Microsoft General Manager of Standards and Interoperability) "citing support in products from Novell, Corel, Apple and others." Disingenuous at best.
Jason Matusow points out on his blog:
A real litmus test for the viability of the ISO/IEC DIS (draft international standard) 29500 (Open XML) is whether or not there are independent implementations. The answer to this question for Open XML is an unequivocal yes. There are independent Open XML implementations based on the existing specification in applications that run on Linux, Mac, Palm OS, iPhone, and Windows.
Again note the complete lack of reference to actual conformance per the definitions in the standard they have driven through the process. These are the people that are responsible for standards management and messaging at Microsoft. They are by definition the folks that should be defending the strict conformance of the standards in which they participate, and not merely suggesting that partial implementations are a "great start".
So where does this leave the government customer that thought they were buying an open document format for document exchange and interop? It is indeed finally time to roll out the certification machine — for everybody. Let the games continue.
Subscribe to the feed