Once More unto the Breach

A Standards Primer

Photo by Dauvit Alexander

I have recently had several long discussions about the motivations and machinations that surround the development of technology interoperability standards. Over the past few years, I've also captured a lot of ideas and experience on the blog. I pulled it all together into one place in the following paper, "Understanding Technology Standardization Efforts" (PDF 86.2K).

For the record, I was a long term participant in the POSIX and UNIX standardization efforts. I was a working group participant, balloted many pieces of the standards and their amendments, and participated in the management of the standards effort at the IEEE as both an inaugural member of the Project Management Committee and a voting member of the Sponsor Executive Committee. I was an international participant at ISO, as document editor, and participated on behalf of three different national body delegations (Canada, U.S., UK) over a number of years. I began my participation in 1989 as a customer (working for EDS with GM and the U.S. government as their primary POSIX-interested customers), but quickly ended up as a vendor, working for MKS developing a conforming POSIX.2 implementation that formed the basis of implementations from IBM, DEC, HP, UNISYS and Sun. In 1995, I put my money where my mouth was on the importance of applications portability, standards and the coming juggernaut of NT and co-founded Softway Systems, implementing the POSIX and UNIX standards on NT to enable UNIX applications to be directly migrated to the platform. A large amount of free and open source software was incorporated into the product. Softway Systems was acquired by Microsoft in 1999, and I worked there for five years. Over the years I've been in regular contact with people standardizing C#/CLI, the Linux Standards Base, and ODF.

Several friends and colleagues from the standards world have reviewed the paper and provided excellent comments. The paper is much better for it. All mistakes obviously remain my own.

Microsoft Office 2007 and Open XML: Lies, Damned Lies, and Statistics

Last week Joe Wilcox (Microsoft Watch) observed that Microsoft Office 2007 apparently doesn't conform to the Open XML standard (ISO/IEC 29500) that Microsoft has rammed through the system. Alex Brown has the full test here. No surprise. I've argued for the past year that the product must have diverged from the standard under construction. It's a normal thing in the standards world as Joe and Alex observe. They each challenge Microsoft to declare itself with respect to the standard and the future of the product.

But here's the problem: Microsoft already has declared itself. Last August Microsoft commissioned a study from IDC on the adoption of document standards. The "study" names Office Open XML as the obvious favourite. "Among the XML-based document standards, Office Open XML seems to be creating the most traction in the market." In the PR push leading up to the September 2007 votes on ISO/IEC 29500, Microsoft was already equating the standard with Microsoft Office 2007. That's what the sales field will be telling customers, with graphs culled from the "report". [srw — If you really want to read the report, follow the link from Mary Jo Foley's editorial. I still refuse to give the paid report link cred, small as it may be.]

Here's more writing on the ISO adoption and next steps:
Microsoft Claims Success with ISO and Open XML Standard

Follow-up on Brad Smith Open Source Business Conference (OSBC) Keynote

It was indeed an interesting keynote. It was not as I had feared it would be. Brad Smith did an excellent job of engaging the audience, explaining the Microsoft position, and encouraging discussion. Smith focused a lot on the diversity in the market of business and licensing models, not claiming a financial high ground (which is a first), and emphasizing shared values (pride of creation of software and what we have collectively accomplished).

The panellists did a fine job, and the audience was also engaged. (It only felt like Smith was filibustering a little in the end, burning the clock, but then he'd had a long time in front of the audience at that point being on the receiving end of the Q&A.) The mini-survey off the previous blog post did correctly predict where most of the discussion was going to be on patents and Linux.

Key points for me:

• "I appreciate that respect for intellectual property is I believe a shared value across our industry." Smith made this statement midway through the panellist Q&A. This to my knowledge is the first public statement by a Microsoft executive that did not label the free and open source community as IP hostile. It is a significant public statement.
• Bottomley and Updegrove did actually catch Smith out in the Q&A. I wouldn't have thought it possible, considering Smith's background as a lawyer and public spokesperson for Microsoft. Smith claims Microsoft wants its property respected, and that patent licensing is not about the relatively small revenue. He was neatly and visibly cornered at one point (to audience chuckles) because the Linux community is willing to respect Microsoft's property and actively work on a solution that avoids it.
• Based on statements made in Sam Ramji's presentation the previous day, and in Brad's keynote and the answers to questions, Microsoft is trying to find solutions to the patent problems. This does not simply mean giving up the property from a Microsoft perspective, as enabling as this might be for the community at large. Smith is all too familiar with other large vendors chasing Microsoft for patent licensing revenues (and he used the Sun US$900M licensing settlement as an example on stage) to be able to understand why Microsoft should just roll over on the patents they allege Linux infringes. For Microsoft it seems it's difficult to take a step that does not appear to be reciprocal in nature.
• There was an interesting discussion about Cathedrals and Bazaars at one point. Smith (Microsoft) is very comfortable having discussions about Cathedrals having licensing discussions with other Cathedrals. But that analogy (historical, relevant, and useful as it has been) also limits their thinking. They seem to only think in terms of Microsoft as a cathedral that can license to other cathedrals. They believe they've enabled the Bazaar in recent licensing statements. It seems they are still trying to understand the actual ecosystem and have been perhaps using the wrong analogy as a lens. Maybe it's time to evolve the Cathedral and the Bazaar.

At one point Smith observed that what the world wants to see is deeds not words — but that words also matter because it sets the bar against which they will be judged. There was lots of interesting things said and debated over the 90 minutes. Smith has set a high very public bar against which Microsoft will be judged. I'm hoping IT Conversations gets this recording up soon so everyone can hear what was said. [My recording is noisy and missing the first few minutes.] Congratulations to Brad Smith, and the panellists (O'Grady, Updegrove, Bottomley, Shuttleworth) for an excellent session, and of course to Matt Asay for pulling it together.

Other commentary:

• Stephen O'Grady commentary
• Matt Asay's direct commentary
• Matt's other commentary

Microsoft Claims Success with ISO and Open XML Standard

Copyright © 2007 by Kordite

"Another key factor is the fact that people recognize the broad use of Open XML in the market as seen by the hundreds of independent implementations of Ecma 376." [Jason Matusow, Microsoft Director of Standards]

Think of the confusion if we only partially implemented the HTML standard. Okay — bad example. What if we only partially implemented a railroad standard? The track gauge would be correct, but the rail width was incorrect, or there was only one rail? Or maybe the track stopped before reaching its destination. Microsoft continues to maintain the Rovian perspective that a standard with "support" (their language is improving to "implementations") rather than complete conformance is good news for the industry. In this particular case it even ignores the very conformance statement in their own standard. It's only good news for Microsoft. It means lots of people are encouraged to do partial things around documents produced by Microsoft Office 2008. The economics is in the vendor's favour, not the consumer's. It defeats the actual purpose of de jure standardization. [In the industry, we call it a vendor specification regardless of standards body imprimatur.]

We now enter the next phase of the dance. Customers will discover they don't get the benefits that they thought they bought. A customer of note [likely government] or a consortia will put together a conformance certification program around the standards in the space. Brands and certifications will be the rule of the day. Microsoft will discover it needs to actually ensure their own products adhere [formally] to the standards they produced. The Microsoft Office team will discover conformance testing to a specification is (i.) hard work, (ii.) different than normal product testing, and (iii) that their product is drifting off the very standard they launched. (The .NET runtime team learned this a few years ago and I'm betting there are still conformance bugs logged against the product as "won't fix".) Implementation conformance will become important.

"You keep using that word. I do not think it means what you think it means." — Inigo Montoya, in the Princess Bride

Other writing I've done in this space:

• Office Open XML Conformance (A Lesson in Claiming Standards Conformance) [31 August 2007]
• Conformance and Certification: The ODF Standard and Microsoft's Office Open XML Specification [7 January 2007]

Brad Smith Keynotes the Open Source Business Conference (OSBC)

[Updated (1-Apr-2008 21:37): I posted follow-up commentary in a separate post.

[Updated (12-Mar-2008 13:20): My apologies — Matt Asay points out that there will be a 30 minute slot for questions from the audience after the panel has had 30 minutes. I would still encourage we begin the tuning and discussion early: What would YOU ask Brad Smith at OSBC?]

We're just a couple weeks away from this year's Open Source Business Conference (OSBC) in San Francisco. Brad Smith, general counsel for Microsoft, is the closing keynote on the first day. This speaking slot has previously been filled by the likes of Clayton Christensen, Geoffrey Moore, and Lawrence Lessig, and each of these gentlemen have given us deep talks that have forced one to think about open source in the world at large from an economics, business, and legal perspective. Mr. Smith has large shoes to fill, and this worries me.

You see Mr. Smith is a corporate executive, and most execs (with a few notable exceptions like Mårten Mickos) feel compelled to "pitch the company." Mr. Smith is a lawyer (and general counsel to boot) so language is his oeuvre. We've seen Brad Smith's pitches before now. Here's what I hope we don't see at OSBC:

• A rehash of last's months announcement about how "open" Microsoft is. It is indeed a historical moment for Microsoft, publishing protocol specifications that were previously secret and offering generous patent licensing terms regardless of their motivation. The non-commercial restriction on the open source patent covenant makes it a non-starter. It demonstrates either remarkable naïveté of how the open source world works, or it's a deliberate snub. Either way it's irrelevant and not appropriate for this audience.
• Yet another lecture on how important patents are, the need to get a return on your innovation investment, or that the open source community wants special IP rules. We value IP. We don't want special rules. We understand the patent system and as software business people we often choose different IP tools by the necessity of our size. We also see the likes of most other large vendors sophisticated use of their asset portfolios. Most of us think US$40B per year is a pretty good return on investment. We've all asked to be told which of your patent claims you think we're infringing, so we can fix it. Microsoft is either in the room playing well with the rest of us or it's not. But don't pretend to play. That's boring and transparent. (The wrong sort of transparent.)
• More declarations on patent licensing innovation with Novell, Xandros, et al. Those are business cross licenses. Really. Move on. You're being lapped by the likes of IBM, Sun, and Oracle with respect to business innovation and open source.

So let's turn it around. What GREAT things could Mr. Smith announce to demonstrate that the Microsoft executive team gets open source software and they actually want to engage? What property or asset could they liberally share into a collaborative development community (that includes businesses), instead of publishing yet more licenses or making positioning statements? In essence, what could they DO. How about:

• Announce the release of the Sharepoint software base as open source software. Let the world know you will be genuinely exploring the revenue streams of support/maintenance/network in the context of this line of business, while encouraging innovation on the platform, and encouraging a community engagement unlike others in the Microsoft world.
• If he wants to talk about patents, then back up the earlier CIFS/Samba announcements with unrestricted patent covenants to any patents required, and scope the covenant to implementing CIFS. This seems a reasonable way to encourage the community (including businesses) while clearly stating the conditions under which infringement will not be tolerated outside of CIFS implementations.

At the end of the keynote, questions have been limited to a distinguished panel consisting of Stephen O'Grady (Redmonk), James Bottomly (SteelEye CTO), Mark Shuttleworth (Ubuntu/Canonical), and Andy Updegrove (Gesmer Updegrove). I started to think about what I would want to ask and came up with the following:

Question: If you have the list of patents whose claims you believe are infringed by Linux, why won't you release it such that the community can deliver on its statement that they will fix the infringements?

Rationale: Regardless of how some people in the free and open source community feel about software patents, we all understand that it is the legal system we have in place. We all understand that changing that system is a different discussion. The community deeply respects intellectual property. The entire free and open source licensing space is based on strong IP law. People want to fix infringed claims. But we can't fix what you won't share.

Question: Why doesn't Microsoft share more software under open source licenses?

Rationale: Microsoft has a wealth of software assets that are not products. So take the discussion of "not a business model we can embrace" off the table. Microsoft has been "studying" open source and "learning" from open source for almost a decade. No one is suggesting the release of the Windows or Office software base. Why have so few small experiments been done?

But we live in the Internet age. I would love to hear what other ideas and questions people have. So here's a web site that will allow you to enter your questions and ideas, and vote on the others already in place: "What would YOU ask Brad Smith at OSBC?"

Thanks to Sandro Groganz for putting the survey site together so quickly once I asked.

Microsoft Open Source ISV Forum at OSBC

The Microsoft ISV team is again hosting the Microsoft Open Source ISV Forum prior to the Open Source Business Conference in San Francisco in a few weeks.

Last year was a really good event. It was well attended (about 120 people). Sam Ramji spent an hour with the group in open discussion around the challenges of open source at Microsoft and progress made. It was a particularly important discussion at the time because of a Fortune article the week before the forum. There was lots of good commentary in booth directions around such things as IP management. Stephen O'Grady (Redmonk) also presented and was his regularly inimitable self. John Roberts (SugarCRM CEO) spoke about their partner relationship with Microsoft, and the ISV team explained the opportunities available through the NXT program.

This year they're tightening the agenda up. The event starts with lunch and registration from 11:30am, and the event begins properly at 12:30pm. (You won't necessarily need the extra night in the hotel to attend.) Sam will speak again this year, as will Stephen. New ISV partners will present, as well as a panel discussion from a new collection of venture capitalists. If you're an ISV, it promises to be a good half day event to warm up before tackling OSBC itself.

Registration is free and the site is now open. As an added incentive, ISVs that attend the Microsoft Open Source ISV Forum get 50% off the two-day registration for OSBC.

The OOXML Ballot Resolution

[Update (2008-2-25 13:45): There's an excellent press release from ISO that outlines exact history and next steps and requirements for this ballot.]

I have long maintained that technology standardization is commercial diplomacy and the purpose of individual participants (as with all diplomats) is to expand one's area of economic influence while defending sovereign territory. This week a lot of people are gathering in Geneva for the ISO ballot resolution meeting for Office Open XML (OOXML), Microsoft's Office product specification. The debate no doubt will be contentious.

Microsoft had a perfect opportunity to participate in the Open Document Format (ODF) standard's development at OASIS. They ignored that opportunity. The best time for technology standardization arises when a problem space is well understood, with sufficient real implementation knowledge to discern what works and what doesn't. Microsoft had arguably the best experience to contribute. They chose not to participate. Standardized document formats with multiple product implementations posed a threat to their Office business.

That threat became real when the Commonwealth of Massachusetts chose ODF as a basis for product procurement to best serve its citizens. Microsoft's response was not to adopt the ODF standard that already existed with multiple implementations (and continues to act as a hub for alignment with other international work like China's UOF standard), but to rush their own product specification into the standardization process.

They have over the two year process done a remarkable amount of work to bring the specification through ECMA to ISO, and have made great gestures to enable others to support the Microsoft specification.

But there's a problem.

Microsoft is an adjudicated monopoly in the United States. The EU continues to investigate possible abuse of their market dominance. (Market leadership and innovation are not what's being punished, but rather the abuse of a dominant position.) Microsoft can complain all they want, but the practices that enabled their success continue to plague them. We cannot collectively rewrite history. Microsoft is indeed held to a different measure. They have forfeited some of the freedoms that other companies enjoy. In many ways, they have lost our trust.

One can not judge Microsoft's newly declared preference for "openness" against the work they've done promoting their own product specification, but against their continued refusal to adopt ODF. In the end, OOXML as an ISO standard (with its attendant market confusion) will best serve the needs of Microsoft over its customers, and that's a shame.

Andy Updegrove has an excellent essay on his blog as we go into this week's ballot resolution deliberations. He takes a different approach. In it he argues that a particular class of standards should be held to a higher bar for acceptance, because they enable fundamental technology access in the world going forward. He makes an compelling case for why OOXML should be flunked out of the ISO process.

This promises to be a fascinating week.

Would you do Open Source Marketing for Microsoft?

[Updated 2008-02-14, 20:15] Sandro also posted on the subject.

Microsoft’s partners team through its NXT initiative launched a campaign focused on open source ISVs in 2007 in conjunction with the Open Source Business Conference.  The campaign is designed to encourage ISVs to explore how best to deliver their solutions to customers in the Microsoft world, recognizing that many high profile open source projects have 30%-50% of their installed base in the Windows world.   The Microsoft program works through partners to assist ISVs.  

In the open source campaign, the NXT partner works with the ISV to:

• Review the ISV’s business model, solution offering, and channel delivery plans.
• Determine the best approach and plan to expand business in the Windows marketplace. 
• Map Microsoft partner programs that best serve the ISV to their needs.

Microsoft wants to expand the program this year, and Sandro Groganz and I are considering working together to join the program as a partner through his company.  

Sandro set up InitMarketing as a consultancy that develops marketing strategy for companies that use open source software in their solutions to customers.  Sandro’s executive experience at eZ Systems and Mindquarry and as a PHP developer serve him well here.  I’ve done strategy work for and with a number of open source related companies, built a product that used a breadth of open source software covered by a variety of licenses, and spent five years at Microsoft so have some understanding of the organization and its culture.   

As Sandro and I investigate the program, we are not without our misgivings.  Do a search for “open source” on the main Microsoft partner site and you discover mostly anti-open source competitive material in the initial search results.  Do the same search on the NXT partner site and you see a set of open source friendly case studies. (I’ve blogged elsewhere about the Microsoft message conflict problems.) And non-IE browsers don’t always render the site well.  And some parts of the partner world want you to install Silverlight “[i]n order to view the rich media content on this page ....”  None of this is encouraging to the average ISV living in an open source world.  

But this is also our opportunity.  The Microsoft programs were developed out of a culture of “open source equals Linux” and anti-Linux competitive work and sometimes reflect that history.  The NXT program is working to recognize the economic reality of the wealth of free and open source software running in the Windows world today.  We have an opportunity to open the discussion with the ISV team about what open source software actually means on the platform.  We have the opportunity to inform the business to mutual benefit.  

Sandro and I believe that we can start a more transparent discussion with Microsoft’s ISV team, starting here and now.  We’re excited at that opportunity. The ISV and open source teams at Microsoft are open to this discussion. We’re curious what the rest of our readers in our business community think.  Please share your thoughts below.  

Microsoft Partnerships, Open Source Software ISVs, and Culture and History

Last week, Mary Jo Foley offered commentary on Microsoft's open source software strategy with respect to independent software vendors based on an interview with Microsoft's Sam Ramji. Matt Asay provides good colour commentary on his blog. Each post focuses on the trustworthiness and competitive history of the company. Let's look at things from a different perspective.

First, if you sell software solutions, and one of the platforms you support is Windows, you're not alone. There is a lot of free and open source software that has company support that is deployed on Windows as one of its platforms. The "big" free and open source software names include:

JBoss: Claimed 50% deployment on Windows when they signed a partnership deal with Microsoft that included technical collaboration in September 2005.

SugarCRM: Claimed 35% deployment on Windows when they signed their technical collaboration deal with Microsoft in February 2006.

Eclipse: Several studies have been done over the past few years show Windows adoption for development and deployment (Dev/Dep):

(80%/60%) [Evans Data Corp., September 2006]
(62%/37%) [Evans Data Corp., September 2007]
(74%/47%) [IDC, Summer 2007]

While Eclipse itself isn't a company, many ISVs build businesses around the Eclipse project.

Alfresco: Claims 30% deployment on Windows according to their internal study published June 2007.

MySQL: Claims 40% download MySQL for Windows.

If Windows is one of the platforms you support, there should be nothing stopping you from considering joining a Microsoft ISV partner program. They provide lots of material and information to make it easier to develop, market, and sell on Windows than doing it all on your own. It absolutely benefits their growth. It benefits yours as well.

To provide the best possible experience for your customers, if the Windows version of your applications needs to use Active Directory, Microsoft Operations Manager, or any other Microsoft specific technology, then I'm going to assume you'll best serve your customers and architect the application accordingly for multi-platform development, deployment and support. We have been architecting applications to support multiple platforms for decades, and it's easier today with common tools and languages (most of them open source), and standards support across platforms. Again, better to join a Microsoft ISV program and cheaply get access to the best technical information possible for the platform to support your customers.

Some consider "doing business with Microsoft" tainted. There is no excuse for the behaviour we all saw from Microsoft with Netscape, or Sun with Java. If not for the high profile of the last U.S. Department of Justice investigation, the preponderance of email, and the ease with which one can search it, we wouldn't have such wonderfully embarrassing email examples that demonstrate how some people inside Microsoft think. (It makes you wonder what embarrassing gems are sitting in the email queues of other companies that haven't had this level of public legal scrutiny.) But most companies aren't in a position to demand their customers shift platforms. Even MySQL AB supported the SCO Openserver platform better to support customers.

[Contrary to popular belief, Microsoft didn't eat all its original ISV chain out of a rapacious need for growth. Customers demanded Microsoft provide a lot of tool support directly to simplify their development and procurement needs as the platform became ubiquitous.]

There are several things to understand about partnering with Microsoft that have a lot to do with the history and culture of the company. Microsoft was the PC software company, i.e. Microsoft customers had enormous deployments of small footprint inexpensive machines, versus the big server company (1990s) with far fewer big-iron multi-processor machines, versus the mini-computer company (1980s) with fewer less powerful systems, versus the mainframe company (1970s) with fewer still very large machines. While today a company will rack out thousands of blade servers in their back-room big-iron environment, in the 1980s and 1990s they didn't.

Microsoft developed programs throughout their channel sales and marketing organizations to enable them to scale their growth. Their margins were completely different from the other large software companies (e.g. SAP, Oracle) or UNIX OEMs. So a VMS or UNIX ISV may have seen OEM staff arrive on site to help migrate or tune an application to the platform because of the margins involved in that business, but Microsoft had to create a program to enable their DOS then early Windows ISVs to get access to expertise differently. Starting an ISV that developed PC-related software required a far smaller capital outlay and so there were a lot of such companies. Programs like MSDN were created. The ISV programs were by definition information-intense but engagement-soft. It was the smartest way to correctly serve both their ISV partners and their business model.

Hardware, software, and the Internet evolved. Windows became Windows NT has become Windows Vista and Server growing into the data centre. The big iron UNIX server world is evolving into scaled arrays of Linux servers. While these worlds collide in the data centre, the financial success of Microsoft remains tied to a culture of scaled programs. If you're an ISV that is used to the way IBM, Sun, or HP treat you, you may be surprised by a Microsoft program's lack of personal engagement.

There's another cultural practice that contributed to Microsoft's growth and success through the 1990s and that effects partner programs (and customers alike). Employees were allowed, indeed encouraged, to regularly shift positions within the company. In the early days it kept smart people that might be prone to boredom fresh and ensured culture wasn't lost through the employee leaving the company. That practice is still considered a strength today. Scale-out programs (not tied to specific people or relationships) supported this model.

But it means that any relationships built with the Microsoft by an ISV will likely change regularly. You're going to need patience and regular contact to follow-up with the program people you do meet, and [hopefully] get quick early warning when that person is about to change. You're also going to need determination to be the squeaky persistent wheel. You may be one US$10M/year company clamouring for attention from a Microsoft employee drowning on 150+ emails a day who is responsible for a program with 100 companies like you. Managing the relationship to continue to get attention requires perseverance. If you can grow your business better through a technical partnership similar to the Zend PHP relationship, you will need to be prepared for a very focused meeting because you'll likely get one shot to get the point across. (Hint: Start with the punchline then keep things short and direct.)

In the end, do what makes sense for your customers. Join the Microsoft ISV program if it makes business sense. Use the materials that help your business grow and ignore the rest. Do it with your eyes wide open and your expectations set. Best of luck!

GOSCON Discussion on Open Document Formats

Deb Bryant is blogging, which is great news. Deb is of course the creator and executive director of the Government Open Source Conference (GOSCON) that is held each year in Portland, OR.

The closing session of last Fall's conference was an executive panel on open document formats that included representatives from Sun, Microsoft, IBM, and Adobe. Deb's latest post points to the video of the panel, as well as the ongoing GOSCON forum discussion between the panellists. If you're interested in either the open document standards debate or government involvement in free and open source software, I would encourage you to have a read.