Once More unto the Breach

A New Blog about Big Data in the Cloud

Ideas behind the future and importance of Big Data and the Cloud continue to thrive. Tim O'Reilly has been hammering home the message for some time that data will become the "Intel Inside" of the next generation of lock-in, (most recently in a fabulous keynote at OSBC). Companies like Cloudera are making big bets on the platform architecture of the next wave. And when the Economist starts to dedicate special reports to the topic, you know it's reached a particular threshold in people's minds.

I've had the pleasure of knowing Carolyn Johnston for some time. She's a math geek and lead researcher in the MSN Advanced Engineering team specializing in location data and data quality. She's started blogging last week, and her topics will be both fun and educational. (I've the privilege of seeing early drafts in the pipeline.) If you care about the evolving world of big data in the cloud, I encourage you to give her a read.

Visualizing Open Source Business Models

Matt Aslett gave a great talk about the evolution of open source business strategies last week at the Open Source Business Conference in San Francisco. In it, he presented a model to allow one to visualize businesses that use open source in their solutions, and what it means in terms of relationships with customers and the community.

The community relationships are driven by the lower triangle:

The customer relationships are driven by the upper triangle:

Matt goes on to apply the model across a collection of companies and business models, visually demonstrating how different companies have made different choices and evolved those choices, as well as playing out a number of scenarios. It's an excellent model.

Matt and I have debated in the past as to whether or not there is such a thing as an open source business model. I've argued there's no such business model, but discuss it as a set of tools that are applied to the business. Regardless of one's opinions however, Matt has provided an excellent visual model for the discussion and presentation of the ideas. It works in much the same way as the Business Model Design template in visually capturing the information to ensure a completeness of discussion and an understanding of how the parts relate. I'm looking forward to the continuing evolution of the model over the coming year as he prepares the next major open source business report from the 451 Group, and would encourage you to give the entire presentation a read.

Book Burning in the New Millenium

The juxtaposition of two recent New York Times articles quite terrifies me. The first which I read a week or so ago concerns the fact that the religious right is now attacking science again, but this time they are not restricted to merely Darwin's theory of evolution. It opens with:

Critics of the teaching of evolution in the nation’s classrooms are gaining ground in some states by linking the issue to global warming, arguing that dissenting views on both scientific subjects should be taught in public schools.

Other fine quotes include:

The linkage of evolution and global warming is partly a legal strategy: courts have found that singling out evolution for criticism in public schools is a violation of the separation of church and state. By insisting that global warming also be debated, deniers of evolution can argue that they are simply championing academic freedom in general.

And:

In South Dakota, a resolution calling for the “balanced teaching of global warming in public schools” passed the Legislature this week.

“Carbon dioxide is not a pollutant,” the resolution said, “but rather a highly beneficial ingredient for all plant life.”

And the article closes with:

After that, said Joshua Rosenau, a project director for the National Center for Science Education, he began noticing that attacks on climate change science were being packaged with criticism of evolution in curriculum initiatives.

He fears that even a few state-level victories could have an effect on what gets taught across the nation.

James D. Marston, director of the Texas regional office of the Environmental Defense Fund, said he worried that, given Texas’ size and centralized approval process, its decision on textbooks could have an outsize influence on how publishers prepare science content for the national market.

“If a textbook does not give enough deference to critics of climate change — or does not say that there is real scientific debate, when in fact there is little to none — they will have a basis for turning it down,” Mr. Marston said of the Texas board. “And that is scary for what our children will learn everywhere.”

It's a disturbing article to read in general. It's terrifying because it presents the idea of small steps, none of which are catastrophic (e.g. South Dakota), leading to a destination where the world has changed in the most fundamental of ways (i.e. Texas setting the tone for the national market).

The second article (actually the first chronologically) was in the business section. It gets scarier still. It has to do with MacMillan, one of the five largest publishers of trade books and textbooks, introducing fully editable textbooks. The article begins:

In a kind of Wikipedia of textbooks, Macmillan ... is introducing software called DynamicBooks, which will allow college instructors to edit digital editions of textbooks and customize them for their individual classes.

Professors will be able to reorganize or delete chapters; upload course syllabuses, notes, videos, pictures and graphs; and perhaps most notably, rewrite or delete individual paragraphs, equations or illustrations. [srw &mdash Emphasis added.]

While many publishers have offered customized print textbooks for years — allowing instructors to reorder chapters or insert third-party content from other publications or their own writing — DynamicBooks gives instructors the power to alter individual sentences and paragraphs without consulting the original authors or publisher. [srw &mdash Emphasis added again.]

I have great confidence that the contracts the authors sign will give MacMillan the ability for appropriate copyright control to allow this sort of re-editing. But let's be really really clear. This is NOTHING like Wikipedia. The ability to change the author's original content to suit one's own needs is not the same as providing a rich editing environment where controversies are clearly apparent. The ability for a professor at a university with a strong religious bent in a State "simply championing academic freedom in general" to edit text books to suit their needs is a recipe for disaster. Credible sources (and the original author's brand and credibility) can be twisted to support the insanity of challenging established science. This is not good.

Writing the rules such that content can be changed without changing an author's intentions is still a recipe for disaster as it will place efforts to police, debate, and correct things on the authors and the system. The damage will have been done. Orwell suggested that language precedes thought. If there is no word for a concept, it cannot be expressed. A proper tyranny would do well to remove such words from use. In the modern web connected world, the concept may still exist on the web, but sowing confusion may replace the need to remove a word from use, or to destroy a book outright.

Textbooks still have weight in our society. It's not just the literal weight of paper, but the sense of organization and flow and prestige and credibility. They are also a legacy of a particular way of teaching subjects. As more professors explore the ability to develop course materials from an array of online sources into a coherent collection that matches their curriculum, the textbook will rightly shift in the minds of students and lecturers alike into something that is less important. In such a case, one might presume that individual course collections maintain copyright appropriately, with individual authors credited for their contributions, as well as the overall collected work copyright. This is a interesting marketplace design problem where individuals, journals and historical textbook companies make materials available for use to lecturers assembling course readings. More importantly, however, it means the integrity of the original materials will be maintained. No "books" need be destroyed in the process.

The Future of Book Publishing Business Models

Tim O'Reilly tweeted a great article from the New York Times on the math of publishing traditional print versus eBooks. If you publish print books, and aren't as aggressive as O'Reilly Media at experimenting with new forms, or looking over your shoulder at Scribd, then you would feel very justified about the entire NYT article. But it ignores the future in a very fundamental way. It assumes the weight of the entire book publishing process from author and editor through paper manufacturing, distribution, and end-retailer needs to be maintained.

I would mourn the loss of book stores as much as the next bibliophile. There are a thousand or so books within easy reach in the apartment. There are amazing bookstores throughout the world in which I find peace and solace from the chaos amongst all that collected human creativity, knowledge, and imagination. Good book stores smell right, and you know a good book store the second you walk into them. Book stores are indeed holy places.

But I remember growing up with Sam the Record Man in Toronto. Three floors of goodness, with the finest collection of jazz, classical, and rock music in Canada. Sam's spawned an entire chain across the country. A&A Records was next door to the flagship Sam's. There were many pilgrimages to the pair of stores through my teen years and early twenties. And like a good book store, Sam's just smelt right. I will always have the memory and my daughters will never know what they're missing, except they don't want to either. They have their own generational memory. The way we consume music has changed. Records were supplanted by cassettes, then CDs. Now many of us live in an iTunes and Amazon MP3 download enabled world. The traditional distribution chain changed. New musicians often self promote for a period of time, producing their own CDs and selling their music through iTunes, before being "discovered" by a label to help them scale. The music now promotes the concert tour revenue stream, rather than the other way around.

This will happen to the book publishing industry. The model will change. People outside the publishing house will re-invent the book and how it's consumed.

• When does someone set up an Internet marketplace for authors, editors, copyeditors, and illustrators to find one another and share the revenues directly? Google has a tool base for online collaboration and are certainly interested in books. With Amazon's latest royalty offering for Kindle, an author can deliver a Kindle edition and could "share" their 70% royalty with editors that made the book better or illustrators that did the cover design. Or maybe the payment system front loads the payments to the supporting "staff" before the author begins to make the lion's share. Indie movies and indie music have been around for a while, when do we end up with a serious indie book industry?
• When does Amazon create the iPhone/Android app and the programme that will allow bookstores to receive a cut of every Kindle edition they sell? I scan the book's in-store barcode with my smartphone, and I get the Kindle edition delivered, and the store gets its cut. Why is this different in concept than Borders on-line store being run on Amazon, or any of the independent book sellers that front through Amazon? It's not the normal book mark-up, but people already browse bookstores and buy on Amazon. This is better than no revenue. (When was the last time you went to a travel agent?)
• If we have an indie eBook publishing industry, does producing limited copies for browseable book stores and gifts become a new publishing industry? Do such copies in bookstores become collectibles because they're more scarce? What publishers (in what countries) will become the de facto efficient producers of one-off or limited run books?
• Public libraries are interesting from an economics perspective. They exist to support and encourage literacy. Their funding model is local government set. The books they buy are often a more robust expensive package (as are their books-on-tape, and their CD prices are often higher to reflect replacement costs). They often provide Internet access but even here on Microsoft's doorstep in Redmond, Washington, the 25 or so PCs are always in full use. I don't think libraries are going to be replaced by eBooks any time soon, but some publishers are already trying to reconfigure to chase strictly the high margin school/library market.
• When is the vanity of coffee table books and browsing the book case when you visit someone's house get replaced by a digital wi-fi connected picture frame rolling the covers of the family's collected eBooks collections? Or when indeed do beautiful photo coffee table books become the download for the picture frame on your living room wall (with the helpful text a bluetooth read on your tablet away). Or does having books themselves become the cultural vanity item?

All of these are of course random ideas of an unknowable future. But as Clay Shirky observed this week: "Abundance breaks more things than scarcity does. Society knows how to react to scarcity."

P.S. Sam's is literally gone now. You can still see a little of the Sam's logo painted on the wall of the back alley. A&A's was taken over by HMV for music and videos. And the Future Shop (like Best Buy for U.S. readers) ironically was there as well. HMV and the FS are expanded and down the street now on better real estate.

Open Source Software Economics in Pictures

Updated [22-Mar-2010]: Added a little text around the Ohloh javascript widget so Google Reader sees a URL to follow.

Recently, I've encountered several technologists that still don't understand open source software economics and got suitably cranky about "people giving away software for free" and "destroying the value of innovation". I thought it time to try to reach for an easier way to demonstrate what's happening in the industry in pictures.

Everyone is familiar with the idea of a normal "bell curve" distribution representing R&D investment over time. As a technology is better understood and a product succeeds in the marketplace, R&D investment increases, and over time as new technologies advance, the R&D investment in the original technology and product wanes. The integral represents the total R&D investment over time. The function can also represent the "knowledge" gained or the increase in the intellectual asset base.

Good companies develop and invest in new successive waves of sustaining technologies. So, looking at Microsoft's success with PC operating systems, DOS was replaced by a greater investment in a more innovative Windows, was replaced by a larger investment in a more innovative NT.

This also fits nicely with Christensen's original observations about incumbent companies being good at sustained innovations and well run companies knowing how to jump from technology to technology along a sustaining innovation path. This all makes sense when considering a single company's R&D investment. It applies equally well to Sun Microsystems when considering that the steeper slope of successive sustaining innovations was on the hardware side versus the slower (but not inconsiderable) investment from SunOS (a BSD variation) to Solaris.

The investment curve for projects like Linux and Apache, with lots of individual and corporate contributors, still looks like a bell curve, but the contributions might better be viewed as a stacked bar chart. Individual contributors invest to meet their specific needs. Because there is enormous overlap in their common needs, they all share the overall investment.

Individual contributors get enormous return on their investment. (One gives a few bug fixes to the Apache httpd team, and in return one gets an entire HTTP daemon.) Corporate contributors give for the same ROI. They get enormous return on their investment in technology they use in a product complement space or as a component in their overall solution to the customer. (Before someone takes issue with my Red Hat example above below, understand the "solution" in the customers mind was "UNIX-like servers on inexpensive 'PC' hardware" and not "Linux".)

Christensen was careful in subsequent work to point out that the disruption wasn't about technology but about business model. The disruption often started when someone assembled standard cheaper lower performing parts into a solution that solved a completely different need with a very different cost basis. The new solution begins its own sustaining innovation curve until the new technology can compete with the incumbent compared against the criteria about which the customer/consumer cares.

The disruptive business model isn't about Linux so much as the ability for corporations to do collaborative development at the component/complement level in a "frictionless" well-managed Internet-enabled community. (The original OSF/1 shared-development of a UNIX-like replacement failed: too few players, too much politics.) Linux is a much stronger disruptive business solution as a way to handle a particular sourcing problem.

It would be interesting to consider the difference between projects with enormous inbound code contribution (versus all the other strengths a well run community brings to the table) distributed across a wide group of players like we see in the Linux and Apache projects, versus projects managed more tightly by a company like MySQL was. Another interesting attribute of this collaborative business model to investigate is how contribution mutates over time. Christensen's work demonstrated that an incumbent gets in trouble when they begin to over-deliver on functionality for attributes their customers consider important. The customer can't absorb the sustaining technology innovation any faster and literally won't pay for it. The slope of the sustained innovation of the competing technology is sufficient to cross into the space covered by the incumbent's solution.

In a shared collaborative development environment, however, because the technology isn't being driven by a single corporate entity, the community of corporations collectively contributes to their own needs and the technology may (i) stabilize where it needs to stabilize, and/or (ii) be taken in new and interesting directions. There is less pressure (if any) to over-deliver with new innovation. The consumers are the developers, but it's a very broad community indeed. This is what I believe just happened with the MeeGo announcement and the combining of the Nokia Maemo and Intel Moblin projects. This is a great inflection point for Linux into the new mobile Internet device space.

One only need read the report from the Linux Foundation charting the growth statistics in the Linux kernel to understand the enormous value generation happening release-on-release, four times a year. Using the Ohloh rules-of-thumb of US$55,000 per person year one gets US$142M of value creation in the 2.6 Linux kernel. The fact that some business models have been destroyed (Sun), or threatened (Microsoft) doesn't mean there's not enormous ongoing value creation in the technology.

Ohloh value chart for Linux 2.6 kernel.

Neither is intellectual property being "destroyed". Again, this is a disruptive business model discussion. Intellectual property is a business choice made on how a company will protect certain intellectual assets as legal property. Which assets to protect, and how, and which property to defend is a business choice based on the cost model a business uses with respect to turning assets into value propositions customers will buy. When a group of companies chooses to collaboratively develop a technology complement/component, they're making a business model choice on how they will selfishly share certain intellectual assets. Nothing was destroyed along the way.