Innovating from inside of an organization is stunning in its difficulty, frustration, and often, it’s difficult to understand why even the simplest of ideas meets with such a high level of friction and sluggish progress. Again, I’ll thank NYU Professor Clay Shirky for his book, Here Comes Everybody, for some sparks that led to this article.

You may recall that my previous post dealt with the connections between the individuals who form a group or a network of groups. Within an organization, those connections are weighted, in part by company hierarchy, in part by control over resources, and in part on the history and fluidity of past relationships. In other words, connections within an organization are often complicated by internal and external factors. And, of course, not every relationship is equally valued. Some connections are stronger than others. You might recall the old 80/20 rule, for example, in which 80 percent of the work is done by 20 percent of the people.

Well, it turns out that the 80/20 rule doesn’t much apply to innovation, or to community interactions. If you look closely at Wikipedia–easily the largest informal group enterprise we’ve ever generated as humans–“fewer than two percent of Wikipedia users every contribute, yet that is enough to create profound value for millions of users. Wikipedia would not be possible if there were concern for inequality.” With a publish-then-filter model now overtaking the older, highly institutionalized model of research-write-edit-rewrite-publish, much more gets written, and errors are corrected along the way, particularly in articles that matter. (Those that don’t much matter are rarely accessed, and so, rarely corrected.) So we have a small number of people–nowhere near 20 percent of the total Wikipedia user base–contributing large amounts for an operation that is a nonprofit, not a business.

It’s here that the divergence becomes interesting. Imagine a business taking on the writing of the world’s largest encyclopedia, one that is never quite published, but always exists in draft form. Companies just don’t work that way–they have processes, standards, and overhead, project management, deliverables, and the entire structure of jobs and careers relies, mostly, upon incremental improvements to the status quo. Very large projects are within the reach of larger institutions, but the process of planning, developing, politicking, funding, hiring and moving people…none of it is simple, and there are ample opportunities for slowdowns, moving off track, shifting priorities, and so much more. That’s how institutions work: they perfect processes over time, but they struggle with entirely new endeavors because the status quo makes so much more sense than the risky new proposition. Massive shifts in thinking are not easy to absorb. Large-scale systemic change does not make sense.

There are fewer than 100 copies of the EB print edition still available (but none in this binding). If you want one, click on this link now (don’t wait!).

Except, of course, that significant, often large-scale, systemic change is becoming a new normal. There is no more Encyclopaedia Britannica in print, no more Tower Records stores, no more Kodak film (well, almost none), no more barriers to global video distribution, no reason why a clever sentence or article can’t be seen by millions of people just an instant after the draft is complete.

So status quo is part of the reason why institutions and innovation aren’t always BFF. But there’s another component, equally important: freedom to fail. When an institution fails, it risks funding, loss of customers, and shifts in leadership. When innovators fail, they may cry in their beer on Friday night, but on Monday morning, they’re back at work, having learned from the flop. No shareholder worries, no customer loss (okay, maybe a little), and in the end, probably more valuable learning than systemic damage. So institutions do all they can to avoid failure, and often, this means extracting the heart of a project or venture, or obfuscating, or demanding more analysis, or some other status quo maneuver. And individuals who are part of, for example, an open source community, correct the errors and move on without substantial loss of momentum (because the primary reason for that community’s existence is to DO things and to avoid NOT DOING things). In this shifted paradigm, the institution struggles to make substantive progress, knowing that the less encumbered other may well cause the death of their venture.

Shirky: “Open source is a profound threat, not because the open source ecosystem is out-succeeding commercial efforts but because it is out-failing them. Because the open source ecosystem, and by extension open systems generally, rely upon peer production, the work on these systems can be considerably more experimental, at considerably less cost, than any firm can afford. Why? The most important reasons are that open systems lower the cost of failure, they do not create biases in favor of predictable but substandard outcomes, and they make it simpler to integrate the contributions of people who contribute only a single idea. The overall effect of failure is its likelihood times its cost. Most organizations attempt to reduce the effect of failure by reducing its likelihood…(making safe choices). Open source doesn’t reduce the likelihood of failure, it reduces the cost of failure; it essentially gets its failure for free…cheap failure, valuable as it is, is also a key part of a more complex advantage: the exploration of multiple possibilities.”

What now? If you haven’t yet read Clay Shirky’s Here Comes Everybody, do it now. If you’ve already done that, you may take the rest of week off. Here he is at Harvard’s Berkman Center for Internet and Society talking about his work….