Innovation versus Institutions

Innovating from inside of an organization is stunning in its difficulty, frustration, and often, it’s difficult to understand why even the simplest of ideas meets with such a high level of friction and sluggish progress. Again, I’ll thank NYU Professor Clay Shirky for his book, Here Comes Everybody, for some sparks that led to this article.

You may recall that my previous post dealt with the connections between the individuals who form a group or a network of groups. Within an organization, those connections are weighted, in part by company hierarchy, in part by control over resources, and in part on the history and fluidity of past relationships. In other words, connections within an organization are often complicated by internal and external factors. And, of course, not every relationship is equally valued. Some connections are stronger than others. You might recall the old 80/20 rule, for example, in which 80 percent of the work is done by 20 percent of the people.

Well, it turns out that the 80/20 rule doesn’t much apply to innovation, or to community interactions. If you look closely at Wikipedia–easily the largest informal group enterprise we’ve ever generated as humans–“fewer than two percent of Wikipedia users every contribute, yet that is enough to create profound value for millions of users. Wikipedia would not be possible if there were concern for inequality.” With a publish-then-filter model now overtaking the older, highly institutionalized model of research-write-edit-rewrite-publish, much more gets written, and errors are corrected along the way, particularly in articles that matter. (Those that don’t much matter are rarely accessed, and so, rarely corrected.) So we have a small number of people–nowhere near 20 percent of the total Wikipedia user base–contributing large amounts for an operation that is a nonprofit, not a business.

It’s here that the divergence becomes interesting. Imagine a business taking on the writing of the world’s largest encyclopedia, one that is never quite published, but always exists in draft form. Companies just don’t work that way–they have processes, standards, and overhead, project management, deliverables, and the entire structure of jobs and careers relies, mostly, upon incremental improvements to the status quo. Very large projects are within the reach of larger institutions, but the process of planning, developing, politicking, funding, hiring and moving people…none of it is simple, and there are ample opportunities for slowdowns, moving off track, shifting priorities, and so much more. That’s how institutions work: they perfect processes over time, but they struggle with entirely new endeavors because the status quo makes so much more sense than the risky new proposition. Massive shifts in thinking are not easy to absorb. Large-scale systemic change does not make sense.

There are fewer than 100 copies of the EB print edition still available (but none in this binding). If you want one, click on this link now (don’t wait!).

Except, of course, that significant, often large-scale, systemic change is becoming a new normal. There is no more Encyclopaedia Britannica in print, no more Tower Records stores, no more Kodak film (well, almost none), no more barriers to global video distribution, no reason why a clever sentence or article can’t be seen by millions of people just an instant after the draft is complete.

So status quo is part of the reason why institutions and innovation aren’t always BFF. But there’s another component, equally important: freedom to fail. When an institution fails, it risks funding, loss of customers, and shifts in leadership. When innovators fail, they may cry in their beer on Friday night, but on Monday morning, they’re back at work, having learned from the flop. No shareholder worries, no customer loss (okay, maybe a little), and in the end, probably more valuable learning than systemic damage. So institutions do all they can to avoid failure, and often, this means extracting the heart of a project or venture, or obfuscating, or demanding more analysis, or some other status quo maneuver. And individuals who are part of, for example, an open source community, correct the errors and move on without substantial loss of momentum (because the primary reason for that community’s existence is to DO things and to avoid NOT DOING things). In this shifted paradigm, the institution struggles to make substantive progress, knowing that the less encumbered other may well cause the death of their venture.

Shirky: “Open source is a profound threat, not because the open source ecosystem is out-succeeding commercial efforts but because it is out-failing them. Because the open source ecosystem, and by extension open systems generally, rely upon peer production, the work on these systems can be considerably more experimental, at considerably less cost, than any firm can afford. Why? The most important reasons are that open systems lower the cost of failure, they do not create biases in favor of predictable but substandard outcomes, and they make it simpler to integrate the contributions of people who contribute only a single idea. The overall effect of failure is its likelihood times its cost. Most organizations attempt to reduce the effect of failure by reducing its likelihood…(making safe choices). Open source doesn’t reduce the likelihood of failure, it reduces the cost of failure; it essentially gets its failure for free…cheap failure, valuable as it is, is also a key part of a more complex advantage: the exploration of multiple possibilities.”

What now? If you haven’t yet read Clay Shirky’s Here Comes Everybody, do it now. If you’ve already done that, you may take the rest of week off. Here he is at Harvard’s Berkman Center for Internet and Society talking about his work….

Group Dynamics – Internet Edition

20121001-220030.jpgOne of the questions that historians may ask about our era is why technology became so ubiquitous, and so central to our lives. The important idea is not technology, of course, but the way we behave as a result of the tools that technology has provided. Mostly, the historian’s answers will focus on new forms of group dynamics, for these provide the underpinning for nearly all of our digital success stories.

(Many of the ideas in this article, and in several articles that follow, were sparked by the brilliant NYU professor Clay Shirky. You should buy his book right now. It is entitled Here Comes Everybody. Stop reading this blog, get your credit card, click here, then c’mon back to finish reading.)

(Welcome back.) While you were away, thirty six of us formed a big circle. And because you were away for a while, we were struggling to pass the time, and the woman next to me proposed a wager. She was willing to bet $50 that no two people in the circle shared a birthday. Nobody took the bet–it seemed like an easy way to lose money.

Shirky: “With 36 people and 365 possible birthdays, it seems like there would be about a one-in-ten chance of a match, leaving you a 90 percent chance of losing fifty dollars. In fact, you should take the bet, since you have better than an 80 percent chance of winning fifty dollars… Most people get the odds of a birthday match wrong… First, in situations involving many people, they think about themselves rather than the group…instead of counting people, you need to count the links between people.”

When counting connections, 1 plus 1 equals 1, but 1 times 4 equals 6. If I’ve done my math correctly, each of the 36 people in the circle has 35 connections, so the equation would be 36*35 or 1,260. If we were calculating unique connections–so we don’t double count both your connection to me and my connection to you, then we would divide by 2, and the number of unique connections would be 630, still a number far larger than 35, the number most people would choose in the bet.

The number of people = 6 (blue circles), but the number of connections = 15 (red lines).

Why does this matter? Consider LinkedIn, an Internet company whose entire operating theory is based in Internet connections. If you are reading this blog, you are likely to be one of my 500+ primary connections (that is, we are directly connected), but you are more likely to be two or three steps away–that is, you may be connected to one of the tens of thousands of people who are connected to my 500+ and even more likely to be connected to the hundreds of thousands (millions?) of people who are connected to the tens of thousands who are two steps away from me.

And why does that matter? It matters because I want to maintain my network, but it is nearly impossible to productively make use of such a large network–the connections are too diffuse, too unreliable, too far out of reach. Instead, my network, and your network, consists of a few dozen people, perhaps as many as a hundred or two hundred. And as long as at least a few of those people–the dozens or hundreds–remain connected to one another, my network remains viable. If, however, I lose contact with a few important connectors, the size and resilience of my network may dissolve.

Shirky again:

“A group’s complexity grows faster than its size…You can see this phenomenon even in small situations, such as when people clink glasses during toast. In a small group, everyone can clink with everyone else; in a larger one, people clink glasses only with those near them.”

And here’s why that matters. If you are trying to accomplish anything meaningful on the Internet that involves connections or interactions between people, you need to understand small world networks. And with that, Shirky closes us out:

“In 1998, Duncan Watts and Steve Strogatz published their research on a pattern they dubbed the “Small World Network.” Small World networks have two characteristics that, when balanced properly, let messages move through the network effectively. The first is that small groups are densely connected. In a small group, the best pattern of connection is that everyone connects with everyone else. The second characteristic of Small World networks is that large groups are sparsely connected. As the size of your your network grew, your small group pattern, where everyone connected to everyone, would become first impractical, then unbuildable. By the time you wanted to connect five thousand people, you would need a half million connections.”

” So what do you do? You adopt both strategies–dense and sparse communities– at different scales…As long as a couple of people in each small group know a couple of people in other groups, you get the advantages of tight connection at the small scale and loose connection at the large scale. The network will be sparse but efficient and robust.”

Thanks to Emil for working out the basic mathematical formula that calculates connections (time prevented us from completing it for all cases). That formula is:

((n)*(n-1)/2 where n = the number of people in the group. Example: ((6)*5)/2 = 15

Since the connection between Person A and Person B is the same connection as Person B and Person A, division by 2 eliminates the double counting.

The formula starts working with 6 people. If anybody knows why the formula falls apart with groups of 5 or fewer group members, please comment below. And, how do we deal with fractions (half a connection divided by two)?

%d bloggers like this: