Only Half of This Is True

Maybe not now. But soon.

Turns out, facts are like radioactive materials, and, for that matter, like anything that’s not going to last forever.

arbesmanMore or less, this is half-life principle, developed just over 100 years ago by Ernest Rutherford, applies to facts, or, at least, a great many facts. This persuasive argument is set forth by Samuel Arbesman in a new book called The Half-Life of Facts. I especially like the sub-title: “What Everything We Know Has an Expiration Date.” Arbesman is a math professor and a network scientist, and, as you would expect, this is a smart book. The book seems more like a musing than a fully worked-out theory, but I suspect that’s because facts are not easy to tame. Herding facts is like herding cats.

HalfLifeOfFactsLet’s begin with “doubling times”–the amount of time it takes for something (anything) to double in quantity. The number of important discoveries; the number of chemical elements known; the accuracy of scientific instruments–these  double every twenty years.  The number of engineers in the U.S. doubles every ten years. Using measures fully detailed in the book, the doubling time for knowledge in mathematics is 63 years, in geology it’s 46 years. In technology knowledge, half lives are quiet brief: a 10 month doubling for the advance of wireless (measured in bits per second), a 20 month doubling time for gigabytes per consumer dollar. With sufficient data, it’s possible to visualize the trend and to project the future.

So that’s part of the story. Of course, it’s one thing to know something, and it’s another to disseminate that information. As the speed of communication began to exceed the speed of transportation (think: telegraph), transfer of knowledge in real time (or, pretty close to real time) became the standard. But not all communications media is instantaneous. Take, for example, a science textbook written in 1999. The textbook probably required several years of development, so let’s peg the information in, say, 1997. If that textbook is still around (which seems likely), then the information is 16 years old. If it’s a geology text, the text is probably valid, but if it’s an astronomy text, Pluto is still a planet, and there are a lot of other discoveries that are absent. And, there are facts rapidly degrading, some well past their half life.

Trans-Neptune

Although you can click to make the image bigger, Pluto still won’t be a planet…

And, then, of course, there are errors. Sometimes, we think we’ve got it right, but we don’t. Along with the dissemination of facts, our system of knowledge distribution transfers errors with great efficiency. We see this all the time on the internet: a writer picks up old or never-accurate information, and republishes it (perhaps adding some of his or her own noise along the way). An author who should know better gets lazy and picks up the so-called fact without bothering to double check, or, more tragically, manages to find the same inaccurate information in a second source, and has no reason to dispute its accuracy. Wikipedia’s editors see this phenomenon every day: they correct a finicky fact, and then, it’s uncorrected an hour later!

Precision is also an issue. As we gain technical sophistication, we also benefit from more precise measures. The system previously used for measurement degrades over time–it has its own half-life. Often, errors and misleading information are the result.

The author lists some of his own findings. One that is especially disturbing:

The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.

And, here’s another that should make you think twice about what you see or hear as news:

The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.

My favorite word in the book is idiolect. It is used to describe the sphere of human behavior that affects the ways each of us sends and receives information, the ways in which we understand and use vocabulary, grammar, pronunciation, accent, and other aspects of human communication. A fact may begin one way, but cultural overlays may affect the way the message is sent or received. This, too, exerts an impact on accuracy, precision, and, ultimately, the half-life of facts.

Word usage also enters in the picture. He charts the popularity of the (ridiculous) phrase “very fun” and finds very strong increase beginning in 1980 (the graph begins in 1900, when the term was in use, but was not especially popular).

Time is part of the equation, too. The Long Now Foundation encourages people to think in terms of millennia, not years or centuries. Arbesman wrote a nice essay for WIRED to focus attention not only on big data but on long data as well.

Given all of this, I suspect that the knowledge in the brain of an expert is also subject to the half-life phenomenon. Take Isaac Newton–pretty smart guy in his time–but the year he died, most of England believed that Mary Toft had given birth to sixteen rabbits.

Last week, on CBS Sunday Morning, Lewis Michael Seidman, a Georgetown University professor commented about our strong belief in the power and relevance of the U.S. Constitution (signed 1787, since amended, but not substantially altered):

This is our country. We live in it, and we have a right to the kind of country we want. We would not allow the French or the United Nations to rule us, and neither should we allow people who died over two centuries ago and knew nothing of our country as it exists today.

CBS News Constitution

Unreasonable

As the year winds down, a call-out to some unreasonable people.

One is called The Unreasonable Institute.

Why We Exist: To create a world in which no one is limited by their circumstances.
Our Mission: To unlock entrepreneurial potential to overcome our world’s greatest challenges.

Three recent college graduates decided to take on the world’s biggest problems–no shortage of idealism here–by causing interactions between promising entrepreneurs with big ideas, mentors, and funders. They do all of this–quite reasonably, I might add–by having everybody work and live together in a big house for several weeks. I’m not sure that “institute” (their term) is the ideal description, but this combination networking fest and dorm experience makes a lot of sense. There are lots of informal interactions between smart, interested, connected people who want to make things happen. I love this idea, and I suspect you will, too.

The second is called Charity: Water.

charity: water is a non-profit organization bringing clean and safe drinking water to people in developing nations.

Two simple ideas here. One is clean water for everyone, everywhere in the world. That’s a tremendous challenge, one that can be solved only on a local level, well-by-well, source-by-source. It’s also a transformative idea: clean water means healthier people, far less time each day caring for the ill; empowerment of women (who, in many places in the world, expend an enormous amount of time at the well or other source, and carrying water home).

20121227-172439.jpg

To play the video, please click on the image.

Both are mentioned here are examples of a new way of thinking about the world’s problems: a small entrepreneurial group with big ideas, unique approaches to management, operations and funding, plenty of attention to details, and, far less reliance upon large organizations to provide solutions. And one more thing: the internet is central to the success of these new conceptions. Be sure to explore Charity: Water’s use of internet mapping for every project, a solid example of things to come.

BTW: while searching for a link, I ran into a Huffington Post story that explains the trend in more detail. It’s definitely worth reading, especially at a time of year when we’re all trying to figure out how to do it even better next week.

A Fact Can Be a Beautiful Thing

It’s often tempting to consider the past through a present-day lens, and that causes distortion. Take, for example, the idea of a fact. Our ancestors did not elevate the fact as anything of importance. Instead, they considered facts to be evil, in opposition with God’s plan. This way of thinking begins with perception, a capacity that we share with animals. “For them, knowledge had to be something more than what we learn from our senses, because it is such a distinctly human capability of our God-given and God-like soul.”

Thomas Bacon, known for the Scientific Method and for his cool combination of dapper hat, moustache-goatee combo, and stylish  collar.

Thomas Bacon, known for the Scientific Method and for his cool combination of dapper hat, moustache-goatee combo, and stylish collar.

It isn’t until the 1700s that the current idea of a fact takes shape. In the Italy of the 1500s, double-entry bookkeeping is among the first presentations of fact as a decision-making tool. In England, in the 1700s, Francis Bacon’s work on the scientific method led the way toward building theories based upon “particulars,” not deduced from a grand theory. Of course, this way of thinking sidelined generally accepted beliefs, a radical idea at that time, and in our times, too.

220px-Thomas_Malthus

Thomas Malthus

You may recall that Thomas Malthus theorized that food supply would not keep pace with population growth. His initial documents were based, mostly, upon deduction. His later documents were based upon well-researched fact. The shift in thinking occurred during his watch, before and after the year 1800.

Portrait of Jeremy Bentham by Henry William Pickersgill

Portrait of Jeremy Bentham by Henry William Pickersgill

Enter Jeremy Bentham, a Malthus contemporary. Bentham’s theory, simplified: government ought to provide “the greatest happiness of the greatest number [of people]” In order to do the job, government would need a clear picture of the people it served (also a new idea, government as service, but that requires another article).

The word “statistics” enters the language around this time: stat, of course, is German for the state.

By the 1830s, the British government is obsessed with this powerful tool: facts. They commission a series of Blue Books filled with facts, statistics, anecdotes, interviews and more. The Blue Books are reports about “poverty, crime, education, and other social concerns.”

Charles Dickens, who made fun of his government's newfound love for facts.

Charles Dickens, who made fun of his government’s newfound love for facts.

By the 1850s, the clever novelist Charles Dickens grows weary of the fact-based Blue Books. From Dickens’ Hard Times, “We hope to have, before long, composed of commissioners of facts, who will force the people to be a people of fact and of nothing but fact.”

By around the 1900s, fact-finding missions had become common, and World War I becomes the first war fought, largely, upon the basis of facts.

At the risk of capturing the obvious idea, our contemporary media environment is skewed because opinion and pontificating is, often, more entertaining than fact-based thinking. Rush Limbaugh gets the ratings; the Encyclopedia Britannica ceases publication. Constitutional scholar Cass Sunstein says, “Many people are mostly hearing more and louder echoes of their own voices.” His context is the internet, where groups of like-minded people share their beliefs, and by their numbers, magnify ideas that may not be fact-based into cultural touchstones. He goes further to explain that members of those groups are becoming less likely to communicate with people outside the group, and wonders whether this supportive groupthink is detrimental to democracy. (So much for the hope that the abundance of information, and facts, on the internet would encourage interaction between these groups.)

And that leads to Nicholas Carr, author of The Shallows: What the Internet Is Doing to Our Brains, who believes that the internet is “weakening our capacity for the kind of ‘deep processing’ that underpins ‘mindful knowledge acquisition, inductive analysis, critical thinking, imagination, and reflection.”

220px-Al_Gore_at_SapphireNow_2010_croppedA parallel path also leads to Al Gore, who asked this question in his book, The Assault on Reason: Why do reason, logic and truth seem to play a sharply diminished role in the way American now makes its important decisions?”

Perhaps the answer to Mr. Gore’s question is complexity. We learn arithmetic but not calculus, we have been taught to think in simple linear terms, not in terms that help us to understand the complex, dynamic system that our society has become. Our contribution to the chain begun by Bacon: the mapping of complex systems that change over time. It is these systems that draw facts into the future, and these models that provide potent vaccination against those who theorize on the basis of beliefs, not facts.

On another parallel path is the passionate amateur. Included in that class would be both Charles Darwin and Gregor Mendel, neither one a scientist, each a relentless cataloguer of observations, and, as a result, a theorist whose ideas are based upon endless study and analysis.

Jenny_McCarthy_at_E3_2006And, there is the celebrity whose role is related to a megaphone. Ideas that might not otherwise reach a large audience become popularized because a celebrity become involved. When former Playboy model Jenny McCarthy and actress Jenny McCarthy started making noise about vaccinating our children, people paid attention. The Michael J. Fox Foundation is attempting to resolve the delicate balance between Mr. Fox’s own story and fame, and the broader agenda that must drive the Parkinson’s foundation (that carries his name). The Fox foundation has been intelligent and thoughtful in its use of social media, engaging individuals, on a large scale, to participate in trials and other research. Here, the “particulars” are the individual cases, the undeniable truth of daily life with a disease not yet cured.

Too Big To Know

How does all of this come together? The fact is, we’re still figuring out the answer to that question. David Weinberger’s book, Too Big To Know, the source of many of the ideas and all of the quotes in this article, does a fine job in raising questions and providing examples. Addressing the crisis of knowledge (his belief, with which I do not wholeheartedly agree), Weinberger suggests that we open up access to a much broader range of facts; link everything in sight; dig deeply into institutions to make their knowledge available to a larger population; and relentless teach so that we all gain a better understanding of how our world works, and how it might work in the future.

BTW: The article’s title, A Fact Can Be a Beautiful Thing refers to love as explained by lyricist Hal David for the Broadway musical, Promises, Promises.

On his way in, Mr. Elbaz meets Mr. Smelie, on his way out

Gil Elbaz’s new company plans to collect, organize and distribute every fact on the face of the earth (and, presumably, above and below it). His company was featured on the front page of today’s New York Times Sunday Business Section.

The sum of human knowledge from a hundred years ago.

Colin Macfarquhar’s old company dates back to about 1770, when Colin and partner Andrew Bell hired William Smelie to put together the first edition of the Encyclopaedia Britannica. Smelie was 28 years old at the time, and he managed to produce, with collaborators, over 2,500 pages in three volumes. This month, after 15 editions, and all sorts of contortionist moves to contain and present an increasing vast store of information, the old EB gave up. No more editions. They’re done. After more than 200 years of successful publication of facts in books, the task became overwhelming as the publication business was overwhelmed by the crowd sourced newcomer, not yet a decade old, called Wikipedia. And the old Britannica ceased publication.

‘Tis a sad day, I suppose, but neither the Encyclopedia Britannica, nor the World Book Encyclopedia (which was always easier to read and more enjoyable to browse because of its reliance upon pictures), nor Colliers or the others, were superior reference tools.

From the NY Timesarticle about the EB’s demise:

William Smelie was responsible for the first Encyclopaedia Britannica, a stunning accomplishment that lasted centuries, but never overcame the digital revolution.

(In) one widely publicized study, published in 2005 by Nature, called into question Britannica’s presumed accuracy advantage over Wikipedia. The study said that out of 42 competing entries, Wikipedia made an average of four errors in each article, and Britannica three. Britannica responded with a lengthy rebuttal saying the study was error-laden and “completely without merit.”

Early in my career, I wrote and researched questions for television game shows, where a contestant’s knowledge (and memory) of facts could be converted into thousands of dollars. We kept several encyclopedias in the office. Each one had a pad above it, where writers and researchers made note of errors. Each pad was dozens of pages long. There were lots and lots of mistakes, some stunning in their stupidity: Paris was the capital of Egypt, that sort of thing, resulting from too many pages being pushed through a manual system at speeds that made sense only to a publisher.

Gil Elbaz, founder of Factual.

Factual’s plan, outlined in a big orange room with a few tables and walled with whiteboards, is to build the world’s chief reference point for thousands of interconnected supercomputing clouds.” Based upon the NY Times article, it would be fair to assume that Factual and EB are looking at information differently–both in terms of process and scale:

Geared to both big companies and smaller software developers, it includes available government data, terabytes of corporate data and information on 60 million places in 50 countries, each described by 17 to 40 attributes. Factual knows more than 800,000 restaurants in 30 different ways, including location, ownership and ratings by diners and health boards. It also contains information on half a billion Web pages, a list of America’s high schools and data on the offices, specialties and insurance preferences of 1.8 million United States health care professionals. There are also listings of 14,000 wine grape varietals, of military aircraft accidents from 1950 to 1974, and of body masses of major celebrities.

There is reason to be confident in Mr. Elbaz’s vision and ability to execute: his prior company, “Applied Semantics software quickly scanned thousands of Web pages for their meaning. By parsing content, it could tell businesses what kind of ads would work well on a particular page. It had 45 employees and was profitable when Google acquired it in 2003 for $102 million in cash and pre-I.P.O. stock.”

So, say goodbye to yet another long-standing institution, acknowledge the intermediary step that caused the change, and welcome yet another new way of thinking enabled by the massive technology shifts that we’ve experienced in our recent lifetimes. Gee, this is moving along quickly!…