Every animate and inanimate object on earth will soon be generating data, including our homes, our cars, and yes, even our bodies”— Anthony D. Williams on the back of a big book entitled The Human Face of Big Data

From the dawn of civilization until 2003, humankind generated give exabytes of data. Now, we produce five exabytes every two days.” — Eric Schmidt, Executive Chairman, Google

The average person today processes more data in a single day than a person in the 1500s did in an entire lifetime.

…Big Data is much more than big data. It’s also the ability to extract meaning: to sort through masses of numbers and find the hidden pattern, the unexpected correlation, th surprising connection. That ability is growing at astonishing speed, it won’t be long before Amazon’s ability to dazzle customers by suggesting just the right book will seem as quaint as our ancestors’s amazement at horseless carriages.– Dan Gardner, from the book’s introduction

Clearly, big data is a massive idea. Let’s see if we can’t break it down, if not by components, then, at least, by illustrations of classes and contexts.

The connection between data collection and pattern recognition is not new. In fact, we know the earliest example, which still exists, in book form, in a small, private Library of Human Imagination in Ridgefield, Connecticut. The book is called Bills of Mortality, and it records the weekly causes of death for London in 1664. This data was used to study the geographic (block-by-block) growth of the plague, and to take measures to prevent its future growth.

Two hundred gigabytes per day may not seem like much data, not in the days when you can buy a terabyte drive from Staples for a hundred bucks or so, but collect that much data day and day out, for a few years, and the warehouse becomes a busy place. That’s what MIT Media Lab’s Seb Roy did to learn how his newborn son learned language. The work was done at home with eleven cameras and fourteen microphones recording the child’s every move, every sound. The recording part of the project is over–their son is now seven years old–but analysis of “unexpected connections between the routines of everyday life and how one child learned his first words” continues as a research project.

On the other end of the age scale, there’s Magic Carpet, now in prototype. The carpet contains sensors and accelerometers. When installed in the home of, say, a senior, the carpet observes, records, and learns the person’s typical routine, which it uses as a baseline for further analysis. Then, “the system checks constantly for sudden (or gradual) abnormalities. If Mom is moving more slowly than usual, or it’s 11 a.m., And her bedroom door still hasn’t opened, the system sends an alert to a family member or physician.”

Often, big data intersects with some sort of mapping project. Camden, New Jersey’s Doctor Jeffrey Brenner “built a map linking hospital claims to patient addresses. He analyzed patterns of data, and the result took him by complete surprise: just one percent of patients, about 1,000 people, accounted for 30 percent of hospital bills because these patients were showing up in the hospital time after time…a microcosm for what’s going on in the whole country (in) emergency room visits and hospital admissions…” Subsequently, he established the Camden Coalition of Healthcare Providers to help address this “costly dysfunction.” He collected the data, analyzed it, then brought out meaningful change at a local level.

One of the many superb photographs depicting the intersection between human life and technology use. The book was put together by Rick Smolan, an extraordinary photographer, curator and compiler whose past work includes A Day in the The Life of America and other books in that series.

Yes, there’s a very scary dark side. Bad people could turn off 60,000 pacemakers via their Internet connections. A real time, technology enabled 2008 terrorist attack in Mumbai killed 172 people and injured 300 more thanks to Blackberries, night vision goggles, satellite phones and other devices.

If you control the code, you control the world. There has not been an operating system or a technology that has not been hacked.

Fortunately, the good guys have tools on their side, too. The $40 million Domain Awareness System in Manhattan includes “an array of 3,000 cameras known as ‘The Ring of Steel” that monitor lower and midtown Manhattan as well as license plate readers, radiation detectors, relevant 911 calls, arrest records, related crimes, and vast files on characteristics such as tattoos, body marks, teeth, and even limps. They can also track a suspicious vehicle through time to the many locations where it has been over previous days and weeks.”

Google’s self-driving car is safer than a human-controlled vehicle because the digital car can access and process far more information more quickly than today’s humans.

By 2020, China will complete Compass/Beidou-2. This advanced navigation system will outperform the current (and decades old) GPS system. Greater precision will be used for public safety (emergency response, for example), commercial use (fishing, automotive), and, inevitably, for far more productive war.

Data can mean the difference between life death when the weather turns ugly. Thousands of lives are saved each year by weather earnings in wealthier countries. Yet thousands of lives are lost in poor ones when monsoons, tornadoes and other storms strike with little public warning, an intensifying threat as the planet warms,,,

If you’ve ever wondered what Amazon’s true business is, or why it uses the name of a gigantic river, the answer is big data. Ultimately, Amazon intends to become a public utility for computing services. Take a careful look at Amazon Prime and you will see a prototype. The streaming side of PBS and Netflix are among the enterprises enabled by Amazon’s big data operations.

For FedEx, “the information about the package is as important as the package itself.”

Whether its eliminating malaria or making art, text messaging for blood donors or tracking asteroids, the future will be defined by the collection, analysis and use of big data. It will shape our individual knowledge about our own bodies, our children’s growth and our parents’ health, our collective tendencies for public good, safety, and bad behavior. It will be embedded in robots and intelligent systems that may, soon, control aspects of life that we once considered wholly human endeavors. It is a change of epic proportions and yet, most of us are unaware of its importance.

The book, The Human Face of Big Data, along with its related website and app, provide a useful gateway into this brave new world.