Im keynoting a conference at the Princeton Center for Internet Polic on Big Data on Tuesday. No, I dont know why they asked me either. Heres an outline of what I plan on saying, although the talk has far from gelled.

“How sweet is the perception of a new natural fact,” Thoreau rhapsodized, and Darwin spent seven years trying to pin down one fact about barnacles. We were enamored with facts in the 19th century, when facts were used to nail down social policy debates. We have hoped and wished ever since that facts would provide a bottom to arguments: We can dig our way down and come to agreement. Indeed, in 1963, Bernard Forscher wrote a letter to Science “Chaos in the Brickyard” complaining that young scientists are generating too many facts and not enough theories; too many facts leads to chaos.

This is part and parcel of the traditional Western strategy for knowing our world. In a world too big to knowâ„¢, our basic strategy has been to filter, reduce, and fragment knowledge. This was true all the way through the Information Age. Our fear of information overload now seems antiquated. Not only is there “no such thing as information overload, only filter failure” Clay Shirky, natch, in the digital age, the nature of filters change. On the Net, we do not filter out. We filter forward. That is, on the Net, a filter merely shortens the number of clicks it takes to get to an object; all the other objects remain accessible.

This changes the role and nature of expertise, and of knowledge itself. For traditional knowledge is a system of stopping points for inquiry. This is very efficient, but its based on the limitations of paper as knowledges medium. Indeed, science itself has viewed itself as a type of publishing: It is done in private and not made public until its certain-ish.

But the networking of knowledge gives us a new strategy. We will continue to use the old one where appropriate. Networked knowledge is abundant, unsettled, never done, public, imperfect, and contains disagreements within itself.

So, lets go back to facts. [Work on that transition!] In the Age of Facts, we thought that facts provided a picture of the world — the Book of Nature was written in facts. Now we are in the Age of Big Data. This is different from the Info Age, when data actually was fairly scarce. The new data is super-abundant, linked, fallible, and often recognizably touched by frail humans. Unlike with facts, these data are [Note to self: Remember to use plural…the sign of quality!] often used to unnail, rather than to nail things down. While individual arguments, of course, use data to support hypotheses or to nail down conclusions, the system weve built for ourselves overall is more like a resource to support exploration, divergence, and innovation. Despite Bernard Forscher, too many facts in the form of data, do not lead to chaos, but to a commons.

Categories: too big to know dw