Information is not knowledge

Shannon Information Theory defines information in a specific way: Information is the amount of “surprise” in communications. If I gave you a print out of this blog post, covered up part of a word in it, and asked you to predict the word after showing just the first two letters…


You might answer therapist, but you’re more likely to answer


That is a very common word, and easily predictable. Hence, the “the” in this post doesn’t carry much information. Certainly a lot less than the word “Theroux” – who might mean a specific person, like Novelist Paul Theroux.

The most information dense communication would be  string of random characters. You cannot predict the next character from the previous one. But practically speaking, a bunch of random letters are meaningless.

One reason why the storytelling project can better inform the world is because it allows more information to flow from communities, and provides a better way to filter out the noise and help people find the knowledge in all that information. Instead of this:


It allows this:


Normally, too much information is a problem. Evaluators design narrow, specific surveys with tightly defined questions because they want the most knowledge to come out of the least information entered in. They seek to achieve a 1:1 information:knowledge conversion. The top diagram represents the way evaluators collect information with community surveys.

But if you have better filtering tools, you can instead maximize the information flow and rely on better filters to control what pieces of this information is meaningful. You can tolerate noise. You can fetch only the knowledge you need from a ton of information. But the next person with a different need can also retrieve the knowledge he needs. Google search does for the web, and the framingham heart study did this for medical risk factors. So why hasn’t anyone succeeded in doing this for poverty and social problems?

This would allow us to learn without starting over each time. Suddenly one set of information has two uses, and eventually hundreds of users – all because the information “firehose” was opened and the filtering was good.

This is smarter design. Maximum information input plus reasonably good filtering yields more knowledge to more people.

I encourage you to go back and read examples I posted on the knowledge we’ve been able to extract from stories with good relevance filtering.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s