Do you know how much science you don’t know, and will never know? Here is a picture showing how many science papers have been published and indexed to date in Pubmed, Web-of-Science (ISI WoK), Google Scholar, and Scopus. Note the the 20-million peer reviewed papers are a hard number based on NLM/PubMed. The rest are estimates:
How little science makes a splash:
This came to mind tonight as the last chapter of my PhD thesis was finally accepted for publication, after 5 years. That leaves me with about 10 papers among the 50 million, 5 papers among the 20 million, and 3 among the 10 million.
I wonder what this picture would look like for research in the world of international development. Sadly, no indexing or peer review exists, so this kind of picture would be impossible to generate.
Six years later, the prospect of building this map for NGOs is no longer impossible. I recently built a list of all organizations’ websites and could conceivably scan the content of all sites for patterns that look like references to papers. (Searching for PDFs is an obvious start!) Then, given the set of about 10,000 organizations that maintain a public website and public content on it, I should be able to detect the names of the 5000-50,000 most commonly cited documents, in spite of an absence of any central database. This is how google does it. All the content that matters is already online and I have a master list of where to look for it. That doesn’t include all the references to papers people in the sector read and mention in conversations, but I think it is a fair proxy.
Why would I care? Because the papers that get cited most might have systemic biases or outdated assumptions of poverty that we need to know about. Or because we can demonstrate how shockingly rare the spread of good information is among NGOs. And if your organization has a theory of change that sounds like, “we prove X and then publish it, and then others copy X,” you are wasting your time if that last step never happens.
The world bank realizes this is a problem. 31 percent of their research papers are never downloaded and 87 percent are never cited by others.