I've resisted writing about Hwang Woo Suk's fraudulent paper in Science about producing patient specific stem cells. It just isn't really that big a deal. A guy who claimed proficiency with chopsticks was directly correlated with producing cutting edge science -- surprise! -- turned out to be not so credible. He was found out. Science wins. Science will always win, eventually.
Yet the affair provides an interesting context for thinking about the tenuous standing of cutting edge science. Uncovering the fraud is frustrating to those waiting for cures for disease or injury, annoying to those waiting for life extension technologies, and disappointing to almost everyone for whom scientific inquiry is the closest approximation to a pursuit of truth; all true. But guess what? Science is a human institution, practiced by humans with all of their faults. It is simply inevitable that those faults affect scientific results and publications.
But what distinguishes science from other human institutions, notably politics, religion, and business, which have all experienced extraordinary fraud and malfeasance recently in the U.S., is that fundamentally science finds its foundations in the physical world. The progress of science, and its authority, are tied to what is measurable. Moreover, those measurements must be repeatable. That is, a result must be testable and verifiable by others to become accepted. True, uncovering the Hwang fraud took almost two years, but Hwang's fall was inevitable because the requirement for repeatability means that science is self-correcting. It happens that the holes in the original Science paper appeared not because of questions emerging from labs trying to repeat the work, but rather from suspicious aspects of the paper itself, such different figures of supposedly different cell lines containing similar images.
The scrutiny of these images and other details of the paper applied by scientists within South Korea, fed by suspicion of the great height to which Hwang aspired, only strengthens the process of science. We didn't actually have to wait for the results of long and laborious experiments, nor did we have to spend money to repeat Hwang's work. The fraud fell apart under its own weight. This is a success.
I, like every other practicing scientist, have to wonder how this episode will affect the public perception of science. I come to the conclusion that the airing of dirty laundry will only improve the position of science in the long run. There is no other human institution so ruthless in chopping out the dead wood. After all, if you are lying or pulling a fast one, the very last thing you want to do is get a bunch of really smart people trying to catch you out, all of whose professional standing improves if they do.
The public perception of all this is complicated slightly by the fact that there is a difference between the science you read in textbooks, and the science reported in journals or on the front page of newspapers and news magazines. In today's New York Times, Nicholas Wade has a very nice article exploring this issue, prompted by the stem cell fraud:
The contrast between the fallibility of Dr. Hwang's claims and the general solidity of scientific knowledge arises from the existence of two kinds of science - a distinction that is often blurred when new advances are reported first by scientific journals and then by the news media. There is textbook science and frontier science, and the two types carry quite different expiration dates.
Textbook science is material that has stood the test of time and can be largely relied upon. It may include findings made just a few years ago, but which have been reasonably well confirmed by other laboratories.
Science from the frontiers of knowledge, on the other hand, is wild, untamed and often either wrong or irrelevant to future research. A few years after they are published, most scientific papers are never cited again.
I find this latter point the most problematic of the scientific enterprise. Of the papers with short lifetimes, some are not read or cited because they aren't very good or very interesting, some are only minor improvements on previous work, and some fall by the wayside because they describe dead ends. In all cases, very little science that gets done, and even less that is finally reported in journals, actually affects the world in a meaningful way. How can you not feel a bit ambivalent about this? Isn't this emblematic of some sort of waste, inefficiency, or a Proxmire-attracting, willful misappropriation of funds? Emphatically not! This is a cost we must bear as part of the never-ending effort to banish our ignorance and improve the human condition. At both the institutional and the personal level this cost is intrinsic to science. Every scientist, and every technologist and inventor, for that matter, has plentiful experience with choosing the wrong path. Alas, the dominant social structures governing funding decisions and career advancement are based predominantly on the number of papers published, rather than upon their content, which means that often the wrong path, the marginal improvement, and the simply boring result in the lab are gussied up for publication to look far more significant that they really are.
The only real defense against this profusion of craptastic papers is the choice of individuals not to write and publish them. So I have little hope of progress there. Enough said about that.
A weaker, but necessary, defense lies in peer review. In lieu of the sudden popularity of all scientists becoming harsh and discerning critics of our own efforts, we must all keep watch on science as a whole, trying to catch mistakes and fraud before publication. Yet this process, too, is far from perfect. Too many anonymous reviewers have political reasons for rejecting papers, and many more just don't do a good job of reading the paper they are judging. I don't really have a solution to this problem, but I have to wonder if removing anonymity from the review process would clean things up. Yes, you would have the problem that younger scientists reviewing the work of their elders would be exposed to wrath from above. But what we have now definitely needs improvement. Witness the Hwang paper.
Mr. Wade explores the notion of improving the quality of papers through requiring authors to state their contributions to a paper, and by requiring all authors to state their explicit agreement with all conclusions in a paper. I don't have any problems with the latter, but I can say from personal experience that writing an author contribution statement can be extraordinarily painful, a struggle to carve out sufficient acknowledgments of your own efforts and give perspective on another's efforts, particularly when control of the text lies with someone else. Still, it's worth a try. And I support the inclusion of author's contributions for the time being.
Alas, this doesn't help with the review process itself, because it doesn't do anything about biases or laziness of reviewers. Mr. Wade thus incorrectly suggests that clarifying the author's roles in research has anything to do with the decision-making process during review at a journal. Nonetheless, save conflation of the review process with writing and attribution, his conclusion is right on the money:
Tightening up the reviewing system may remove some faults but will not erase the inescapable gap between textbook science and frontier science. A more effective protection against being surprised by the likes of Dr. Hwang might be for journalists to recognize that journals like Science and Nature do not, and cannot, publish scientific truths. They publish roughly screened scientific claims, which may or may not turn out to be true.
And thus we must labor on, and through those labors attempt to keep science honest and thereby produce a better world. Science will always win, eventually.