Preprints in the Biomedical Sciences: The Future is Here

IPPA few days ago, fittingly in the context of Open Access week, we had an event to explain what are preprints and how they have the potential to change communication and career development in the biomedical sciences. You can follow the recording here: http://asapbio.org/event/preprints-biomedical-science-publication-in-the-era-of-twitter-facebook. The event counted on the participation of publishers, funders and users; a summary has been posted in The Node and I encourage you to look at it and contribute to the discussion. There is much to talk about in the wake of the event. Here I shall concentrate on a few issues in the context of publications which highlight the momentum for change.

Preprints are not peer-reviewed papers but importantly, they are nothing new as they have been tried and tested in the Physical sciences for over twenty years, with much success. Interestingly, at the time of the event EMBO J published an article from P. Ginsparg, the founder of arXiv, the Physics preprint server, called  ‘Preprint deja vue’   (http://onlinelibrary.wiley.com/doi/10.15252/embj.201695531/full); I recommend everybody to read it and reflect on what it says about the biomedical sciences for it shows us up (my reading, I hasten to say) as a small minded community, with a narrow notion of competitiveness and an overreliance on commercial publishing for the evaluation of our work.

Preprints and preprint servers are a rising culture and are emerging as a good first solution to some of the problems. But we have to get it right. There are two concerns that are often levelled to preprints: the worry about their lack of peer-review and the fear of being scooped. Both fail to recognize the essence of Science and the crisis of the peer-review process in the biomedical sciences. For example, on the first one, it would be foolish not to recognize that an important element fuelling the rise of the preprint culture is the degradation of the peer-review process which, up to now, has been the cornerstone of modern science. Nobody can deny that papers improve with peer-review but it is obvious that from a system of checks and balances and with the connivance of journals, the peer-review process has become –particularly at the high end of the market- a device to delay publications and, in the process, to give reviewers the power to determine the content of the work and, in some instances, use the anonymity of the process to produce unfair allegations and decisions. Much has been written about the anonymity (blind, double blind, open reviews) but it has always worried me that one of the arguments to preserve the status quo is said to be the protection of young PIs from retributions that established peers might launch in the face of the criticisms that might be levelled to them. How come we cannot own what we say and think? What this says really is that the reason for the anonymity is FEAR, a typical situation in totalitarian systems. How come fear is a justification for anonymity? It should give us pause for thought. Where have we taken our scientific culture: people afraid of signing what they believe in? It would be good if we could change the scientific culture, if we could encourage and practice more an open discussion of our work (which let us not forget, goes on in private, in journal clubs and cafeterias). Journals give you the option of commenting and discussing but only after publication. A preprint is there for discussion, so you can comment, openly, and influence its shape and help, rather than hinder, the authors.

But, in the end, we have to ask what is the purpose of peer review? It has been suggested that it is validation of the research and yet, if you ask (and we did at the event) how many people have failed to reproduce a published piece of work, the answer will be loud and clear: many, most. So, peer-review is not validation. It is a form of certification of the quality of work, like that of the rating agencies on credits and, as it was the case in the financial crisis, there are too many subprime papers AAA rated because they are bundled in HIF journals with a few quality ones. We have lost our bearings and it is unfortunate that we value our work not for its intrinsic merits but because of where it is published.

The issue of scooping cannot be separated from the strange contraption that the peer-review process has become. Here, again, we biologists have a very different understanding of a notion (scooping) that highlights our small mindedness. As Ginsparg puts it (see EMBO J above) ““scooping” in the context of biology research appears to mean the race between laboratories working on overlapping” and herein we highlight again that the form matters more than the content. There is little question that posting your work, whether in preprint or peer-review form, gives you priority… if there is anything to give priority for………… In Physics there is no question, again Ginsparg: “Posting work on arXiv gives authors a datestamped priority claim, which is accepted by the community, and gives immediate visibility to authors’ work”. I and many agree. The large number of papers with 0 citations in HIF journals is a sad comment on the huge amount of work that in the biomedical sciences goes into useful pieces of information but largely irrelevant pieces of science. How much money and pain would those authors have saved if they had posted their work in a preprint server! I shall leave you to take it from here but in my mind, there is little to fear about posting your work in a preprint server and much to be gained. In many ways, preprints are the “ultimate open access”. Preprints can create a more democratic, cost effective way of managing science and at the event it was excellent to see Journals promoting them and seeing their value.

 

There is much to work on to get the community to embrace preprints. We were lucky to have exceptional speakers at the event where the Wellcome Trust, as a funder, expressed their support for the culture and explained their own efforts towards it (Wellcome Open Research & the Open Science Prize). Interestingly, shortly afterwards we learnt that EMBO will allow for preprints to be cited as evidence of output (as long as they are accompanied by at least one peer-reviewed paper). I am aware of efforts along these lines in the US where many institutions already encourage applicants for jobs to cite preprints. This is good news because it does unshackle students and postdocs from the handicap of not being able to refer to their toilings when they are looking for fellowships and jobs. Preprints are here to stay. The reasons are many and being increasingly discussed and if you wanted a particularly one, this one was stated by Richard Farndale (Dpt of Biochemistry, University of Cambridge): preprints give us a way to let funders know that the work that they gave money to do, has been done. This simple statement alone, makes the point of what preprints are for and should lead funders not only to encourage preprints but to demand them. It should be the funders and not the journals, who decide whether the work has been done.

There is much being said and written about the current situation which really is an expression of movement. A particularly thoughtful piece by P. Walter and D. Mullins appeared recently in the ASCB: ‘on publishing and the sneetches: a wake up call?’ (http://www.ascb.org/on-publishing-and-the-sneetches-a-wake-up-call-november-december-2016-newsletter/ ). Much to mull over here and many arguments for preprints. The article ends up on a note “The end goal seems obvious: The knowledge that we produce in our publicly funded works belongs to humankind and must not be locked up behind pay-walls— newly submitted papers should be open-access and older ones open-archive. Our real challenge is to find the paths that get us there.  But major change can happen, even if it seems impossible to imagine now” I say, let us use preprints. Not only use them but work, together, to shape the future of biomedical science communication.

Further readings on preprints from this blog:

http://amapress.gen.cam.ac.uk/?p=1635

http://amapress.gen.cam.ac.uk/?p=1415

http://amapress.gen.cam.ac.uk/?p=1434

http://amapress.gen.cam.ac.uk/?p=1239

Expensive or Insightful Biology?: Single Cell Analysis as a Symptom

160727_800px-Musei_Wormiani_HistoriaLists, catalogues and classifications have always been the business of the biological sciences. The nature cabinets of the XVII and XVIII centuries, the collections that occupied much of the XIX century and which fuelled the work of Darwin are good examples of this. Beetles, butterflies, fish, pigeons, plants occupied (and occupy) the time of individuals, often amateurs, interested in Nature. The nature of this enterprise is captured in Umberto Eco’s book “The Infinity of Lists”

When we don’t know the boundaries of what we want to portray, when we don’t know how many things we are talking about (….) when we cannot provide a definition by essence of something and so, to be able to talk about it, to make it comprehensible or in some way perceivable, we list its properties (…………….). We call this representative mode the list, or the catalogue

Indeed: to make something whose limits or meaning we ignore, we make lists, if they are organized according to some criterion (and since Linnaeus but even before, they are) they have the potential to reveal something of the essence of that which is being classified. Physicists and chemists know well how this works: stars, spectra and the elements come to mind. But the level and intricacy of what the biological world offers to the catalogue aficionado is different, probably, boundless. To go back to Eco, it is unclear where the limits of the biological world lie. No wonder E. Rutherford said that one could reduce the sciences to Physics and stamp collecting; he may have had the biological (then natural) sciences in mind and this perhaps is why S. Brenner famously retorted that what Rutherford did not know is that there are some stamps that are worth collecting.

In the History of Science lists have the potential to highlight generalities which allow precise questions to be asked and answered. Physics and Chemistry have been good at reaping the benefits of this activity. In Biology a most famous outcome of this cataloguing is, of course, Darwin’s great work which revealed a principle running through the continuum of transformations that stares from large collections, ordered collections (the word ordered and in what manner the order comes about being important here), of plants and animals. In a different way, the work of Mendel is a culmination of less structured but no less significant collection of lists of the output of many lists; after all, it is seeing patterns in the outcome of crosses of plants that leads to genetics. In all cases the assumption is that if the lists are arranged according to the right criterion, they will reveal an order and, behind that order, some mechanism -in the sense of a causal explanation for a set of observations and not as the usual Figure 7 characteristic of modern biology papers- that will provide an insight into a system. In the end, sometimes, the insight can lead to the manipulation of the system for the benefit of the observer: lists lead to science that leads to engineering which leads to progress.

There is a danger in these lists and it is that they might become an end of themselves. That the scientist becomes a collectionist, forgets Brenner and gives credence to Rutherford. Surely the lists are valuable resources for those that want to ask questions, but the truth is that as we turn into list makers, we can forget that there are questions behind the observations and habit turns us lazy and content in our collecting. Sometimes one feels that this is happening in the biological sciences, that biologists are becoming professional collectionists. There might be a reason for this:  the essence of biological systems is the generation, selection and competitive propagation of novelty and variation. As a result, every species, every genome, every cell in every genome, every organelles in the cell, every protein in the organelle, is subject to this continuous generation of variation, to the exploration of a large space of form and functions. If one assumes that every cell type in an organism is different and that these differences are species specific, one can do a simple calculation: the range of different cell types varies between 3 in a plachozoan to about 1014 in a human and if, as it is currently assumed, there are on the order of 8.5 x 106 organisms on the earth, one could say that there might be on the order of 1020 different cell types to explain (NB this is assuming that all individuals within a species are similar and forfeiting the development of an organism during which large numbers of transient cell types are generated that differ from their final types). This number, 1020 , is already a large number relative to the approximate number of stars, 1012 . It may be small relative to number of atoms, 1080 in the Universe –and one has to remember that atoms need to be proportionately distributed into 117 elements which is where the differences appear i.e all atoms of an element are essentially the same and thus, the 1080 number needs to be tempered by its being bundled up in the abundance of each element. It is here, in this notion of similarity of all the atoms of an element, that the main difference between the biological and physical systems appear. The stars are very similar to each other in composition, and this is why we can study them from a distance by using the spectra. On the other hand, every organism, every cell type in every organism is different, unique. In fact you and I are very similar but our cells in similar places in similar organs are likely to be different. Enter DNA, which is the way to explain uniqueness in Biology: if we accept, as we must, that every cell type responds to a ‘transcriptional code’ of sorts, and we focus just in humans with our approximate 20,000 genes (I am not interested in philosophical discussions of what is a gene and hope that you and I will agree that this is a lower bound), simple calculations allow for 220000 combinations, to account for those 1014 different cells (and don’t forget those developmental intermediates). If you throw this number into your calculator, it will be confused as it will approach infinity. Of course, the toilings of Natural Selection ensure that only part of that repertoire is used but still, the number is large and dwarfs anything the inorganic world can produce. Surely we are stardust, like the moons of Jupiter, but DNA and RNA have found a way to turn that dust into a creative material device.

Where am I going with this? Over the last few years technical developments have allowed us to peer into single cells at the level of their transcriptional complement and, with increasing accuracy, at the level of their genomes. The observation is that even within what histologically is a (one) cell type, there is a great deal of heterogeneity. It is difficult to silence the genome, and we are learning that cells –particularly in development- are exploring their transcriptional space in a dynamic manner. The result is that within an organism much of that space of 220000 combinations is likely to be explored and much of it represented. The technical developments are allowing increasing volume and accuracy in the observation of this process (gene expression at the level of single cells) and of the delivery of these results. In consequence this creates interesting challenges for classifying, for making lists, which are taken on by groups of computational biologists whose interests lie in dealing with complexity rather than in understanding its meaning. Meetings are held on the subject of gene expression at the level of single cells and while at the moment the possibilities lie in honing our ability to describe the expression patterns of single cells and of characterizing the genomes of cells in tumours, the holy grail on the horizon is the analysis of epigenetic marks at the level of single cells and the ambition of getting the genome, epigenome, transcriptome and proteome in single cells. Our infatuation with these techniques, what it reveals and the possibilities associated with it are powerful and thus reviewers and editors lurk in the background to ask you for a single cell analysis of your favourite system, if everything else has failed to hamper the publication of your work. But, at the moment, it is also expensive and begs the question of where does it lead to? What is the meaning of this work? Are we paying lip service to Rutherford?

160727_Untitled.001The analysis of single cell gene expression can have -and sometimes had- an impact in three areas of Biological research: Cancer Biology, Immunology and my area of interest, Developmental Biology, which aims to understand how an organism builds itself. In all cases, single cell analysis allows the identification of ‘rare cells’ which sometimes have a function and sometimes, they don’t. The issue is that more often than not and in the best tradition of Biology, these studies reveal the temptation of collecting data under the banner of its ‘importance’ without realizing that we have fallen to a fad, that cataloguing has taken precedence over understanding. The description of a biological process demands a link between a cellular and a genetic description of the process and there is little doubt that the arrival of single cell transcriptomics and associated techniques, particularly single cell lineage tracing, has revolutionized the field. However we should be careful not to be swayed by the collectionist syndrome and remember that behind the data there are questions and that if we cannot see them, we should acknowledge that. We should not confuse cataloguing and collecting with Science. In some ways there is no great difference between beetles and genes, and we might be developing a XXI cabinet of genes and cells. It might require more challenging techniques than those collections of the past but there is no difference between collecting one or the other. Already papers in journals tend to be divided into two: either analysis of gene X in tissue Y in organism Z, or increasingly, single cell analysis of process W in organism Z. And in the best tradition of classical Systems Biology, one hopes that in the analysis of the data, the question and the answer will emerge at the same time as one stares in hope at the data.

Single cell analysis of expression is the epitome of this strange hypothesis-free science that is often hailed in reviews and social media. We are in the midst of it. Slowly we fool ourselves that large data and cataloguing will lead us to the essence of a process, that it will allow us to talk about something that we cannot define. And while it is true that Biology has a habit of revealing principles from lists I cannot help but thinking that with this trend of hoarding data, we are losing perspective of the processes that still need addressing. It would be good if, as R. Feynman said, we don’t confuse naming something with understanding something. Developmental Biology in particular, is losing itself in this naming game and single cell analysis will –unless checked- provide the ultimate distraction from questions that are there but we are too…..may I say ‘lazy’? to ask. We should not forget that there are things to explain, that cataloguing is a way to answer, and sometimes to unlock, those ‘things’ but also that we need to make an effort to search for them.

The allure of the information that can be gathered in one of those experiments is enormous but one needs to remember that in addition to being expensive and data rich, it needs to be insightful. The difference between a collectionist and a scientist should lie not in the ability to make observations but in the ability of the second to use the observations to answer specific questions about Nature. Biological systems have a boundless ability to generate (constrained) variability and it seems to me that the challenge is to understand the nature of the machine –for it is a machine- that generates, processes and uses that variability, written in that tape that is the DNA, interpreted by the transcriptional machinery and supervised by Natural Selection. It is the process, not its output, that needs to be explained. Questions are cheaper than data gathering but good questions are hard to come by.

Epilogue

One of the most disturbing aspects of the current trend in the single cell field is the lack of cross reference or discussion of the data. Often the same system is surveyed in more than one paper without any reference to the other, related, pieces of work or even, on occasion, to the general problem. While this is in keeping with the current trend in the biological sciences in which the publication rather than the finding is what matters, it is no less disturbing. If we do not get hold of the boundless nature of that data by using questions to clean it up and thus reveal what is good and bad data, we shall do a disservice to the system that puts up the money for that research and, more importantly, to Science itself. Surely, there is meta-analysis, Darwin’s great work can be construed as a meta-analysis- but nowadays, often this is done not so much with a question in mind but with the idea of multiplying the data-analytical power. The boast tends to be not in what has been learnt but in how large the data set is. And in the end, the danger is that, increasingly, what we do is expensive collecting; XXI century cabinets of genomic data, without a good reason, without a good question –which exists. We seem to have relinquished our ability to interpret what we observe and lost our interest in asking questions because, I agree, it is easier to order and catalogue this diversity that we call Biology. Still, the issue buzzes in the back of my mind: there are questions, important questions, to be asked and….all that data!

ASAPbio: The Dusk of Peer-Reviewed Glamour (a report from a virtual attendance)

The issue of publications of science manuscripts is reaching breaking point. Breaking in the sense of tearing down the enthusiasm of young investigators, the patience of seasoned ones and generating a lot of debate at institutional level. A few days ago, a survey by Nature News showed that more than 35% report taking 1-2 years to publish a manuscript and 20% 2-5 years (http://www.nature.com/news/does-it-take-too-long-to-publish-research-1.19320). What the bottom line of the article hid is that the time does not mean time sitting in the same journal –though some times it does- but time from submission to the first journal. For a more informed case study of the topic you should look at several posts in the always excellent S. Royle’s blog (https://quantixed.wordpress.com/); in the context of this post, I particularly like this one https://quantixed.wordpress.com/2014/04/15/some-things-last-a-long-time/. The point is that in science, but particularly in the biological sciences, something very fundamental has changed. Scientific publishing is an evolving enterprise but over the last twenty years it has moved, imperceptibly, from a medium scientists use to report their findings to a consumer good that uses scientists to develop its business model. It could be argued that this should not be a blanket statement but, actually it is pretty much that, particularly in the non-academic tier. Compound this with a sociological trend of too many scientists, disenchantment with career prospects and the emergent importance of media and this explains why scientific publishing has become a source of employment and a business that, as has been repeated many a time, uses scientists at all ends –from the bench to the editorial desk- of its fabric.

The problem is that the essence of the enterprise: finding-report-publication, has not evolved and what we have is a system that, basically, cannot cope with its essential fuel: science. Too much is being produced, all needs to be peer reviewed and peer review has evolved into an interesting sociological contraption that at some point should be the subject of some PhD thesis. The aim of peer review in its current incarnation is not to improve the work (this is the gimmick) but to ensure that a paper takes the longest possible time to be published and that its cost escalates to the point that what is considered good research is, basically, expensive research. And so it is against this backdrop (a much simplified version of it) that last week a meeting took place at HHMI headquarters with the cunning acronym of ASAPbio (Accelerating Science and Publication in biology: http://asapbio.org/). The meeting gathered an interesting mixture of senior scientists, administrators and editors with the aim of discussing preprints and preprint servers in Biology. For those of you who have not heard of preprints (and surprisingly there are many out there who haven’t and people at ASAPbio who acknowledged that they did not know they existed before the meeting!), a preprint -in the context of science communication- is a manuscript which has not been subject to external peer review and which is posted in a site for public viewing; it even has a Wikipedia page (https://en.wikipedia.org/wiki/Preprint), whose first words are “Publication of manuscripts in a peer-reviewed journal often takes weeks, months or even years from the time of initial submission,…..”, it says all.

ASAPbio was broadcast live on the internet and you could follow it on Twitter and social media; some of us did and you can see the videos now at the site (http://asapbio.org/). It was interesting. The gathering was representative of what we could call the establishment of science, principally the US establishment, and gave careful consideration to the culture of preprints and its future. Perhaps the most surprising thing to come out of it is that we have to fight a fear to preprints as forms of unvalidated science, preprint postings as a danger to “true research” and, surprisingly, some of these voices from qualified scientists (yes, the ones who have created and control the system we enjoy –or endure, as you choose-). There were exceptions amidst the old guard. The objection raised by those who worried was that preprints are not ‘controlled’, are not reviewed (forgetting really what lies at the heart of a manuscript at the time of submission), vetted. One had the impression that preprints are some kind of modern samizdats (in case you have not heard this word: https://en.wikipedia.org/wiki/Samizdat) forgetting that samizdats were a good thing in the cold war era. In fact, as a response to ASAPbio some went as far as posting tweets wondering whether what is in preprints is true or safe for consumption!. The answer to this question is simple: read the paper. Furthermore, physicists have been using preprint servers since 1991, particularly in the form of their very successful arXiv (http://arxiv.org/) home to over one million e-prints on all aspects of physical and mathematical sciences. Paul Ginsparg, founder of arXiv, participated at the meeting providing details of its history, value, mechanics and validation of and making clear that in many ways, preprints (e-prints at arXiv) are the bread and butter of the physics community. He also reminded (in some cases revealed) to the audience that arXiv introduced a section on quantitative biology which is proving popular. Along these lines a few years ago under the initiative of John Inglis, CSH started a parallel server, bioRxiv, which aimed at catering to do for Biology what arXiv does for Physics and Mathematics. There are other preprint servers of different flavours but bioRxiv aimed from the beginning to be arXiv like i.e. community oriented, user friendly, data rich. As it was reported at the meeting (and you can read here http://asapbio.org/biorxiv) bioRxiv is succeeding and that many of us –who have been using bioRxiv from the beginning- believe that it is a seed for and leads the way to the future of bioscience publishing.

The problem, for now, is that Biology lags culturally and practically behind Physics and this is reflected in many practical aspects, not only the publication ethos. Biology, as a science, hiding behind mindless collection of data has become more a culture of glamour than of content, of haves and have nots in which a good idea or a good point is likely to go amiss and where many bad ones get too much fleeting attention by virtue of where they are published. The question people most often ask you when you try to tell them about your work is not WHAT is it about, but WHERE have you sent the manuscript or, often, how big is your group. The science of the bioscience is a means to an end and so, it is not surprising that publishers have latched on to it and make the most of it. I don’t want to be misunderstood, there is a lot of good science around, it is just that not all that appears as good science is good science (though all data are useful). And thus it emerges the current tangled web of bioscience publishing. You, like me and many, have felt the dismay of starting the path of trying to publish something in what some people would call a ‘decent journal’. By now it is clear that the demands of peer review are not very different in most of those ‘decent journals’, that the difference between them is that in NCS you get rejections with arrogant and patronizing demands and messages from the editors, whereas in other journals you just get a directive to answer the often arrogant and patronising comments of the reviewers. I shall not bore you with details, we all have our stories. As has been said before, when people finally publish, there is a sigh of relief more than of elation. There are exceptions and I can only cite, within the realm of what I know, EMBO Press and The Company of Biologists. I wished I could cite eLife and PLoS but I am afraid that each in their own ways have failed to live up to their expectations (PLoS ONE has done a lot of good but has become slow and cumbersome as reviewers ask for more and more work, and PLoS Biology has become a place where people go when their papers have been rejected from NCS; in the case of eLife, with much that is good, it is unfortunate that so much power and resources are being channelled to beat NCS at their own game rather than in being really creative –for those who praise their excellent reviewing methods of eLife, you should know that EMBO journal had been using them for several years before eLife.

So, in terms of publication time and effort, things are getting worst all the time (see above). Enter preprints, that apparently dangerous and unsafe way of disseminating science. I must say that to label a preprint as dangerous or scientifically unsafe is to miss the point that publications in NCS can be worst because, as has been pointed out many times having been ‘peered reviewed’ they show the cracks of the system and the pitfalls of the glamour world those magazines promote: Arsenic life, the STAP problem or cold fusion come to mind and, in the case of STAP, don’t forget it led to the humiliation of a world renowned institute and the suicide of a much respected scientist with little consequence to the journal which published the manuscript in the first place. To say that preprints are dangerous is to overlook where the real danger lies and where the pursuit of glamour over science has led and leads. For those who say that preprints will be honey for garbage I shall say that having followed bioRxiv from the beginning, one thing has surprised me: there has been almost zero crank science. BioRxiv has had a steady increase in submissions and all of them -up to now it is possible to follow most of them- are sound science, no less controversial than what is published after peer review. If anything, one of the valuable aspects of preprints –if properly used- is that they represent raw science, before it is tampered with by editors and other scientists, who often have more competition in mind than the advancement of he science. It is clear that preprints in Biology are here to stay and that in the future will be the best way to present your science. I suggest that you go to the ASAPbio site and read through some of the views of the meeting and even if you have time, watch some of the discussions. More importantly, make sure that you use preprints to disseminate your science.

There were two other important issues discussed at the meeting. The first one was whether preprints should be allowed as an evaluation of scientific research. In theory, and in the light of DORA, this is a no-brainer but the discussion is important as the people gathered at ASAPbio represented a significant element of the establishment. Despite some concerns, it was also reassuring that, for the most part, people saw this in a positive light and many pledged not only to accept preprints (in preprint servers) as evidence of scientific quality, but to encourage their citation in job and fellowship applications. Many young scientists will appreciate this and I suspect that we shall see more of this in the future. The second issue was the mechanics and detail of such a cultural change. You cannot move this idea forward without attention to the organizational mechanics of the server, something arXiv and now bioRxiv know well. There were several references to this and it is important that in the enthusiasm of the move, we do not forget this important aspect of the enterprise.

ASAPbio was led by Ron Vale, whose excellent http://biorxiv.org/content/early/2015/07/11/022368.article-metrics served as a fuse for the meeting. There were well known people in the audience, like Michale Eisen, who have been doing a very open campaign for open science. However, while Eisen is good as a ramming bat against the establishment, we should not forget the many who in a quiet way have been adding to the momentum of preprints by convincing students and postdocs of the value for their careers and science of preprints servers. We need the chief scientists and administrators on board but, more than that we need the inertia created by people submitting their papers to preprint servers commenting on those servers (and don’t forget that you can comment in PubMed through PubMed Commons). It is clear when you come to think of it, that not only there is nothing to be lost by using e-print/preprint servers, but much –all- to be gained, particularly if you are a young Scientist. What is good for physicists cannot be bad for biologists.

ASAPbio was a formal recognition that something is changing and hopefully a formal start of a much needed move into new directions. I can see the worries of so called major publishers about the watershed that is coming, they might lose their stranglehold on the Science they own and shape, but a date comes to mind, 1789, and a voice that reportedly said “let’em eat brioche”. There will be no blood here but much needed change and a recovery of what is scientists’ privilege, the control over their work. I get the impression that something is moving. I sense not the dawn of a new era but the dusk of a short lived one based on much that is wrong and that we need to turn around. As it is being said these days, make sure that your next paper goes to a preprint server as it goes, or before it goes, to peer review. One senses that this is the way to solve, not only the emerging tyranny of journals but also peer review itself.

The case of the Irish Elk, a parable for the weight of the glamour journals

The case of the Irish Elk, a parable for the weight of the glamour journals

Irish Elk 2In one of his wonderful and educational essays, SJ Gould discusses the story of the Irish Elk, a spectacular species of elk that became extinct because……well, it is unclear why but the late specimens did have a very visible trait: enormous –and I mean enormous- antlers; the elk was over 3 meters tall and had antlers 3.3 m across. There have been many theories to explain the mysterious extinction of this magnificent animal but the one Gould discusses and the one I like to think about in certain contexts is that the Irish Elk was brought down by the weight of its own pride. The speculation goes that selection was in action for bigger and bigger antlers which, in the end, brought down –literally- the elk. And for selection, read in many instances, sexual selection. I am aware of the controversies associated with deciding whether selection is involved in a process or not and, more so when sexual selection is involved but, have always been interested in the Irish Elk as a parable from that perspective.

As a practicing biologist these days it feels like groundhog day in certain issues, particularly that of publications/glamour journals/Impact Factors (IF)/evaluations and the like. By now we all agree, more or less explicitly, that the biological sciences (I can only speak about them) are in a crisis because of a change of emphasis: what matters is the publication and not necessarily the research. Of course, there is some correlation between the two and so called high IF publications tend to publish more appealing reports than others, but it is difficult to accept that ‘more appealing’ means ‘better science’. In fact how we measure good science is something that is rarely debated outside the arena of the IF/h-indexes and related metrics and perhaps we should reflect upon this and try to return to value science for its intrinsic value, for the question that the scientist asks. I see few debates about what is a good question, what are the important questions but, in the context of my comments about the “LMB hut” probably this –questions in the biological sciences- is another issue that should go down to the fossil record of the history of science. Be that as it may, in the midst of the latest storm about how to deal with journal glamour (the latest idea to remove journal titles from websites) it is difficult to feel optimistic about any change soon, though it is clear that change is needed and, as I say, not only in our appreciation of the just value of scientific outputs, but about the actual value of the science we do. But…sorry, perhaps inevitably, I digress….allow me a thought.

Maybe the NCS’s (Nature Cell Science for those who are not familiar with the acronym) and the likes, there are some crude imitators around –are like the antlers of the Irish Elk. They are useful in mindless combat, they have a selective value, but as they grow they become more important than other body parts and, above a certain size, they will bring the organism down. I suppose the only response to that would be to grow a body size that keeps up with the size of the antlers. This did not happen in the case of the elk -though their body mass did increase and they were formidable specimens- and certainly cannot happen in the case of the biological sciences. The impression of many is that the kind of pressure that exists to value publications is distortive, creates serious problems for the development of the biological sciences and is certainly affecting the development of careers (the antler v the body and the long term survival of the organism). The question is not as simple as some people would make it sound and this is why nowadays, at meetings, there are entire sessions devoted to discussions of the issues associated with this topic. The problem, I think I have said it before, is that we are running a XXI century enterprise with a mid XX century business model, one that catered for a smaller, more focused community, a content centred enterprise with a smaller constituency. Today there is too much, too much that is good –at least technically sound- and a very large constituency. We need to evolve. Unfortunately the way we are doing it now is by selecting for bigger antlers without thinking about the consequences. There is too much talk about the form (publications) and very little about the content (science) and, slowly we are forgetting what this is about. Look at the indexes of most journals and have a think. The mantra that the science has become the publication is true and, because of its nature, the biological sciences will lend themselves to this gimmick because you can always find a new gene, a new function for a known gene, a new cell, a new drug, a new technique, any of which will be hyped by the impact department of any of ‘those journals’. No wonder some of us often ask if there are any Questions left.

The main problem with, let us call it, the IF question, is that it is breaking up the biomedical sciences into two: those who can afford to publish in certain journals and those who can’t. It is not only about science and ideas, it is about whether you have the stamina and the resources to deal with the whims of editors and reviewers. As it has been pointed out before, the editors have lost the plot and they will ask for bigger antlers (experimental responses to reviewers’ comments) that add very little to the content of the paper, propagate the myth of the specific journal as a tough place to publish and conflate the antlers with the rest of the body. Of course, not everybody will be in a position to respond in kind to the reviewers’ comments, to grow bigger antlers. The consequences of this are dire in the short term though I am convinced that in the large canvas of history the system, like the Irish Elk, will be extinct (don’t forget that this is an evolving system) and in the future we shall look foolish from the perspective of a more sensible science adapted to the times and to the people.

The good news is that slowly, and certainly in Great Britain, I begin to see some sense emerging and while there are still some old fashioned colleagues looking at the publication, more and more are realizing that in this manner you select, mostly, for antler size. If that is what you want, go ahead, grow your antlers and, on the side of the panels and the editors, pick your elks. Content, Science is something else.

NB SJ Gould essay follows an article he published: Gould SJ (1974) The origin and function of the bizarre structures: antler size and skull size in the Irish Elk. Evolution 28, 191-220.  The picture is a modified version of a picture first published by JG Millais in 1897, often reproduced in the web and shown in Gould’s essay: Natural History. 82 (March): 10-19 which you can read in Gould, S.J. 1977. The misnamed, mistreated, and misunderstood Irish Elk. Pp. 79–90 in Ever Since Darwin. W.W. Norton, New York. The person at the bottom of the picture could be construed as a panel member looking for some substance that can keep the elk up.

 

Publish: What? Why? Where? How? – Part II: Solutions?

These are notes for a lecture given by AMA in a workshop about Responsible Research held at LMU in Munich (Germany) on 24 July 2014 (www.responsibleresearch.graduatecenter.uni-muenchen.de/index.html). The lecture is broken into two parts, the first one dealt with biomedical publishing, its origins and current state. This is the second instalment on solutions. Videos of both the lecture and the subsequent panel discussion are available at www.responsibleresearch.graduatecenter.uni-muenchen.de/presentations/videos/index.php 

The problem is, to a certain degree, clear. Let me recap. What was conceived as a way to communicate between scientists and between scientists and the public has become a measure of success, a ruler of quality and an arbiter of professional development. The change in character has altered what science is, how it is done and how scientists are evaluated; too many papers, difficulty of separating the wheat from the chaff, limited funds….Nowadays to do good science is not good enough. To survive in science one needs skills like being able to ‘sell’, being a good story teller, being able to chat up editors, being savvy on more than the subject matter, and then combine these skills to get money. Furthermore an increasing blur between data and thought, between science and accounting creates a climate of confusion in which money talks. Spin and good PR are as important as deep science and enquiry. And an important reason for this is that what we are working with is a XIX century system that has never adapted to the times. What we have, as I have said before is a collection of XVII century tulip bubbles about to burst –the glamour journals- sustained by a few. But there is change coming (I think) and we need to support it and make it grow. Here are what in my view are the important elements that are leading this change

First, Open Access. This is a most important development and one that for the most part has achieved its goals. You have heard a great deal about this so I shall not dwell on it. Open access is a natural response to the attempt of several publishers to own your/our work, to the fact that science has become a business for the journals but it is our toilings that they work with. Open access is not free publishing but it is rational and sensible publishing and it is good to see that funders of science have rallied to support this move. Everybody should publish Open Acess. While everything is good here, we should not lose sight of the fact that the big publishers have seen the goose of the golden eggs and have rallied to produce their own Open Access journals that take advantage of their brand names to make more money. And this takes me to the next all important issue New Journals

Partly because of the publishing niche opened up by Open Access, partly because of the increasing demand for space to publish driven by an exponentially increasing output, new journals emerge every month. Do we need them? How do we decide where to publish? Are these journals changing anything or are they mere derivatives of what we already know? The latter is often the case and clothed with the mantle of Open Acess there is a barrage of faked and real journals which tempt us with more or less success.

There are however positive exceptions which are actually trying to move away from traditional models and aims. At the forefront is a new journal called eLIfe; you have heard about it earlier today. It is an online only journal with much to be commended for. I am particular fond of their reviewing procedure and many technical aspects of how papers are presented, the kind of discussions it posits and the support it has from three heavy weights of research funding (Wellcome, Max Planck and HHMI) which makes it a statement of intent. It is a scientists journal which is trying to carve the future. But…….they have a problem, namely that the people who run the journal are the same people that brought us Cell, the rightly maligned impact factor, who review and publish in Nature, Cell and Science and who control where you publish, how you publish and whether you get a grant or not. How can you implement change with the people who created, and still favour, what you want to change? I guess the answer is with difficulty. Furthermore, they promote openness and yet, many letters of rejection are signed by their chief scientific editor, Randy Shekman, independently of the field, because the editors in charge want to remain anonymous. This is not a good advertisement, But remember what I told you, most of the people behind the journal are the same ones who have created the problem that eLife claims to want to solve. So, while I applaud what they want to do, they still have work to do. Also, the stated aim of the journal is to compete with the glamour journals, but to do so in a fair manner, by attracting quality. But here we hit another problem and this is the difference between quality and cool and eLife, inevitably, is so far a mixture of the two with a heavy dose of cool. But I do not want to knock them down because they have much riding for themselves and have an opportunity to do something transformative and help us move forward. Take these comments as a recognition of their toilings and their openness to new ideas. They can succeed but they need to be bold, really bold and not fall sleep in their coolness. And it is for us to make sure that they deliver.

I mention eLife because it is, in my mind, the most interesting project but do not forget others. Most interesting amidst these, the loved and hated PLoS ONE which, as anybody who has published in it knows, is not just a place where you pay and publish (there are many of those). It is a serious journal in the spirit of publishing sound, rather than soundbite, science (notice the irony! I have often heard as a criticism of PLoS ONE that it ONLY wants to publish sound science……….). Sure it is a mixed pot but I have never had a paper published there without proper revision and their acceptance rate is around 70%. It is a pioneer. Other journals have emerged as derivatives of mainstream products and thus the Company of Biologists have Biology Open and Nature and Cell Press continue to expand their portfolio to fill their pockets with Open Access journals. So, the choice today is very large. My advice here is simple: publish where you feel it is more appropriate. Do not waste your time in the lengthy and morale-sapping process of peer review in the glamour journals. It is not worth it and certainly not worth your time. And yes, wherever possible use scientist based journals.

Untitled.1-002One interesting development in terms of ‘new publications’ is the emergence of preprint servers. This is a notion that comes from the physicists’ arxiv (http://uk.arxiv.org/), which has been running successfully for over twenty years (founded in 1991). In fact arxiv represents the main avenue for publication of new findings for the physics community and they do not worry too much about impact (the biosciences flavour); what they worry about is precedence and being read and discussed. How does it work? When you have results that amount to a manuscript, you prepare them and post in arxiv; you get a doi and you wait for comments or simply mature the work (you can upload new versions of the manuscript). In the meantime people can see the paper. Then when you think it is ready for ‘official’ peer review, you submit it to a journal, and most journals, including Nature and Science, accept papers that have been posted in arxiv (http://en.wikipedia.org /wiki/List_of_academic_ journals_by_preprint_policy). There are good reasons for this: Nature and Science are journals where physicists publish, as arxiv is a central element of communication for the physics community, Nature et al have to accept the rules of their game. A lesson here: Nature and Science have to accept the rules of the scientists and notice that Cell press does not like arxiv. There must be a good reason for this: it is a publication. Over the last few years, biologists with a more quantitative inkling have begun to use arxiv, and this has led to a new section in the journal on quantitative biology. As a response to this interest of biologists in using arxiv, and as a way to rally the more traditional biology community, a few months ago BioRxiv (http://biorxiv.org/) was launched, a biological version of arxiv. It is working well, has not yet gathered momentum, but it hails a cultural change and I very much encourage you to use both arxiv and bioRxiv. There are other preprint servers, as they are called e.g PeerJ, Figshare and F1000. So, again, you have a choice.

Untitled.3-001Why use Preprint servers? There are many reasons and I have discussed the matter before ((http://amapress.gen.cam.ac.uk/?p=1239) but here you have two which should be of interest to you. One, because it gives your research quick visibility and establishes precedence. Two, because it gives you a doi and with it, the ability to refer to it in applications and also in papers. As I say, this is the bread and butter of the physics community and I do not understand why it should not become ours.

And publication rolls on to the next topic, an all important modern classic, Peer review, which echoes much of what I have said in the first part, so I shall be brief. Few people would disagree that peer review in the biomedical sciences is in crisis and changing it should be the next target of our community after Open Access,. Peer review is not doing the job that it is supposed to do. At the moment, the main role of peer review is to make it difficult for you to publish and the degree of difficulty is proportional to the perceived glamour of the journal and, in the journals where this index is high, inversely proportional to your interactions with editors and members of the editorial board. Remember the remarkable advice from the Cell Press editor I mentioned earlier. Anybody who believes that peer review does a good job is dreaming. Please do not think that I do not want peer review. Nothing further from the truth but, I do not want a surrealistic process which has so much element of chance and cliquishness.

There are no easy fixes here. The peer review is the community. We do need peer revew, but we need a system which avoids the extended essays and legal arguments which bedevil the system at the moment. The longer you are in this business, the less you trust the system. These days the collection of reviewer’s comments and replies can be longer than the paper itself! Some journals, like EMBO J and eLife have interesting leads that should become common practice: one, and only one, round of review (both of them and extending) and, in the case of eLife, comments reduced to 500 words (one page, if you want to be generous). There is no reason why one needs more space to assess a paper. A review is not about the paper the reviewer would like to write with the data and resources of the reviewee, but simply a comment on the paper. You will soon see what publishing a paper in a glamour journal means and you will not like it. Perhaps the worst thing about the process is the number or unnecessary experiments and their cost which do not advance the paper (on this see the almost classic : http://www.nature.com/news/2011/110427/full/472391a.html). A few comments and an editorial decision is what should happen. As we have seen this is what it used to be and maybe we should go back to go forwards. I wish the examples of EMBO J and eLife would be followed.

Then there is pre- and post- publication peer review. This is, like anything to do with this issue, a huge subject so I shall summarize. Many people advocate for postpublication peer review and to a large degree this goes on, in private, at lab meetings, tea rooms, discussions at meetings. But there is no will to do this openly. Most journals have now, routinely, places for comments which are not used. As we heard earlier: why are happy to write an impromptu review of a restaurant, a hotel or a book but we cannot do it with a piece of scientific work? What is wrong with bioscience? The only public comments that are allowed are positive. It seems to me that the notion and escalation of anonymous peer review has much to do with this situation. What we have is fear of backlash from criticism. The STAP case is a good, positive example of the cleansing power or open discussion of results. There is also a case for prepublication peer review but for these there are venues: preprint servers. As usual, for all the talking what happens is less than what one might expect. Preprint servers are not bursting at the seams. Lots of talk, but less action. The reason is because, for the most part, the scientists that make this tick are sensible and to prepare a manuscript takes time and care. The so far limited use of those servers, and the limited interest of bioscientists in them, should also be food for thought but, in any case I repeat my advice: use them!

Untitled.005-001And, of course, we need peer review. But we need to recover some sanity. We need responsible editors who do not fall prey to the endless sequence of reviews which cost money and careers, and we also need sensible reviewers who understand what reviewing a paper is and, especially, that it is not a way to block a piece of science seeing the light. We need to remember what a scientific article is and remember that it is the work that matters, and if you want to see more of the future let me tell you about another important development: San Francisco Declaration on Research Assessment (SFDORA or DORA for short).

In 2012, at the annual meeting of the American Society for Cell Biology, a group of publishers and funders signed a declaration, SF DORA, in which there is an explicit statement for “a pressing need to improve the ways in which the output of scientific research is evaluated by funding agencies, academic institutions, and other parties” (http://am.ascb.org/dora/). The main point that DORA wants to press home is that people should be judged by what they publish and not where they publish. To date the declaration has been signed by over 10,000 individuals and over 400 institutions. My advice is sign and, more importantly, enforce it. Make sure that those who have signed it, abide by it. What DORA says is obvious and it is surprising that it needs to be said.

And so to the future

The point of this lecture was to discuss publishing in the biosciences, why you might want to publish, what you would like to publish, how would you like to publish it and also where to publish. We have seen that answering these questions is more complicated than you think at first sight. There are two main reasons for this, the first one is that publishing today is not just a way to reveal and share our work but rather a complex action which will impinge on our future and job prospects, and one that is not straightforward. Together with this there is the fact that, in contrast with the science itself, the structure of scientific publishing has not changed much over the last hundred years and that therefore we are working with a system which is not only overloaded, in terms of supply and demand, but also, simply, not fit for purpose. As a consequence of this and, really, because of the impact where you publish has on your career prospects, there is a very open debate and scrutiny of the publications system and associated issues. There are many ideas around as to how to change it, and change is happening –slowly- but it is happening. So, where is all this going? To predict the future is always difficult but to close I would like to enumerate a number of elements of the tangle which need to be addressed for change to happen

1.The system of assessment of our work by the journals has to change. The way papers are reviewed by editors and reviewers and the roots of high rejection rates need to be reviewed. Some creativity and leadership is needed here and I would suggest that eLife and EMBO Press have made interesting steps in the right direction about this.

In this context if a reviewer wants to have an extended discussion with the author i.e exceed a certain length in the review, and suggest new experiments (which mean money), the identity should be revealed and the author of a manuscript should be able to have a proper discussion of what is being suggested to see what is reasonable. Like many I suspect that having to reveal the identity will change the tone and the content of the review.

2. Journals should implement seriously their policies of ‘conflict of interest’ and at this, they should make sure that their editorial boards do not have members that are also in multiple editorial boards of competing journals. This will ensure a commitment of the editorial board members to the particular journal and will, also, expand the group of people that make decisions about the content of those journals.

3. The journals have to adapt to the times. Supplementary Material was another added on to the operating system and as could have been predicted, it has got out of hand. Journals like Nature and Science now publish incomprehensible short papers where the actual science is in the supplementary. The reason why the text of many of these papers is gibberish is because it has been fragmented in the reviewing and publication process. Journals need to find new ways of conveying the science and a way to normalize the size and content of a paper i.e they need to adapt to the times.

4. Science is made by the scientists and it is for the scientists. There is a trend, promoted in particular by Cell Press, of dumbing down the science, of favouring impact and headline over content. Of favouring certain authors who frequent meetings and editors. Pieces of research become stories and journals rather than reporting advances, tell stories about genes and proteins and it is how you tell a story, rather than the content, that begins to matter. This could not happen in Physics but biology lends itself to this (and if you don’t believe it remember Rudyard Kipling’s ‘Just so stories’ some of which, with added materials and methods, could work well in Cell Press.

Journals need to take us seriously, we need to take the journals seriously.

5. If journals, particularly the generalists, want to keep their clout, they have to be more stringent on how they choose and train their editors. Teaching how to be an editor rather than learning it on the trot should be compulsory. In an increasingly professional world it is surprising that the main qualification to work on those journals (Nature, Science, Cell) is not to like, be bored with or have shown inability in the subject matter they are going to decide on. Imagine that the main reason to become a Chef is that you are bored with cooking or do not have a palate…… There should be a form of professional qualification for scientific editors e.g a masters on the subject (maybe there are). The consequences of the way editors are selected are clear for all to see. This can and should change.

But in the end, and for change to happen, WE have to make choices. At the moment we have chosen to emphasize form over content and science has become a routine mixture of salesmanship and one-upmanship, a power game where a few have an advantage. There are lots of meetings and lots of papers, but (with some exceptions) very little discussion and certainly open discussion. In public, all papers are great, all findings are exciting and there are lots of breakthroughs. In private, the picture is different. Part of the reason for this is that we have substituted questions and science for data presentation and therefore there is little conceptual to talk about. There is also the problem, allow me to emphasize, that the journals that drive the most visible biosciences are run by people that are inexperienced at both science and publishing. My advice is always, to avoid those journals. As long as we continue to search for their endorsement as a proxy for scientific quality, we shall be delaying change and we shall continue in a path towards forgetting the heart of the scientific quest.

The future belongs to you, so: make sure that you see it before it catches up with you. Nowadays the form of science is as important as its content and for this reason you should make sure that you shape it, that you don’t let the shaping to others. Open Access has become normal, let us make sure that other aspects that need change also become normal. Move away from glamour journals and towards journals that are led by scientists, use preprint servers, be open, careful and mindful when you review papers and grants (don’t do a review that you don’t want to see yourself), implement DORA. Let us make sure that science gets back to a normality adapted to the times.

PS: I was about to post the second part of the lecture when I heard the news of the suicide of Y. Sasai. This needed some pause and reflection. To many of us who worry about the current trends in the biosciences, the STAP affair has highlighted much of what is wrong about our enterprise and this is why I dedicated a part of the lecture to it and so did some of the other speakers. Whatever happened and whatever the reactions and possible consequences nobody would have thought of this end (so far) because a human life, and the blurring of a tremendous scientific reputation built over years of hard work, is not a prize to pay for the search of notoriety gone awry. Reflecting on this we should also remember that the chase of a publication in Nature, Science or Cell has many casualties we rarely hear about in the form of careers, enthusiasm, personal lives….while science was at the heart of the STAP affair, it was the search for the limelight that was its Achilles heel and, of course, Nature was only too happy to collaborate. Their job is to sell their product. It would be good if, as a community, we would think about how this came about and take corrective measures because otherwise our endeavour will lose its meaning, if it has not lost it already.

Additional miscellaneous reading

Kreiman, G. and Maunsell, JR.  Nine criteria for measuring scientific output. Front Comp Neurosci.5, 1-6.

Pulverer, B. (2010) Transparency showcases strength of peer review Nature 468, 29-31

Segalat, L., (2010) System crash EMBO Reports 11, 86-89.

van Dijk, Manor, O. and Carey, L. (2014) Publication metrics and success in the academic job.  Curr. Biol. 24, R516

Vale RD. Evaluating how we evaluate. Mol Biol Cell. 2012 Sep;23(17):3285-9.

Vosshall, L FASEB J. 2012 Sep;26(9):3589-93. The glacial pace of scientific publishing: why it hurts everyone and what we can do to fix it.

Williams, E., Carpentier, P. and Misteli, T. (2012) Minimizing the “Re” in Review.  J Cell Biology 197, 345-346.

Publish: What? Why? Where? How?

Untitled.001-001

These are notes for a lecture given by AMA in a workshop about Responsible Research held at LMU in Munich (Germany) on 24 July 2014 (www.responsibleresearch.graduatecenter.uni-muenchen.de/index.html). The lecture is broken into two parts, this one deals with biomedical publishing, its origins and current state. A second instalment on solutions will follow. Videos of both the lecture and the subsequent panel discussion are available at www.responsibleresearch.graduatecenter.uni-muenchen.de/presentations/videos/index.php 

The answers to the title of this talk should be obvious. You want to publish your work in the most appropriate journal/place so that people know what you have done, use it in their research and ponder the consequences. As we shall see, like much of the biomedical sciences (and it is this that I shall refer to when talking about publishing), the answers are less straightforward and you should think about every one of them. None of this was a consideration for F Miescher when he discovered nuclein (later known as DNA) and in 1869 wrote a paper which he submitted to the journal of his boss Hoppe Seyler, the Hoppe Seyler Zeitshrift fur Physiologische Chemie. Hoppe Seyler found the finding of P and N in an organic material so remarkable that he refused to publish it until he had observed this himself; now this is proper peer review. Two years later, satisfied with his own experiments, he published the paper –and one of his on the subject to support the observation. Much to ponder here –notice that he did not scoop his pupil- but I will leave you to think about it. The delay, the publication, the interactions with Hoppe Seyler did not have much of an impact on the career of Miescher.

How much things have changed! Today biomedical sciences publishing is, like many other aspects of the field itself, in turmoil; subject to debate not just in the scientific arena but also in magazines and papers. The Economist and the Wall Street Journal, not to mention the New York Times and The Guardian, periodically raise issues about scientific reports, in particular the biomedical sciences. The reasons for this media attention are complicated and you have heard about and discussed some of them earlier in the day but essentially boil down to a complicated tangle that has emerged between publications, jobs and money. Matters like peer review, the process whereby a scientific report is judged to be suitable for publication, have become under scrutiny and perhaps it is not surprising that last year one can find over 1500 and rising peer reviewed (notice the irony) publications on peer review in PubMed (where there were less than 500 ten years ago), with many thousands more in blogs and comments in journals. Why? The main reason is that, as we all know, a publication today is not so much about reporting progress on experiments and findings but about jobs, about grants, about careers, about upmanship. Publications have become, without us realizing it, the token whereby we are judged and ranked. There is a rather poisonous knot here that I shall try to untangle in the second part of the talk. First, let us get some perspective on peer review, the axis of modern science reporting.

It surprises me that we use papers like the famed Watson and Crick manuscript on the structure of DNA in Nature (by the way there were two other papers in that issue, one by a Rosalind Franklin) as a reference of something wonderful which, if possible, should be imitated. That paper would not have passed the editors desk nowadays and the famed sentence at the end: ‘it has not escaped our notice…..” would only have served as a ticket for the editorial rejection paraphrased as “it has not escaped our notice that while this is an interesting speculation, it would be important to get some experimental results to support it”. Sure, the topic was hot but, under today’s editors, the paper was a speculation on somebody else’s data and should be published, if at all, in the hypothesis section. Whereas in the 1950s science publishing was a small business catering for a small science community, today science is a very large enterprise and, I would say, publishing has not kept pace in the right manner. The fact is that today, the whole publishing business is a tangled web that results from growth without design. What do I mean by this? How did we get here?

Untitled 22.002-001There has always been some control over what people wanted to publish, and before the Renaissance, and also afterwards, the catholic church had that power. After all this is what Copernicus and, most famously, Galileo had to endure if their papers, books at that time, were not to the liking of the church. Copernicus was most careful but Galileo was obliged to recant his beliefs in public under the threat of torture; a rather stern form of rejection. Think about it, today your paper is not published, in those days if the reviewers did not like the paper you could lose your life. There is some progress!.

In more enlightened countries where the catholic church was not such an obstacle, Science (Natural Philosophy as it was called) was flourishing and some learned societies emerged to cater for this. Amidst these, the Royal Society of London. More of a gentleman’s club in the beginning than a society, it was a place in which people with a Science inclination got together to discuss what they were finding. People got into the habit of writing up reports and the Royal Society produced a repository for those reports: Philosophical Transactions of the Royal Society (1665) which is still being published today. Nothing new here, as at the time there were many different places where people could publish their observations and disquisitions e.g Acta eroditorum and Miscellanea curiosa. The Philosophical Transactions was another one but one which attracted the best in Europe. For example, it was here where Leeuwenhoek published his letters on what he saw down his lenses. The procedure was simple: you submitted a report, a scientist had a look at it and put it to print to be read by the people who were interested. It was here that, as the number of reports increased, the first hint of peer review appeared in the form of people, associated with the editor, who had a say on what had been submitted; basically making sure that it was not silly. But peer review in the sense of an external person to the journal checking the work was first used by the Medical Essays and Observations of the Royal Society of Edinburgh (1731). This kind of light touch review, mainly through the editor or someone very close to the editor of the journal, was adopted by the modern Science journals in the XIX century and this is the way in which Miescher and Mendel published their works. Another way to publish was books and this is, of course, how Darwin published his great work. No, his book was not peer reviewed though he did have what today we would call postpublication peer review. And a lot of it. Can you imagine what would have happened if Darwin would have had to subject himself to word limit, supplementary material and, above all, peer review?

Untitled 22.003-002And so, by the turn of the XX century the industry had not changed that much and the small enterprise that was science put out their findings in small journals, run by editors with an interest in science. Nature had been born in 1869 and the Elzevir family began publishing books in the XVI century becoming Elsevier (home to The Lancet and Cell) in 1880 and Science was founded in 1880. The bunching around the late XIX century is not a coincidence, it is a reflection of the rise of journalism and, within this, of the interest to get science to the public. The function of those journals was not to be the arbiters of a scientists career, as it has become today, but to get the scientific advances to the people. This is what hides behind that emptiest of lines in a rejection letter from Nature or Science: “your work is not of sufficient general interest….”. A vestige of a past in which, unlike now, such journals did sell to the people. In any case, in those days scientists had more specialized forums for discussions amidst themselves. It will not be a surprise to you that the mechanics of a scientific publication was different in those days.

Over the last few months, in the tangled discussions of Peer Review and as an act of rebellion, a famous incident involving Einstein is often quoted in the context of the negative effects of peer review (for a proper account of the event see Sean Carroll www.preposterousuniverse.com/blog/2005/09/16/einstein-vs-physical-review ). Einstein had submitted a paper to Physical Review on gravitational waves and the editor took the then unusual step, of asking for one referee report as he thought, rightly it would appear now, that there might be a problem with the work. When he sent the comments to Einstein for correction, Einsten’s reply was harsh:

“We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorized you to show it to specialists before it is printed. I see no reason to address the ”in any case erroneous” comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere”

Untitled.002-001And this he did. The point of the story is not the one people often quote it for. The point is that Einstein was right at being surprised to have the paper reviewed, because in those days, papers were not reviewed; certainly not in the manner that we do and understand today. Editors needed papers, often commissioned papers, and while they might make editorial comments, they certainly did not share the papers with anybody else. This type of review applied to the Watson and Crick paper and to the plethora of papers that form the basis of molecular and cell biology. However, as the number of scientists and papers increased and the material for publication began to accumulate, together with specialization, we start to see something more like the peer review process of today. Nature introduced a version of it in 1953 and implemented the seeds of what we have today in 1967. But The Lancet only introduced it in 1976! And with this small step, of a fair system to control the quality of what is published a complex machine is set in motion that today is a very complicated business, which can keep your paper going around in circles for up to two years in the high end of the market….the notion of high end of the market is, also a new development….basically, all has gone pear shaped. For a nice account of the life span of an average paper see S. Royle’s blog “some things last a long time” (http://quantixed.wordpress.com/2014/04/15/some-things-last-a-long-time/)

Untitled.003-001Many of us complain about the influence that editors have on the fate of the papers that we submit to them for publication. However, if you stop and think about the brief historical perspective I have given, you will appreciate that the relationship now is the same that it was at the end of the XIX century: editors decide what is being published. So, what is the difference? A number of things. The first one is that where the editor needed to find papers, nowadays they want to get rid of papers. The other one is that, in the journals that we could call ‘general interest’ where the editors were enlightened individuals with interest and knowledge, today we have basically a whole bunch of poorly trained scientists acting with a lot of power in their hands. The editor scientist has moved to more ‘specialists journals”. Today, more than ever, the generalist journals are not means to tell science to the people but represent a career asset, a ticket to a job and fame. But the other ingredient of the equation hat has changed is that where the editor (always in what I want to call the generalist genre) was an informed person making sure that the science got to the people, today is someone with the limitations derived from lack of experience and the size of the field, using more or less consciously a power they have. If you don’t believe me, read this post that will tell you how your science is not enough because it needs the editor to help you shape. This is certainly the view of Boyana Konforti (Cell Reports at the time of writing http://www.elsevier.com/connect/10-things-you-need-to-know-about-the-publishing-process) who states clearly that a paper is a collaboration between the scientist and the editor who, believe it or not, will help you tell and see a story. As she tells you: keep it simple! Sell a story! …….how low can we get? Are we writing for the editors? Is the scientific level of the editors what we should aspire to imitate? I guess what this is telling you is that if you want to publish your science, take it to ‘specialists journals’ run by scientists but if you want THE product that will give you a job, take it to them. This is where things begin to go pear shaped, where science departs from its original aim and where we have a disconnect between science and the journals. Particularly as I want to emphasize that those journals were not born to be the arbiters of science but to tell science to the people.

A recent case highlights everything that is wrong and dangerous about the situation and how the relationship between journals and scientists is sailing dangerously close to a storm.

STAP, a case in point (the perfect storm)

In January 2014 the journal Nature reported a remarkable finding. Somatic cells subject to a simple treatment (addition of lemon juice, as a friend of mine said), which involved stressing them in controlled conditions, could be reprogrammed to an embryonic state. This, the claim went, represented a huge step forward in the ability to produce embryonic stem cells as this would be natural and not involve the genetic manipulations associated with the Yamanaka cocktail. Furthermore, whereas the iPS cells are pluripotent, STAP (Stress Triggered Activation of Pluripotency) were totipotent, as they were able to give rise to extraembryonic tissues in addition to embryonic ones. This was a remarkable finding and because of this there was not only excitement but also suspicion. The simplicity of the experiment and the wide availability of cell lines with appropriate markers led to a widespread interest in doing the experiment –we did think of including it in our undergraduate project repertoire-. The papers were published by Nature and they had important and tested names most notably Y Sasai, H Niwa and T. Wakayama amidst the authors. So, what could go wrong? The first author Obokata became a celebrity and Y. Sasai, corresponding author in the main paper, was only too happy with the outcome. Maybe it was all too good to be true.

We live in the era of social media which has empowered people to speak. Paul Knoepfler, a stem cell biologist in California (USA) runs a blog on stem cells (http://www.ipscell.com/) and from the beginning expressed some scepticism on this finding. He set up a crowdsourced section in his blog where he proposed to report on attempts to reproduce STAP. At the same time he ran a periodic poll on whether people believed in the finding. In February, hope was high and the yeses outnumbered the noes. Slowly, the site was filled with failures to reproduce the finding. What is more, people in websites started to report issues of figure manipulation and text plagiarism which began to raise suspicions on the work and all of a sudden the limelight was on the first author. The web is full of what followed (see sources below) but by the end of May things were not looking good and by the beginning of July the papers were retracted because it became clear that the experiments were fraudulent. The big names put the blame on Obokata, who by now was under a huge amount of pressure (and has been ever since) and Nature claimed innocence. There was one most serious consequence of what happened here. The CDB, the host institution of the researchers, became under close scrutiny and in a surprising development, RIKEN the funding body, recommended its closure. The ball is still in the air but one hopes that the actions of individuals are not taken against an institution which plays an important role in modern developmental and stem cell biology.

The STAP case represents a collusion of interests, jobs, grants and limelight. The details and ramifications of this affair are too complex to dissect here, and I am sure that there will be books and analysis on the matter. Happy to answer questions at the end. There are nonetheless elements that are worth emphasizing. What could have happened here? I am not the only one in thinking that somebody, Obokata perhaps, did see something like STAP at some point. We all have seen black swans in the lab at some point; the problem is that they don’t breed, so we move on. In fact TH Morgan in his book on the Genetics of Drosophila has a section on ‘non heritable characters’ to quote an example. The magnitude of the possibility that harbours the finding and pressures from different sources that ensue then lead to the slippery slope. A group of people see glitter where there is dust and then you get the package: no questioning from above (if you understand Japanese you might be interested in this, and if not and you are interested, get someone to translate this for you: http://headlines.yahoo.co.jp/hl?a=20140416-00000024-wordleaf-sctch), the seal of approval of the big names, the seal of the big journal and, somewhere, the hope that somebody will see it again. I do not think that someone would expose themselves in the manner that Obokata did, unless there is some truth, and I suspect that she thought that somebody would see another black swan. How much Sasai, as corresponding author, knew is difficult to say but the fact remains that he signed as corresponding author. Much to ponder here on the structure and responsibilities in modern science. Ultimately the problem is that what was driving this affair was not science but the limelight. On the other hand, what brought this out was not just the scientists, it was the scientific community through the open discussion provided by social media, which has empowered people. And what responsibility with Nature? More than they are prepared to admit. They claim that the peer review process could not have picked up the problems (www.nature.com/news/stap-retracted-1.15488), but it was the science community which picked it up!. Furthermore, the manuscript had been rejected from Cell and Science. Furthermore, a quick standard run of the manuscript through image manipulation software run by EMBO J picked up problems almost instantly (http://onlinelibrary.wiley.com/doi/10.15252/embj.201489076/full). So, what one can conclude is that Nature’s peer review process, not THE peer review process, failed to detect the problems. As Nature refuses to provide details of the review process, we do not know what happened, but the refusal to be open (perhaps reasonable but not justifiable) does not help.

Now, this is not the first time that something like this happens in one of the main journals. Nature, Science and Cell have had their share of high profile retractions. In a remarkable one a few years ago, JH Schoen retracted 8 papers from Science and 7 from Nature (not to mention several from prestigious physics journals). This highlights the problem because as has been stated, the issue here is the scale which highlights the depth of the problem. How come that the journals do not take any impact? Have you heard of any editors resigning out of any of these affairs? Have you heard of the journals closing, receiving public scrutiny? Imagine that they were companies trading in the Stock Market. If something like STAP happens, and happens repeatedly, what happens to the shares? What happens to the company? And yet, in these journals nothing happens.

Where are we?

In the end, what we have is a situation in which these generalist journals have a different aim from their original one. Where they (though I should point out that Cell does not fit this mould) meant to be a vehicle to get science to the general public, they have become a, THE, measure of success. In fact, in China this is made clear in terms of outright payment proportional to where you publish (http://scholarlykitchen.sspnet.org/2011/04/07/paying-for-impact-does-the-chinese-model-make-sense/ ). But do not be surpised, as pointed out earlier, in Europe and the US we offer something much better: a pension!

We, a collective we, have used Nature, Science and Cell to create a rock star culture which has eroded into the base of what we do. The main reason for this, I believe, is the fact that the structure of the system we are using is, basically, the same as it was in the XIX century to which we have just added layers without restructuring it. An overloaded camel at the end of a long trip. This, together with a change of emphasis and aims, leads to  a machine that churns out papers with a lot of power in its hands and under the control of ill-prepared professionals which, basically, follow orders. NB I am not discussing here scientists led journals, some of which have a few of the same problems but which certainly do not have the ‘impact’ that lies at the heart of the malaise.

Fortunately we are waking up and solutions are on the way. In the final part of this discussion I want, briefly, to address some of them.

 

Additional Sources

Burnham, J. (1990) The evolution of editorial peer review. JAMA 263, 1323-1329.

Cyranoski, D.(2014) Cell induced stress Nature 511, 140143.

Kronick, DA (1990) Peer review in 18th century scientific journalism. JAMA 263, 1321-1322.

Nielsen, M. Three myths about scientific peer review (http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review/)

Normile, D. and Vogel, G. (2014) STAP cells succumb to pressure Science 344, 1215-1216.

Spier, R. (2002) The history of peer review process Trends in Biotech.8, 357-358.

On the Schoen affair, take as a start: http://en.wikipedia.org/wiki/Sch%C3%B6n_scandal but there is much more on the web.

Very readable, interesting and informative history of Nature magazine: http://www.nature.com/nature/history/timeline_1860s.html

On Elsevier, with its long and distinguished history: http://www.elsevier.com/__data/assets/pdf_file/0014/102632/historyofelsevier.pdf

 

The unbearable lightness of being a developmental biologist at the start of XXI century

Let us play a game. Here you have three selections from the recent indexes of three famed journals publishing in the area of Developmental Biology. See if you guess which belongs to which (sure you can put the titles in PubMed and you will find them, but try to do it blindly):

Index 1

1. Large hypomethylated domains serve as strong repressive machinery for key developmental genes in vertebrates

2. Homeotic Function of Drosophila Bithorax-Complex miRNAs Mediates Fertility by Restricting Multiple Hox Genes and TALE Cofactors in the CNS

3. KFoA, a ß-catenin interacting protein linking canonical and non-canonical Wnt signalling during mouse gastrulation and tumorogenesis

4. Actin Cytoskeleton Reorganization by Syk Regulates Fcγ Receptor Responsiveness by Increasing Its Lateral Mobility and Clustering

5. Phosphorylation of CDK2 at threonine 160 regulates meiotic pachytene and diplotene progression in mice

Index 2

1. PIWI homologs mediate Histone H4 mRNA localization to planarian chromatoid bodies

2. Without children is required for Stat-mediated zfh1 transcription and for germline stem cell differentiation

3. Vegfd can compensate for loss of Vegfc in zebrafish facial lymphatic sprouting

4. Gene expression profiling identifies the zinc-finger protein Charlatan as a regulator of intestinal stem cells in Drosophila

5. The DEP domain-containing protein TOE-2 promotes apoptosis in the Q lineage of C. elegans through two distinct mechanisms

Index 3

1. Actin Cytoskeleton Reorganization by Syk Regulates Fcγ Receptor Responsiveness by Increasing Its Lateral Mobility and Clustering

2. RagA, but Not RagB, Is Essential for Embryonic Development and Adult Mice

3. Shh Signaling Protects Atoh1 from Degradation Mediated by the E3 Ubiquitin Ligase Huwe1 in Neural Precursors

4. D6 PROTEIN KINASE Activates Auxin Transport-Dependent Growth and PIN-FORMED Phosphorylation at the Plasma Membrane

5. Developmentally Regulated Elimination of Damaged Nuclei Involves a Chk2-Dependent Mechanism of mRNA Nuclear Retention

It is difficult. It is difficult even to guess which tier (of glamour) do each of them belong to. Rest assured that the abstracts would not help either; they all read the same and yet, when you reveal the brands attached to each of them, turns out that one of them will give more brownie points in job interviews than the other two; and one of them will not give you any points at all -answers later on-. But before getting to the point of this post, apologies to the authors of those papers (if any of them reads this post) for picking them; it is the result of random sampling. They are all good papers and any others would have served equally well. In fact, this is one of the points I want to make: it really doesn’t matter because the intrinsic value of a piece of research does not appear to matter these days.

The first point I want to make concerns the titles, the topics, what is happening to developmental biology. The titles are variations on the theme “gene X/motifM in protein B is required in cell type Y for this/that/ or the other”. Ever since mutant screens took hold of the field, the aim is either to identify new genes and link them to known functions or processes, or to find new functions for known genes (signalling pathways get a prize here). The emerging alternative is those high throughput maps that reflect the toilings of a lot of people, which we are supposed to mine. There is little doubt that this is useful but, is it interesting? Sure, we need to do an inventory of the genes, their products and their modifications and link them to phenotypes but, is there a challenge here? Are there not interesting questions left to answer? I agree that there are technical challenges but, for the most part, these can be dealt with and a product will emerge, a formulaic one. Most papers look like each other, though in the discussion sections there is a bit more latitude in how the results are framed but, for the most part, they are all the same; another piece of a large puzzle, a puzzle whose template we have lost a long time ago. There are problems out there, there are large questions but nobody pushes us to think about them. I have discussed some of these issues before (http://amapress.gen.cam.ac.uk/?p=1094); all I will add here is that slowly, our enquiry is becoming light, unbearably light for those who want to know about and address important issues, or who want to hear those issues. The real questions, that exist, are being diluted by facts.

Thirty years ago we wondered what was the nature of the genes that mediated developmental processes. Many people tackled this question from different perspectives. Some never went anywhere, some did. C elegans and Drosophila brought genes to where chickens and frogs had laid questions and this provided answers to problems. It was (still is, as far as I am concerned) amazing the way it turned out: that the genes we identified in worms and flies had a bearing on vertebrates. But this was, with perspective, satisfying because it meant that there are rules, principles and today, though the indexes of the journals might betray them, we thrive on those principles. With the passing of time, the questions and the answers have become light. Driven by referees, editors, increasing meetings of hype and a general confusion of science with craftsmanship and technology, the lightness of the Science becomes unbearable. An apocryphal comment says that Science might end with us learning more and more about less and less and I am afraid that this is happening in Developmental Biology. If we are not careful, we might end up in the logical conclusion of that path: we shall know all about nothing. What are questions of interest? It will be good to identify them but two of my favourite ones lie in the emerging observation that cells churn out numbers and we do need to understand what those numbers are and where they come from. We need to know how the stochastic molecular world is averaged to the determinism of development and also what is the structure (not the hairballs that people call networks) of the machinery, the molecular devices that mediate the averaging. More on this in the future but here be dragons and perhaps this is why it is easier (but not technically) to carry on with the catalogues.

The reason for all this is that, for the most part the biomedical sciences have become a job which spins out other jobs, particularly in the journal business. I would claim that more than 50% of the publications in the areas of molecular, cell and developmental biology have as their primary aim to serve as career token. Some of them provide information but most of them are just registers of a job done. And here lies the reason why biomedical sciences is becoming unbearably light, because there is less and less hard science in much of what is being produced (read the indexes above). The ‘careerism’ and overcrowding of biomedical sciences is being discussed in several forums. Like many, I hope that we can find ways towards a solution that will restore ‘scientific enquiry’ to the heart of biomedical sciences (at some point I promise to discuss the relationship between the physical and the biomedical sciences which has some interesting lessons for us, biologists). And in this low grade industrial game we are playing, the most important things seems to be not the scientific value of the work but the ranking of the place where the work is published. Yes, I know, this sounds familiar. Time to get to the second point and disclose the journal to which the indexes above belong:

Index number 2 is taken from Development, index number 3 from Developmental Cell and number 1 is a concoction from three journals: Development (title 1), Dev Cell (titles 3, 4) and Developmental Biology (title 5) with a made up paper, did you guess it?, number 2. The point is indeed that they are all the same and that it really doesn’t matter how we distribute them. The titles I like are because they highlight what this journal game is about:

“Vegfd can compensate for loss of Vegfc in zebrafish facial lymphatic sprouting” and “RagA, but Not RagB, Is Essential for Embryonic Development and Adult Mice”,

essentially the same issue at stake. Different genes, different organisms, but the same. Almost mirror images of each other and yet, the paper in Dev Cell will give more kudos to the authors. Why? If there was ever a case for SFDORA I think this makes it clearly. It is the same product branded differently but a paper in DevCell will be looked upon as higher quality than on in Development or Dev. Biol. A recent observation in Google Scholar makes the similarity of the two journals more blatant: Dev Cell has an impact factor of 12 (not that great for all their bravado), Development 6.9 and Developmental Biology 4. However, if we looked at the h5-index we get an interesting perspective: Dev Cell’s is 89, Development’s 80 and Dev Biol’s 58. Difference vaporize, particularly between Dev Cell and Development.

googlecholar

One hopes that this will be noted. Not the lack of difference but the inflated nature of some perceptions. In the meantime, a “collective we” still fall for the gimmick of Cell Press (or Nature or Science). Their editors (and publishers), who engineer the game, will tell you that they have the highest standards, particularly in the context here those of Dev Cell. I remember talking to one of the big editors of Cell many years ago, suggesting that I did not like the way they were twisting molecular and developmental biology. This person said to me: ‘we don’t back any horses’. To which I replied that they did not need to, because they bred their own. They have now such a stable that they can organize their own races/conferences around it. It will be claimed that the papers in Dev Cell will have had a more tortuous reviewing process than the ones in Development, which makes them of higher standard, and that some of those in Development or Dev Biol might have been rejected previously from Dev Cell. Some of this is probably true (the rejections, the lengthy pointless review process) but do not be fooled, read the titles, the papers. The quality is the same and the advance that they convey –their scientific content- will also be the same. The difference is the marketing of the product and Cell Press is very good at selling their own. The kudos of a paper in Dev Cell is not based on the rigour, the novelty, the interest or the impact of the paper –as shown above- but by the marketing of the product. And, do not forget that indeed, it is harder to publish there, that the papers in Dev Cell will have more of a spin, that they will have costed more money (in revisions); but nothing else. It is a bubble (remember the tulips in Holland in the XVII century: http://en.wikipedia.org/wiki/Tulip_mania) and we should burst it. Cell Press is general is mostly tulips.

In the end, coming back to the science, none of the journals discussed here address or deal with real problems, with serious questions. This is not for lack of these, it is that the real issues don’t sell (apparently this is what these journals want to do) because they might not have an easy answer, because they cannot solve the problem in a marketable way. What does a journal rather publish: “gene Z controls DNA repair and HXYAc in development and cancer” or “the process of time averaging of gene expression cannot be explained by epigenetics”? The good news is that we see less and less screens at the forefront of the journals, the bad news is that this is being substituted by an ever increasing catalogue of more or less interesting but certainly unbearably light assessments of the functions of genes. A lot of work but, for the most part, facts in search of a reason. And most of it driven by journals, with their impact factors relish. Techniques should answer questions not hope for questions.

Solutions? The key lies in the attitude of the current generation of postdocs and graduate students since the future belongs to them. Some things are not going to change but there is something that needs changing: we need to recover the control over what we do, we need to get rid of the tyranny of the journals and the editors. An involuntary collusion of funding bodies and journals has created a very complex atmosphere in the biomedical sciences which has trivialized the science. We need to prise back from them what is ours. SFDORA is one step in that direction and I am constantly surprised how little known it is within the postdoc community. SFDORA is, at the moment the only forum which we can fuel to begin to solve the problem. The other is, of course, a bit more difficult as it relies on thinking what are the real issues in modern developmental biology. But perhaps what I sense as an unbearable lightness in the field is because there is nothing else of interest to be found out…….I don’t think so.

 

At the start of the World Cup: Football and the Biosciences

As the World Cup begins, perhaps there is a point of talking about Biology in the context of Football.

I am a supporter of Athletic Bilbao and, like the main character in ‘fever pitch’ I sometimes see the world, life, through the eyes of football. The problem is that football has changed and so has life and that the optics need some readjustment. Football is not anymore what you breath in ‘fever pitch’.

When I was a child, and even when I was an undergraduate, football was about the teams, about special players who could conjure up a moment of excitement that sometimes led to a goal opportunity or, occasionally, better to a goal. Coaches were there, in the background, steering the teams but the attention was on the football and the players. How things have changed! Now the talk divides itself between the manager, the transfer market, the money a player is worth, the budgets of the teams, the referees and, above all, success. Modern football is about success and football is just a vehicle for success. Somewhere down the line there are teams and players who, with the exception of a few chosen ones, are just warholian searchers of an afternoon moment of glory, spendable units in a machine built to attempt to succeed. Nowadays football is part of a complicated circus in which you can only compete in the elite, and sometimes just compete, with money which buys success and success which brings in money. Living in the UK still allows me to savour some of the old flavours because here, there is a tradition which is hard to kill, but for the most part it is in the lower divisions. And, of course, there is my Athletic which with only basque born players (I am 50% basque genotypically) has made it to the 4th place and champions league next season……..because of a coach which has put football and the team, rather than himself, first. Sometimes miracles are possible; some of you might relate to this.

So it is with Science, particularly with the biological sciences. What drove me to where I am now in the first place were my readings, as a student in Madrid, of books that posed questions, which combined metaphors with visions, which spoke of the wonders of the unknown and the thrill of probing into it. As I started into science some teachers led me slowly into it, emphasizing the value of the problem and the question. I found my way to the works of Mendel, Morgan and his group, Avery and his colleagues, MacClintock, Sanger, Jacob and Monod; with their consistency and their steadiness, these became an inspiration. And like many of you, I started my sailing through the world of research following these examples, with science at the core and the scientists –defined as those which practice the art of finding out about Nature through experiments and reasoning- as a reference. Ah, yes, there were journals! In those days, a publication was the rendition into paper of hard earned findings, which reported, sometimes small and sometimes big, strides towards knowledge. Journals served scientists and published their work. There was a hierarchy but one was in general aware of the value of ones findings and so were the editors.

Like football, this has changed, and in a similar direction, though in a narrower manner. Today Science is a way of living for a lot of people (see the recent paper from Alberts et al “Rescuing US biomedical research from its systemic flaws” 2014 Proc Nat Acad Sci. doi: 10.1073/pnas.1404402111 which highlights a situation in the US but which is much more general) and the goal has changed. Noawadays, the aim is to succeed and this attracts attitudes and individuals which make research a tool for success rather than a pursue of the unknown. Science and the biological sciences in particular, has become a place in which, if you are lucky (I guess they say successful) after an ever increasing period of apprenticeship, you can set up your own business (sorry, lab) and become a PI. More often than not this means that you get a chance to apply for money, to get graduate students and postdocs to do the work that you would like to do, so that you can go to an ever increasing number of meetings to market their findings and hope that this will help you raise more money to start the cycle again. Journals have come to play a central role in this complex cycle and, for the most part, rather than serving the scientists, they have managed to turn the tables. Rather than being a tool for the scientists to disseminate our work, the scientists have become a commodity for the journals which now tell the scientists what they need to do and how they need to present it in order to survive. In this process, editors have become arbiters of the way money and influence are distributed and, conscious of this, they exert this power-you might have seen at meetings the queues to speak to the editors of the journals that ‘matter’ in the hope of catching their eye so that they will look at work and…….yes, it is like this. In the same manner than in football (biomedical research) we move away from the players (postdocs, young PIs) and the game (findings) and focus on the managers (PIs) and the influence of money (grants) which depend on the commercial enterprises (publicity/sponsors) that help this process. And the only thing that matters is success defined not by the value of the finding but by the value that the speculation in the publications market gives to your research. As in football, today Science is about money, returns, commercialism and above all, success, rather than about the essence of the enterprise.

If you are a young aspiring scientist you need to understand this, and adapt. Good science is just a start. If you want to be able to practice it you will need to intuitively develop, or learn, a number of skills amidst which the ability to influence people, in particular editors, is a must. Salesmanship is more important than scientific talent and if you don’t have it, learn it because things are not going to change quickly. As in football, there are many who start but few that make it and remember that, for the most part, the limelight is on the managers, what people want to be is a PI not a scientist (and there are differences nowadays). Two very common questions that you will get often are ‘how many grants you have’ and ‘how big is your lab’ -as if the bigger the better when what this amounts to is that a PI with a big lab (say over7/8 people) really doesn’t know what is going on and that while the lab is productive in terms of churning out results, there might be little content to it.

Will this change? I don’t know; and I do not want to despair about it because there is a lot that is good, interesting and exciting about modern science. But something has to change, something deep about the spirit of how we go about science. It would be good if we could find a way to reel in some of the values of old and blend them with some of the modern attitudes. It would be good if we could put, again, science at the center of the game rather than the hype, the marketing, the publicity and the futile pursue of a fleeting sense of success. The focus of football in success has not been that good for the game but at least, if you want to enjoy football, all you need is a ball, an open space and time. Science needs a community, to support a structure which now is, increasingly, being restricted to an ever increasing minority which is selected not on merit but on the money they can get which, of course, will lead to a winner gets all situation that will take many people out of the arena. Let us hope that we can find a way forward which will be good for science and those who want to do science.

PS. As I was about to post this I hear about events at Kings College London (http://bit.ly/Svnwov). It confirms the worst of the analogy: only success narrowly defined will suffice. Scientists are just commodities valued by what money they attract and not by what they produce (there is even a scale of value). Of course, we have been fostering this attitude and, in many ways, it is implemented by stealth when people (and this is true of many institutions) are asked for certain kinds of publications which imply two or three rounds of sterile but money draining reviews which never change the message. Only those with that kind of money have access to those journals which makes the circle: funding, research, jobs narrower and narrower.

There ought to be a way to progress out of this situation because in the medium term it does not favour creativity nor what we used to call science.

On prepub servers and DORA, a glimpse of a future that is upon us

The current publication peer review system is heading for a terminal phase.

Submission to a preprint server is essential for the job market as it provides visibility to your work. And if the funding body considering the application tells you that this does not count, check whether they have signed DORA and if they have, tell them to unsign.

The reasonings of those who refuse papers in servers recognize that a paper in a server is, indeed, a publication.

As the work prevails over the publication, the scientist over the publisher, we shall regain our ground.

Last week an article in Science (http://www.sciencemag.org/content/342/6154/60) created a huge amount of e-steam and in the resulting fog a few things got, not surprisingly, confused. In particular the relationship between Open Access and Peer Review; they are related but not in the manner that was implied by the article and suggested by some of the comments. The article was a shoddy piece of scientific journalism, more fitting to the UK’s Sun or Daily Mail than to Science Magazine, and it joins the editorial of G. North in Current Biology (http://bit.ly/1a2D59c) as examples that something is afoot with the science publishing system as it is –and some of this very centered in the biosciences. The article in question did not tell us anything that we did not know and hid much that we know (some of it exposed in M Eisen’s piece: http://www.michaeleisen.org/blog/?p=1439). Much indeed was said in the cloud of comments generated by the article, so I shall not expand but one thing was clear: the current publication peer review system is heading for a terminal phase. It is some observations about the emerging future that I would like to raise here.

The main reason why the system is becoming terminal is a well known principle in physics and engineering: if the volume of water that flows through a pipe is bigger than what the pipe can contain, eventually it will burst the pipe. As the volume of scientific publications increases the current vehicles for its publication cannot cope with it and this is why the number of publications has increased. I should point out that the combination of what Biology is about –lots of stuff and information- and the relatively easy availability of powerful technology to deal with it, the quality of  most of the work is medium to high and journals, if they want to reject more than 80% of the papers, have a very hard time deciding what should and should not be in the 20% they want. This of course creates an opportunity for new publishers and publications. Couple to this OA and you get the brew for the emerging situation. Of course the big publishers have seen the golden goose here and they have created their own OA journals to absorb many of these publications and make even more money i.e. as Lampedusa put it “everything changes for everything to remain the same” or to put it plainly, they (the ‘big’ journals) have got us by…..the throat. And indeed, the system –our addiction to Nature Cell and Science and the ever more baroque and unsatisfactory system of peer review- holds, just holds, because a certain population of scientist sticks to it and because the institutions have, unconsciously, allied themselves with them. This population is the one that grew up as scientists in the 70s and the 80s and, interestingly, is made up of Cell, Molecular and Developmental Biologists. Genome and Systems Biologists do not suffer these hangups, or not so obviously.

Will the situation change? The answer of course is YES -no clairvoyance here, situations always change. The real question is what will it change to? Will it evolve a more rational system, attuned with the times? How can we get to this without losing the most important attributes of Science? Can we do it fast? Let me remind you that the people that suffer most in the current situation is the young generation and those who, while doing good science, are not in the power lobbies that control publication and grants. The question is: how can we make Science flat without losing quality and keeping many of the attributes that control this? Of course, if there is something that has made the world flat it is the Internet.

Enter the preprint server……. and combine it with SFDORA (http://am.ascb.org/dora/)

Yes, like you I have heard of preprint servers before and, like some of you, I had taken it as a distant rumble of something to be ignored. After all….well, you know all that can be said about them and…….. who reads papers there when we have our revered and valued Journals? Sure physicists use them but biologists?……..however the rumble, if it has a cause, can become a thunder as the source gets closer and at some point, one can gauge that there is a physical force behind it. I suspect the preprint server is like this and its time is coming. In certain contexts it has the potential to be the big game changer since OA, particularly when (not if, as I think that it will happen) we take it seriously. It is interesting that many publishers now will accept papers for review which have been posted in these servers as it is equally interesting to see the publishers which do not agree to this (bit.ly/PsnQjw ). As the reasonings of those who refuse papers in servers recognize, a paper in a server is, indeed, a publication. However a server is not a substitute for a publisher, it fulfills a different function, it is complementary. In many ways it puts everyone in their own place: scientists can make their work available to each other on a truly open basis and Journals can do their own business which is ….well, I am increasingly not sure what it is but what they do is to channel some Science, much of it good, within certain contexts. Properly curated preprint servers under the umbrella of San Francisco DORA (with its emphasis on the quality of the work rather than on the publisher as proxy for such evaluation) can provide an alternative to this and offer an opportunity for the scientists, biomedical scientists in particular, to recover control over what we do. This aside, let me give you two practical example of why we should embrace this combination.

A common complain when people apply for fellowships is that many funders will not look at an application if there is no first author publication. They will not specify the impact they might want to see (not these days) but they will simply say that without a publication there is no fellowship. Now it is likely that the reason why you applied is because you had submitted a good paper to one of those 10-15 journals we deem to be in some top tier of quality and hoped that by the time the application was considered you could write and say, hey here it is, published. But as your work falls through the publications gradient trying to find a place, you still do not have the publication. If you had submitted your preprint to ArXiv or BioRxve it would be, effectively, published (this is why Cell Press will not accept this), the committee can look at it –if they want to- and, according to DORA, it should be able to evaluate you on the basis of your work. Thus, submission to a preprint server is essential for the job market as it provides visibility to your work. And if the funding body considering the application tells you that this does not count, check whether they have signed DORA and if they have, tell them to unsign. DORA should be one of the main reasons why we should begin to use preprint servers. I insist, these are not a substitute for Journal publications but, if we use them correctly, they will put everybody in their place. Furthermore, in the long term I wonder whether the BioSciences might finally grow up and start being less dependent on Journals to evaluate their own work which then might become magazines highlighting work that they deem fashionable or interesting, commenting on it but not being the main vehicle for its dissemination. After all has this not happen with the music industry? Is it not happening with Newspapers and books?

I can also see the value of these servers when writing grant applications. Same logic, many funders do not like you to quote papers in preparation or submitted and yet you do need to get the information across. Simple, get your work in one of the preprint servers and quote it. It will count. Reviewers will be able to look at the data and judge it.

Let me repeat, we should not get rid of Journals, it is simply that the times are making their ways unhelpful and the sooner we force change, the better for everybody. If you stop and think about it, this is a no brainer and I suspect that if we do things right, we shall change publishing for the second time (the first time being the advent of OA). If we begin to use preprint servers it might be that journals will have to go there to look for work. It is not difficult to imagine an e-Bay of Science in which we auction our work to the journals instead of what happens now. As the work prevails over the publication, the scientist over the publisher, we shall regain the ground. It all could also make publishing cheaper….surely.

So, next time you are about to submit a paper, think about how you do it. If you are a student or a postdoc, see the enormous advantages to send it to a server at the same time that you submit to a journal. If you are a PI, do not hide your interest in an NCS publication behind the career or the student or the postdoc for if you really care about these, you should submit to a server first and if the journal you were thinking about does not allow this, ditch that journal. Everybody can gain.

So, the answer to the question of who in the biological sciences reads papers in prepublication servers when we have our revered and valued Journals is that all will depend on how interesting the papers are. For the moment this is going to be a minority but, as we wean ourselves off the intellectually nurturless high impact factor journals, as we recognize that science is about content and not about covers, as we put cool in its place (the fridge) and go for rigour and interest, more and more people will find the best of these servers useful; particularly if we work together to make them our place of exchange of ideas and discussion. More importantly, looking into the future I have little doubt that the younger generations will prefer this to the old, clutchy, clunky, clicky system that we have in Journals at the moment.

The cooling impact of Cool Science in the biological sciences

My colleague Steve Russell, a seasoned Drosophila molecular geneticist, has on a wall of his office a relic of the 80s, a copy of the only existing issue of “Cool”, a spoof  journal that arose in response to the birth of “Cool” in the biological sciences; you know the journal on which it is inspired. Steve is a good, serious, honest Drosophila biologist whose work has inspired students and postdocs and has helped the local and international communities in a variety of manners. Having “Cool’ on the wall appeals to his Glaswegian humour (download Cool here).

Cool science has come to be considered a good thing and the spoof is now an accepted accolade forgetting that while “cool science” is good for some things, it is dangerous for others. We like a cool pass in football, talk about cool footwear or clothing, we like cool movements in music, but in science what some of us look for is insight, rigour, impact, value, interest, depth, eloquence, inspiration. Cool science is good for the public understanding of Science, particularly to engage children and young people in it (http://www.theguardian.com/science/2010/apr/13/science-cool), which is different from what we should look for when we practice Science. I suspect that I am in a minority here, but I sense dangers in a trend of ‘cool science’, particularly in the biological sciences where it is becoming increasingly prevalent. My main problem is that, often, cool science is more ‘cool’ than ‘science’ and that the emphasis in ‘cool’ has a dumbing down effect on important questions and, certainly, a lowering of the level of proof and analysis. In many ways Cool says much of what this trend, a joke in the late 80s but a reality now, can do for Science and I can’t help but smile when I am in Steve’s office and see its cover on the wall.

But, why talking about “cool”?

Science publishing has become a complicated business. Perhaps in the spirit of the times, instead of rationalizing the system i.e thinking about what is best for the community, we have let it run loose, allowed the number of publications to increase and let the demand dictate the market. The vicious circle: more scientists, more affordable technology, more output,……leads to more publications; what then distinguishes one from another? There is a hierarchy established by history (and yes by Thomson Reuter) but….. if a new publication wants to be successful, the best way is to become “cool”.

One of the late newcomers to the top end of the fast moving market of publications in the Biomedical sciences is eLife. A venture of heavy weight funders which aims to change the publication landscape and the way scientists perceive their publications; at least this is what they claim (one wonders why DORA is not enough). There is much to be liked and promoted about eLife and I hope it will succeed -by which I do not mean that it becomes part of the NCS (Nature Cell and Science) elite but that it creates an imprint for progressive action in other journals. Their reviewing process , for example, is particularly innovative and sensible.

It is because I want eLife to succeed that I was surprised last week by the publication of a paper. [NOTE If you are not interested in the details of this paper you can jump to the last two paragraphs]. Mammalian genes induce partially reprogrammed pluripotent stem cells in non-mammalian vertebrate and invertebrate species (http://elife.elifesciences.org/content/2/e00036). Now, this is cool! Use the Yamanaka cocktail on somatic cells from different organisms and get iPS like cells. What is the problem? There could be something interesting in the basic observation (if it were true), though it is not clear to me how one would use iPS cells from different species? In any case, I can see (and I am told by experts in the field) some problems in these experiments. At first sight, nothing earth shattering in getting iPS cells from birds and fish. Chicken embryos have been discussed in the context of pluripotency and, with regard to fish, fish Nanog can make iPS cells in mammals. Still, these experiments should be scrutinized. However, the next stunt is, if true, the stunt of the century. It is what makes the paper cool. The authors claim that they can apply the Yamanaka cocktail to Drosophila cultured cells and obtain what they call iPS like cells. Again, think about it: apply the Yamanaka cocktail to Drosophila cultured cells and obtain a cell type for which there is no parallel, nor a way to test it in Drosophila. A cell type that, if we apply all we know about Drosophila, would be equivalent to having made an evolutionary jump. Things get more interesting because the cells used as a source for these iPS-like, Drosophila S2 cells, are variably aneuploidy and heterogeneous, do not differentiate and are basically a test tube for some experiments in biochemistry and cell biology. Those of us who have worked with them know their limitations. Their claim is equivalent to saying that iPS cells have been obtained from HeLa cells. Impossible. Have they really achieved what they claim? Can one make iPS cells from aneuploids? Don’t get me wrong, this all might be true and wonderful, though I cannot believe that the Yamanaka cocktail will repair aneuploidy (and iPS cell are nor aneuploidy) but standards should have been kept. Particularly in eLife.

I have posted a comment in the paper’s website raising some questions. The reply of the authors makes interesting reading, particularly the bit in which they claim that aneuploidy is not incompatible with pluripotency………..we are talking basic Biology here. I shall let that speak for itself.

There are three questions that this publication raises in my mind:

1. How come this paper was sent to review by a journal with the aspirations of eLife, when the claim was so clearly outrageous?

2. How come the review process, which is on line and can be seen, missed the flaws that are so obvious?

3. Why does this matter?

Responses to the first two questions are easy: the findings are “cool”. eLife wants to compete with the very cool NCS, so they need cool papers and there is nothing more cool these days than iPSness (well, yes, there is also epigenetics). As has been said by Cell Press, a paper (particularly in the high end of the market) is a collaboration between the editors, the reviewers and (lastly I would add) the authors (http://bit.ly/187zMkD), so, a published papers is not just the responsibility of the authors. The editors will always look for the balance between real science and cool but the latter will work well to attract publicity so, this will often win. In fact this is currently the main criterion for publication in NCS so, if eLife wants to compete, it has to publish cool papers, and this one is not bad at that. Thus, the paper was sent to review because it was on a hot (will use this adjective for variety, not to repeat cool too many time) topic and had a hot title The answer to the second question is well known: the peer review process is a complete shambles. This publication is another example, at the high end of the market, that the system does not work, for only in this manner can it be explained that the obvious problems with this work were not picked up. I shall not dwell on it.

The last question, why does this matter?, is more interesting and although I have gone too long already, it is important to address it. There are many papers like this one published monthly, some of them in high impact factor journals. Perhaps not so blatantly misleading, but also flawed. For the most part we ignore them. We shall pan them over a cup of coffe or a beer, but we shall not go beyond this. We are too busy trying to publish ourselves (sadly the aim of the game has become the publication rather than the science), maybe we are afraid to go public with critical comments or somehow we believe that, as they say, all will ‘come out in the wash’ -and sometimes it does but for the most part it does not. This is a pity, as it is our duty to highlight flaws when we see them and, particularly, when we see them in high profile journals. We should comment openly on each others work (I can hear you say: I don’t have time……and I understand, but then we need a different system to deal with the output of our work. On the other hand, Journals need to be careful. As the volume or work and the ability to perform technically sound work increases, the value of the analysis and interpretation of the results, decreases. Cool is an idiom, a trend and has become an important part of our society but in Science it has two edges: good to attract young people to Science, to show the public the interest and the excitement of what we do, but dangerous if it becomes part of the work itself.

When thinking about cool, remember some of the origins of the financial crises in which we are immersed. It resulted from the collusion of a number of elements and circumstances but there was one that grew up and corroded the system: the financial instruments and models that led to the creation of something out of nothing.  They were cool, very cool, and they were sold as such; but they were not rigorous and increasingly, as they became more cool, they became more unreal and ended up having a cooling effect on the economy and society. Let’s make sure that this does not happen in Biology.

PS: Cool appeared in the late 80s in many FAX machines all over the world. It was clearly ahead of its time. As far as I recall there were no restrictions on its dissemination and therefore its reproduction here is on the basis (promoted by the journal) that Cool should be spread for the benefit of cool science.