Last Editted (minor tidy up): 7th Jan 2012

OPEN SCIENCE

What is Open Science?

There is a simple answer... and then there is a more nebulous but ambitious vision.

Preamble

Science is special because of its capacity to uncover objective truth. Of course, individual scientists are people too, with a world of subjective, intuitive and more-or-less objective reasons for examining certain questions using chosen methods. While the 'who' and the 'why' can be interesting, especially historically, they do not matter much when judging scientific claims. Scientific conclusions (ideally) stand or fall based on the evidence for and against their correctness.

The process of Science depends very much on the assessments of peers. Often this works well, especially given the expert knowledge and specialised experience that can be needed to understand and contextualise technical work. At other times the peer system resembles a court in which the defendant judges the plaintiff. Sometimes the prevailing wisdom of the establishment, intellectual fashions, collegiate nepotism, or any number of human subjectivities can retard the clear-eyed assessment of correctness. So it has always been.

In modern times the Science professions have changed radically. It is said there are more 'scientists' active today than have ever practised prior to the 20th Century, and the enterprise continues to expand. We no longer work with seven or a dozen laboriously collected data points against which we fit carefully considered models by hand; rather, we are awash with data and computational tools. It is very easy to make mistakes. It can be very hard to make sense of, or verify independently, the work of others. And so, through necessity or laziness, work is often judged superficially on the names (of people, institutions, journals) or the appeal of the 'story' told.

We must find ways to push back these subjective incursions.

Technology is part of the way forward. The internet is an infant, yet already powerfully connects people and ideas.

Open Science Lite

Open Science in basic form requires two things: (i) clear and complete presentation of data and methods, and (ii) for the authors to care genuinely about the correctness of their work, and to act in response to any mistakes or problems that arise, before and after publication.

To do Open Science at its most basic is just to respect long-established and well-understood principles and practices.

Web pages are a powerful way of maintaining a record of data, methods, and the thinking upon which decisions are based, and for presenting these in a more accessible way than the peer-reviewed literature generally allows. This web based approach allows for contributions and feedback during and after the 'doing' of the science.

One cornerstone of science is replication, a topic that can bear some discussion. It does not matter where or when or by whom an experiment or analysis is performed: if it demonstrates something that is objective truth, the same answers will arise in the hands of any competent practitioner, anywhere, anytime.

Replication is more complex than it might at first appear; the concept spans a continuum, exemplified at one extreme by the mechanical processing of a recipe, and at the other by total reconsideration and reconstruction of the means by which to test the claim. Recipe-style replication is important for fault finding, but has limited weight in supporting claims (it is necessary, but not sufficient). Conversely, the replication-by-new-analysis might not mean so much when it fails, but acts as powerful support when it succeeds.

To practice Open Science is to embrace critical analysis of your work. This allows for fault finding in the first instance, and enables deeper understanding of conclusions in the longer term.

Open Science - a Stronger Form

OzEA is an experiment in a stronger form of Open Science. It is one thing do our science live on the internet, the work open to scrutiny and comment. It is a further step to present the model we do (The Stories, Data, Analysis, Models, Literature, Discussions). These levels are foundational to the stronger form of Open Science that we seek to facilitate and lead; this being to build a place where a virtual community can constructively work on answering the questions we set ourselves.

Most of us have preconceptions as to the likely outcomes of this work; however, even within the OzEA core team these preconceptions pretty much span the rational spectrum on touchstone topics. We do not renounce our preconceptions, we just leave them at the door. Careful treatment of data, clear-sighted analysis, robust and coherent models -- these things will transparently dictate the claims and conclusions that arise here. A diversity of preconceptions helps to prevent any particular view of what should be the answer subtly biasing the work. In this way the results that develop here can be demonstrably correct, and thus lay claim to the authority of scientific objectivity.

A clear goal here is to develop the interactivity of the web beyond that of the forums and blogs with which many are familiar (and familiar also with their failings and limitations). Hence the rules of etiquette and the separation of data from analysis from our stories, the models, literature, and discussions (i.e. the menu bar). These divisions are tentative, but the logic underpinning them is not. By demarcating the introduction of datasets to the datasets area, by focusing the consideration of literature into its own area, and by maintaining these divisions we achieve two important things; first, we have a defined work flow that can be iterated in waves of development (you may need to observe this in action to properly understand our method), and second, problems and controversies are contained into the areas where resolution most likely lies, and thus need not cripple the entire flow of work.

We are developing methods and standards for high-quality Open Science. We seek to develop a community here working on the renewable penetration problem: a diverse, smart, friendly and open community. We offer a model and a place to engage. Join with us.