Strict data sharing policies should apply to models too

What do funding agencies and journals say about data sharing, especially of models? What mechanisms for sharing models exist or are in the works?

Moderator: tom_morse

Post Reply
ted
Site Admin
Posts: 5542
Joined: Wed May 18, 2005 4:50 pm
Location: Yale University School of Medicine
Contact:

Strict data sharing policies should apply to models too

Post by ted » Mon Aug 28, 2006 5:42 pm

Federal funding agencies and journals have fairly strict data sharing policies regarding
sequence data. The "instructions for authors" for almost every scientific journal
includes a statement that sequence data are to be deposited in a publically-accessible
database such as Genbank, DNA Databank, or EMBL Data Library; many, if not most,
require that the final revision be accompanied by proof, in the form of an accession
number, that the sequence data have indeed been deposited.

Model source code should be subject to similar requirements. In this context, "model
source code" means just the code that specifies the biological properties that have been
represented in a model, and how they are represented (equivalent ulitimately to the
equations and parameters). The same logic that requires sharing of sequence data
applies. For that matter, so does the logic that requires sharing of "experimental
resources" such as reagents, cell lines, strains of experimental animals etc..

In the absence of such data sharing, computational modeling is vulnerable to the canard
that it is a rigged game--that modelers are just experimenting on "private virtual
organisms," generating results that cannot be verified or challenged because they are
inherently irreproducible. The unfortunate truth, however, is that most articles that publish
modeling results do not provide sufficient information for others to be able to reproduce
those results. This sad state of affairs is completely preventable. In theory, computational
modeling should be able to achieve a level of reproducibility that would make any
experimentalist envious.

There are some authors who have taken the initiative of posting their own model code on
the WWW or depositing it in a searchable database. However, this practice is far from
universal. It is not so much that model authors are opposed to making their model
specifications available--when asked privately, almost all agree that this would be a good
thing to do, both for the field and for their own careers, and they themselves would be
glad to do it. The problem is that when it comes time to make it happen . . . somehow it
doesn't happen.

Neither the journals nor the federal funding agencies have taken a firm stand on the issue
of sharing model code. There are lots of statements about resource sharing in general,
and one or two soft statements about sharing model code in particular, but as yet there
is neither a carrot nor a stick that would trick authors into doing the right thing.

Post Reply