Work in science, especially at “high” level (whatever it means to you) produces limited number of valued artifacts. Mostly they include peer-reviewed papers, invited reviews, fellowships and grants, PhD theses, papers and posters at conferences.
Recently the community started to emphasize pre-prints as a valuable output of scientific process. Datasets are getting attention as “artifacts” that can be published, similar to protocols.
However, that is not enough. Much of the knowledge goes unnoticed, unrecorded, unorganized, and unchecked because we don’t value a lot of artifacts. Papers that reproduce published work are not valued and rarely noticed. Work and experience of technicians is not valued because of lack of formats to make it visible. Troubleshooting work that requires expertise and patience is not valued, because it is mostly done in the silence of the lab.
Making new formats for recording artifacts of scientific process will make it easier to show work and achievements and highlight what practices are accepted by the community.
We can compare it to software development. In the recent past the only value was in LOC – lines of code, namely KLOC as in “1000s of lines of code”. Then came time of Test-Driven Development, where (over-eagerly sometimes) most valuable became “test coverage %”. World of Agile brings forward “stories” that needs to be filled as quickly as possible. DevOps people value uptime and latency metrics. Startup companies value user base size and acquisition rate. None of these metrics (artifacts) are perfect, but they provide range of possible goals to aim for.
Software engineers have tons of metrics and artifacts they can produce that potentially capture value and represent useful work. Scientists have far fewer.
Modern science need to start producing – and valuing – more artifacts. We are currently learning to better acknowledge work of Core Centers / technology providers; role of open-source software; role of pre-prints and open peer-review; role of open-access publications. We need to highlight value of other items such as:
- technical articles
- assembly instructions for custom hardware that might not be publishable
- first-hand experimental protocols and checklists from techs/students who actually do the work
- meta-scientific articles and tools (how to make sciencing better)
- records of collaboration and technical assistance/consulting
- record of upgrading or reviving old protocols/tools/equipment
- work to make other experiments possible (e.g. building workstations or experimental setuplets)
- experiment design (perhaps through pre-registration)
- teaching and mentoring examples
The only requisite, really, is that these artifacts need to be preserved, potentially archived (so personal blogs are out 🙂
Every lab (or PI) should be able to define a set of valuable artifacts and find medium to make them public. It is OK to care about different things, but right now there is no shared set of such artifacts, or the list is too short. By elevating some of these artifacts we can fine-tune the research process to deliver value faster and more efficiently.