eglen-cost.utf8

/* .remark-slide-content { */
/*   padding-top: 20px; */
/*   padding-left: 20px; */
/*   padding-right: 20px; */
/*   padding-bottom: 20px; */
/* } */
.remark-slide-content > h1:first-of-type {
  margin-top: 0px;
  margin-bottom: 0px;
}
.remark-slide-content > h2:first-of-type {
  margin-top: 0px;
  margin-bottom: 0px;
}

a:link {
    color:  #1f618d;
}

a:visited {
    color:  #1f618d;
}

.small { font-size: 70% }
</style>

# (Don't be) Bullied into Bad Science

<br>

```
Stephen J Eglen                  Cambridge Computational Biology Institute
https://sje30.github.io          University of Cambridge
sje30@cam.ac.uk                  @StephenEglen
```

Slides: <http://bit.ly/eglen-cost> (CC-BY license)

<br>

#### Declarations

1. Member of *Bullied into Bad Science* campaign
2. Affiliate editor of *bioRxiv* 
3. Senior editor of *Scientific Data*

#### Acknowledgements

Corina Logan and Laurent Gatto.  http://bulliedintobadscience.org has
most of the key background

Thanks to Ben Farrar and Yvonne Nobis for material.

---
## What does it mean to be bullied into bad science?

1. Science is a competitive arena; long work hours culture

2. Race to publish first

3. Not being allowed to shared resources

4. Career and family often in competition.

---

class: center, middle

# Problems

---
## We are evaluated on where, not what, we publish

Evaluation of research by impact factor or journal title

`$$I_y = \frac{C_{y-1} + C_{y-2}}{P_{y-1} + P_{y-2}}$$`

`$C_y$` = number of citations in year y, `$P_y$` = number of publications
in year y...

How could this possibly go wrong?

Strong pressure e.g. in China to pressure in journals with impact
factor over some threshold (5).

So many ways to game the system: *A user's guide to inflated and
manipulated impact factors* [(Ioannidis and Thombs, 2019)](https://www.ncbi.nlm.nih.gov/pubmed/31206647).

---
## Reproducibility crisis

Crisis?  What crisis?  Science is getting better, right?

"Negative results are disappearing from most disciplines and
countries" [(Fanelli, 2012)](http://dx.doi.org/10.1007/s11192-011-0494-7).  Study of ~4600
papers:

---

class: center, middle

# Solutions?

---
## What can you as a researcher do?

- Forget about the journal title and its impact factor, share with your
colleagues interesting research that you find (Twitter).

- Evaluating research: use the Sydney Brenner "trick" 
[(Brenner, 1995)](http://dx.doi.org/10.1016/S0960-9822%2895%2900109-X)

"we should remind ourselves that what matters absolutely is the scientific
     content of a paper and that nothing will substitute for either knowing it
	or reading it"

- Investigate preprints

- Registered reports

- Find a mutual network (open science / reproducibility crowd)

---

## Preprints

.pull-left[

### Advantages: [PLOS](https://www.plos.org/why-preprint)

1. Rapid dissemination of results

2. Establish priority

3. Increased attention/citations

4. Career advancement

5. Community

6. Unlimited / timely updates

7. Early feedback / error reduction

8. Some editors fish for papers on preprint servers

9. Preprints are valid research outputs for REF2021 [Naomi
Penfold](https://asapbio.org/preprints-valid-for-ref2021).

]

.pull-right[

]

???

(NIH early experiment stopped by journals in 1960s.)

Ask who submits preprints, and if so ask them why

---
## Pre-registered reports

"Registered Reports is a publishing format used by over 200 journals
that emphasizes the importance of the research question and the
quality of methodology by conducting peer review prior to data
collection. High quality protocols are then provisionally accepted for
publication if the authors follow through with the registered
methodology."  <https://www.cos.io/our-services/registered-reports>

One key advantage, noted by Dorothy Bishop, is that reviewers get to
critique experimental methodology BEFORE the experiment is done,
improving science, rather than just critiquing it after.

---
## Open science for the win

- "Open science" refers to freely sharing your resources (data/code/protocols).

- Allows others to build on your work, and see exactly what you did.

- I hope in a few years, the  "open" adjective can be dropped once
  "open science" becomes the norm.  Who wants to do "closed science"?

- Transparency encourages faith in what you have done.  (Yes, there
  will be errors, but is it better to keep them in the dark, or find out
  about them?)

"Most models are wrong, but some are useful".  (George Box)

"Most papers are wrong, but some are useful".  (Me).

---
## The Imperial College modelling of COVID-19

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Everybody who is concerned about the Imperial College model must read this immediately. If true, it is scandalous.<a href="https://t.co/Y2SphE228u">https://t.co/Y2SphE228u</a></p>&mdash; David Davis (@DavidDavisMP) <a href="https://twitter.com/DavidDavisMP/status/1258143326764761088?ref_src=twsrc%5Etfw">May 6, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

---
## "It ain't pretty, but it works" (Hilda Bastian)

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Independent review <a href="https://twitter.com/StephenEglen?ref_src=twsrc%5Etfw">@StephenEglen</a> confirmed that <a href="https://twitter.com/MRC_Outbreak?ref_src=twsrc%5Etfw">@MRC_Outbreak</a> team&#39;s <a href="https://twitter.com/hashtag/COVID19?src=hash&amp;ref_src=twsrc%5Etfw">#COVID19</a> simulation is reproducible: thumbs up from code-checking efforts <a href="https://twitter.com/nature?ref_src=twsrc%5Etfw">@nature</a> <a href="https://twitter.com/hashtag/COVID19?src=hash&amp;ref_src=twsrc%5Etfw">#COVID19</a> <a href="https://twitter.com/hashtag/covid19science?src=hash&amp;ref_src=twsrc%5Etfw">#covid19science</a><a href="https://t.co/vpa7CkPZjV">https://t.co/vpa7CkPZjV</a></p>&mdash; Sabine L.van Elsland (@SabineLvE) <a href="https://twitter.com/SabineLvE/status/1270789727059349505?ref_src=twsrc%5Etfw">June 10, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

---
## Conclusions

- Many reasons to feel bad...

- But there are lots of reasons for optimism at new ways of doing
  science.

- Learn how to become an open scientist and use modern tools.

- Science improves one preprint submission at a time.

- Read more at http://bulliedintobadscience.org and sign up!

---
## Evaluate journal subscription model

Think about which journals you support, by sending papers and
reviewing.

Support OA journals as authors and reviewers.  (Ever asked a journal
for permission to republish one of your own figures?)

Check out status of journal via Sherpa/Romeo service.

---
## What can your institutions & funders do?

- sign up to DORA (Declaration of Research Assessment)
(Wellcome Trust have solved this problem).  Signing is easy, enforcing
is hard.

- Give ECR a voice within institutions.

- Recognise all research artefacts, not just papers, as valuable
  outputs.

- Give academics more ownership of funds.  "Do you really want to
   spend 5K on an APC, versus other uses in your lab?"

- Routes to OA need to be generate less admin for librarians /
   funders, not more.  Green OA achieves that.

- Support diamond OA initiatives led by academics and academic
   societies.