BioWatchdog

Takes a bioRxiv preprint, extracts claims that can be tested, finds the linked GitHub repository, and runs smoke tests to see which ones hold up.

By Nikolas Kalavros. Inspired by OpenResearch.

How it works

  1. Paper — download the preprint and extract claims
  2. Repo — find the code repository and generate smoke tests
  3. Run — execute the tests and write up the results

Install

pip install -e . biowatchdog refresh-auth biowatchdog run <biorxiv_url>

What you get

Each run produces a folder with the paper, extracted claims, a clone of the repo, and an EVAL.md report.