BenchmarkHub: an open-source platform for reproducible evaluation of large language models in healthcare

Background: Large language models (LLMs) are solely judged by their scores on biomedical benchmarks, yet these scores are rarely reproducible since the prompts, inference settings and model builds are seldom reported. We developed BenchmarkHub, an open-source platform for auditable evaluation of language and vision-language models for biomedical research. Methods: BenchmarkHub is a Python web application (Django 5.2, SQLite) released under the MIT licence and available at https://github.com/juanjuanignacio/benchmarkhub. The application can be accessed through a browser interface, a set of comm

This content is for informational purposes only and is not medical advice. It is not intended to diagnose, treat, cure, or prevent any disease. Consult a healthcare professional before starting any supplement, treatment, or program: especially if you are pregnant, nursing, taking medication, or managing a health condition.