DAYJOB introduces a new benchmark of 130 tasks for healthcare and finance, each requiring 13-16 hours of professional work, to evaluate AI systems in handling complex, open-ended requests.
The tasks are estimated to take a professional 13.6 hours on average in healthcare and 16.6 in finance.
We release all healthcare tasks, 50 of the 80 finance tasks, the evaluation harness, and the leaderboard.
We introduce DAYJOB, a benchmark of 130 tasks built by professionals in healthcare (50) and finance (80).
a benchmark of 130 tasks built by professionals in healthcare (50) and finance (80)
