SherlockBench

The SherlockBench Problem-sets

SherlockBench problem-sets are just files written in Clojure which can be loaded by the API server. Each has it's own repo in GitHub.

Sherlock1 Problem-Set

This is the problem-set we use on the Leaderboard. It has a mixture of easy and hard problems.

GitHub Link: sherlockbench-problems-sherlock1

Moriarty1 Problem-Set

This problem-set is designed for testing frontier reasoning models which get very high scores on Sherlock1.

GitHub Link: sherlockbench-problems-moriarty1

InterroBench Problem-Set

This is a port of the version 6 problem-set from the old InterroBench benchmark.

GitHub Link: sherlockbench-problems-interrobench6