| metrics |
Distribution, threshold, safety and usefulness metrics |
| scorers |
Composite, test-runner, file-contents, sandbox-state and database-diff scorers |
| solvers |
Repo setup, test running, snapshots, diffs and eval export/import |
| solvers.exports |
Child solvers that export chat, sample files, sandbox files and the store |
| solvers.imports |
Child solvers that import chat, sandbox files and the store |
| solvers.diffs |
Child solvers that diff code and databases |
| tools |
Deferred agent tools and a self-scoring tool |
| datasets |
Pluggable local and GitHub dataset sources |
| fixtures |
Docker sandbox specifications from templates |
| utils.coding |
Language-specific repository setup and test running |
| utils.collections |
Small helpers for mappings and sequences |
| utils.databases |
SQLite diffing and a selector language for asserting on diffs |
| utils.eval |
Safe evaluation of expressions |
| utils.evals |
Dynamic-metric attachment, message filters and scorer/solver result helpers |
| utils.filesystem |
Path, file-map, data-file and repo-location helpers |
| utils.format |
Formatting helpers (YAML) |
| utils.math |
Numeric helpers (clamping, rescaling and float comparison) |
| utils.parse |
Value resolution and brace-style templating |
| utils.sandbox |
Within-sandbox copy, file checks and reference diffs |