Skip to content

Latest commit

 

History

History
59 lines (46 loc) · 2.78 KB

File metadata and controls

59 lines (46 loc) · 2.78 KB

Documentation

Choose the path that matches what you are trying to do.

Get started

  • Project README: preview the results site and verify the pipeline without credentials.
  • Run the stages yourself: inspect planning, execution, saved state, and evaluation with the small example.

Run and integrate

  • System-under-test requirements: find out what your generation system must support, and at which conformance level, before a benchmark can measure it.
  • Protocol schemas: connect a generation system through the language-neutral adapter interface, including generation protocol v2.
  • Artemis integration: understand the accepted Artemis and Hyperion boundary.
  • Process evaluators: run an out-of-process evaluator under its own version, bounds, and secret handling.
  • Results-site guide: validate, build, preview, and publish release data.

Datasets and studies

Understand the benchmark

  • System design: components, stored records, resume behavior, and release process.
  • Methodology: what is measured and how formal comparisons are designed and reported.
  • Telemetry profile: what process evidence is captured, which claim it can support, and when a capture fails closed.
  • Reference pricing: report a dated OpenRouter catalog estimate without confusing it with provider billing.

Reference and policy

  • Glossary: project terms and their corresponding protocol fields.
  • Changelog: changes to the generation, evaluation, dataset, and release contracts, and what an adapter or evaluator author has to do about them.
  • Restricted archives: packaging, verifying, and storing private run evidence.
  • Security policy: trust boundaries, isolation requirements, and private vulnerability reporting.
  • Licensing and third-party material: how project code, dependencies, and benchmark or result material are licensed, and what may be redistributed.
  • Citation metadata: authorship and software citation information.

Contribute