Stable lane

A 30-minute path to a checked report

This path creates every dataset and prediction input locally, evaluates it strictly, and works unchanged outside the source checkout.

Install → proof → validate → evaluate

0–5 minutes

Install the evaluation extra in an isolated environment

python3 -m venv .venv
. .venv/bin/activate
python -m pip install "yolozu[coco]"
mkdir -p yolozu-quickstart && cd yolozu-quickstart

Expected: The active environment contains YOLOZU and pycocotools. For a candidate build, install the candidate wheel plus pycocotools instead; the remaining commands are unchanged and run outside the source checkout.

5–10 minutes

Create every tutorial input locally

yolozu doctor --proof --output reports/quickstart/doctor.json --proof-dir reports/quickstart/proof

Expected: The proof writes reports/quickstart/proof/toy_dataset, known_predictions.json, eval_report.json, and proof_report.json without downloading a model or dataset.

10–18 minutes

Check both interface boundaries explicitly

yolozu validate dataset reports/quickstart/proof/toy_dataset --split val2017 --strict
yolozu validate predictions reports/quickstart/proof/known_predictions.json --strict

Expected: Both validators exit successfully. These paths are outputs of the preceding proof command, not repository fixtures or guessed user paths.

18–23 minutes

Run real COCOeval with concise flags

yolozu eval-coco -d reports/quickstart/proof/toy_dataset -p reports/quickstart/proof/known_predictions.json -s val2017 -o reports/quickstart/eval_coco.json

Expected: eval_coco.json has status ok, dry_run false, validation mode strict, repair disabled, and real COCO metrics. Evaluation is strict by default.

23–27 minutes

Inspect the real report

python -m json.tool reports/quickstart/eval_coco.json

Expected: Confirm status ok, dry_run false, validation mode strict, repair disabled, the evaluated image/detection counts, and non-null COCO metrics.

27–30 minutes

Read before comparing

Expected: Confirm task, dataset/split, prediction source, export settings, protocol identity, metric family, and any skipped lane before comparing two reports.

I have a checked report Something failed

Canonical procedure: CPU-only DoD.

Core-install fallback (not real evaluation)

Use this only when pycocotools is intentionally unavailable. It performs strict validation and conversion, writes dry_run true, and leaves all metrics null; do not present it as evaluation evidence.

yolozu eval-coco -d reports/quickstart/proof/toy_dataset -p reports/quickstart/proof/known_predictions.json -s val2017 --dry-run -o reports/quickstart/eval_coco_dry_run.json

Stable Python API

Use the same strict lane from Python

The stable yolozu.api surface returns typed results and machine-readable exceptions without launching a subprocess or changing the working directory.

from pathlib import Path

from yolozu.api import evaluate_coco

workspace = Path.cwd().resolve()
result = evaluate_coco(
    dataset=workspace / "reports/quickstart/proof/toy_dataset",
    predictions=workspace / "reports/quickstart/proof/known_predictions.json",
    split="val2017",
)
print(result.to_dict()["metrics"])

Expected: The call uses the inputs created by doctor --proof, validates strictly by default, and returns a CocoEvaluationResult. Pass dry_run=True only when metrics are intentionally unavailable.

Supported symbols and error policy: Stable Python API.

Continue to the two-hour path

Complete the 30-minute stable path.
Export or import predictions from an external stack through a checked bridge.
Run a CPU-only evaluation with the same dataset, split, and fixed protocol.
Compare two wrapped prediction artifacts and their reports.
Fill out docs/evaluation_protocol_template.md for the run.
Choose stable evaluation, backend parity, or a Research lane from evidence—not from a missing feature assumption.

Use the evaluation protocol template before comparing artifacts.