Stable lane

Your first evaluation

Create a small local dataset and score its predictions in about 30 minutes. This synthetic example checks the workflow, not model accuracy.

Install → proof → validate → evaluate

0–5 minutes

Install in a virtual environment

python3 -m venv .venv
. .venv/bin/activate
python -m pip install "yolozu[coco]"
mkdir -p yolozu-quickstart && cd yolozu-quickstart

Expected: YOLOZU and pycocotools are installed. For a candidate build, install its wheel and pycocotools instead. The remaining commands also work outside the source checkout.

5–10 minutes

Create the sample data

yolozu doctor --proof --output reports/quickstart/doctor.json --proof-dir reports/quickstart/proof

Expected: The proof writes reports/quickstart/proof/toy_dataset, known_predictions.json, eval_report.json, and proof_report.json without downloading a model or dataset.

10–18 minutes

Validate the dataset and predictions

yolozu validate dataset reports/quickstart/proof/toy_dataset --split val2017 --strict
yolozu validate predictions reports/quickstart/proof/known_predictions.json --strict

Expected: Both validators pass using the files created in the previous step.

18–23 minutes

Run COCO evaluation

yolozu eval-coco -d reports/quickstart/proof/toy_dataset -p reports/quickstart/proof/known_predictions.json -s val2017 -o reports/quickstart/eval_coco.json

Expected: The command writes eval_coco.json with COCO metrics. Validation is strict by default; inputs are not repaired.

23–27 minutes

Read the report

python -m json.tool reports/quickstart/eval_coco.json

Expected: Confirm status ok, dry_run false, validation mode strict, repair disabled, the evaluated image/detection counts, and non-null COCO metrics.

27–30 minutes

Read before comparing

Expected: Confirm task, dataset/split, prediction source, export settings, protocol identity, metric family, and any skipped lane before comparing two reports.

I have a checked report Something failed

Canonical procedure: CPU-only DoD.

Stable Python API

Evaluate from Python

The Python API returns typed results and exceptions. It does not launch a subprocess or change your working directory.

from pathlib import Path

from yolozu.api import evaluate_coco

workspace = Path.cwd().resolve()
result = evaluate_coco(
    dataset=workspace / "reports/quickstart/proof/toy_dataset",
    predictions=workspace / "reports/quickstart/proof/known_predictions.json",
    split="val2017",
)
print(result.to_dict()["metrics"])

Expected: The call uses the inputs created by doctor --proof, validates strictly by default, and returns a CocoEvaluationResult. Pass dry_run=True only when metrics are intentionally unavailable.

Supported symbols and error policy: Stable Python API.

Continue to the two-hour path

  1. Complete the 30-minute stable path.
  2. Export or import predictions from an external stack through a checked bridge.
  3. Run a CPU-only evaluation with the same dataset, split, and fixed protocol.
  4. Compare two wrapped prediction artifacts and their reports.
  5. Fill out docs/evaluation_protocol_template.md for the run.
  6. Choose the next workflow using its documented requirements and support level.

Use the evaluation protocol template before comparing artifacts.