Idea: export a batch of Predictions as a portable EvalPort result set? #468

Hi! I maintain EvalPort, an open, model-agnostic spec (JSON Schema + Python/TS SDKs) for portable LLM eval datasets — test suites, test cases, and result sets that can move between eval tools instead of every framework inventing its own file format.

I've been going through inference-client SDKs looking for the natural point where someone would want to save a batch of runs as a reusable eval fixture, and replicate-python's Prediction object is a good fit — it already carries input, output, status, error, metrics, and timestamps for every model call.

A small conversion helper (thinking an example, not core functionality) could look like:

from evalport import ResultSet, TestResult

def predictions_to_evalport(predictions: list[Prediction]) -> ResultSet:
    return ResultSet(results=[
        TestResult(
            test_case_id=p.id,
            input=p.input,
            actual_output=p.output,
            passed=p.status == "succeeded",
            error=p.error,
            metadata={"model": p.model, "version": p.version, **(p.metrics or {})},
        )
        for p in predictions
    ])

That would let anyone already scripting a batch of replicate.predictions.create() calls (e.g. regression-testing a fine-tune, or comparing two model versions on the same prompt set) dump the results into a format other eval tools can read, without writing a one-off serializer each time.

Spec, for context: https://github.com/adhabnr-ux/evalport/blob/main/SPEC.md

This is genuinely a "does this seem useful to you" check rather than a PR — happy to put one together if there's interest, and totally fine if it's not a fit for this library. Thanks for reading either way!