LangWatch Evaluation icon

LangWatch Evaluation

Check if evaluating, record results, run and record evaluators; optionally set outputs to dataset (auto writes to dataset only when not running from a dataset/batch).

Actions5

Auto (Recommended)

AI-generated

Summary

Auto-selects evaluation mode based on provided fields.

Inputs

None.

Output shape

Main item connection

Output 1: success/yes item. Output 2: no/else item (only used by Check If Evaluating).

Examples

Example 1: Run evaluator and record result

Set evaluatorSelectionMethod to dropdown or provide evaluationData; the node auto-detects runAndRecord mode.

Example 2: Record existing evaluation result

Provide result JSON via the 'result' parameter; node records it via /api/evaluations/batch/log_results.

Example 3: Write to dataset

Provide datasetSlug and optionally format/inputPath/outputPath/expectedPath; node writes entries to /api/dataset/{slug}/entries.

Discussion