LangWatch Evaluation

Check if evaluating, record results, run and record evaluators; optionally set outputs to dataset (auto writes to dataset only when not running from a dataset/batch).

Actions5

Set Outputs (Dataset)

AI-generated

Summary

Writes evaluation entries to a LangWatch dataset via POST /api/dataset/{slug}/entries, shaping each entry from input/output/expected paths or custom field mappings, and returns items augmented with dataset write metadata.

Inputs

Main input flow (required) — Receives items with JSON data. The node processes each item in the batch and writes dataset entries.

Output shape

Two-item return array.

The first output contains the original input items each augmented with _langwatch.datasetWrite metadata. The second output is always an empty array [] for this operation.

Examples

Example 1: Standard format with row metadata

datasetSlug: my-dataset, format: standard, inputPath: {{$json.input}}, outputPath: {{$json.output}}, expectedPath: {{$json.expected}}, includeMetadata: true

Example 2: Custom format with field mapping

datasetSlug: my-dataset, format: custom, outputs: [{name: score, value: {{$json.score}}}, {name: reason, value: {{$json.reason}}]}, includeMetadata: false

LangWatch EvaluationInstall