LangWatch EvaluationInstall

Summary

Runs a LangWatch evaluator and logs the result to the batch context, returning items augmented with evaluation data.

Inputs

• evaluatorSelectionMethod — Manual Input or Select From Dropdown (shown for runAndRecord/auto)
• evaluatorId (required) — ID or name of the evaluator to run (shown when Manual Input selected)
• evaluatorSelect (required) — Choose from the list, or specify an ID using an expression; loadOptionsMethod getEvaluators
• name (required) — Friendly name for this evaluation execution
• evaluationData (required) — Payload for the evaluator (e.g., input/output/expected)
• evaluatorSettings — Optional evaluator-specific settings (thresholds, etc.)
• asGuardrail — Whether to act as a guardrail and enforce evaluation results
• failOnFail — Fail workflow if evaluation fails
• runId — Override the run ID; inferred from upstream dataset/evaluation context by default

Output shape

Main x2

Output 1: each input item returned with json augmented by { evaluation: , _langwatch: { recorded: true, ... } }. Output 2: empty array for this operation.

Examples

Example 1: Run evaluator manually and log result

Set evaluatorSelectionMethod to Manual Input, provide evaluatorId, set evaluationData payload, name, and optionally asGuardrail/failOnFail.