Reranker vLLM icon

Reranker vLLM

Use Reranker vLLM to reorder documents after retrieval from a vector store by relevance to the given query.

Reranker vLLM

AI-generated

Overview

This node uses the Reranker vLLM model to reorder documents retrieved from a vector store based on their relevance to a given query. It is useful in scenarios where you have a set of documents and want to prioritize or filter them according to their relevance to a specific search or question, such as improving search results or refining document recommendations.

Use Case Examples

  1. After retrieving documents from a vector database, use this node to rerank the top results to show the most relevant documents first.
  2. In a customer support system, rerank knowledge base articles based on the user's query to provide the best answers quickly.

Properties

Name Meaning
Model Name or ID The model used to rerank the documents. Users can select from a list of available models or specify a model ID via expression.
Top N The maximum number of documents to return after reranking, limiting the output to the most relevant documents.

Output

JSON

  • index - The original index of the document in the input list.
  • relevanceScore - The relevance score assigned to the document by the reranker model.
  • pageContent - The content of the document after reranking.
  • metadata
    • relevanceScore - The relevance score added to the document's metadata for reference.

Dependencies

  • Requires an API key credential for OpenAI or compatible service to access the reranker model endpoint.

Troubleshooting

  • Common issues include failure to fetch models due to incorrect or missing API credentials, resulting in errors like 'Failed to fetch models'. Ensure the API key and host URL are correctly configured.
  • If the reranking request fails, check network connectivity and API endpoint correctness. The node handles HTTP errors gracefully but will throw errors if the response is invalid or empty.
  • Ensure the input documents array is not empty; otherwise, the node returns an empty array without reranking.

Links

Discussion