Bee2Bee Indexer icon

Bee2Bee Indexer

Index GitHub repositories and generate embeddings for RAG

Index Repository

AI-generated

Summary

Download a specified GitHub repository branch and index its code files into chunks, optionally generating embeddings, for use in semantic search or repository analysis.

Inputs

  • Repository Owner (required) — GitHub repository owner (user or organization) to identify which repository to index.
  • Repository Name (required) — Name of the GitHub repository to index.
  • Branch (required) — Git branch to be downloaded and indexed, defaulting to 'main'.
  • Output Format (required) — Format of the output data: 'embeddings' (code chunks plus embeddings), 'chunks_only' (code chunks without embeddings), or 'full' (chunks with embeddings and full metadata).
  • Selected Files — Optionally specify particular files to index for incremental indexing; leave empty to index all files matching specified extensions and patterns.
  • Additional Options — Collection of advanced settings including embedding provider (OpenAI or local), chunking strategy (by function, class, or file), inclusion of docstrings and file hashes, maximum files to process, file extensions to include, and directory patterns to exclude.

Output shape

A JSON object reflecting the indexed repository data, containing code chunks optionally enriched with embeddings and metadata depending on the selected output format.

Output size may be large for big repositories; operation can fail on invalid inputs or time out after 10 minutes. Embeddings generation requires appropriate API keys. File hashes are included if enabled for incremental indexing. Selected files can optimize indexing when used with metadata nodes detecting changes.

Examples

Example 1: Indexing a public GitHub repository main branch with OpenAI embeddings per function.

Owner: 'facebook', Repo: 'react', Branch: 'main', Output Format: 'embeddings', Chunk Strategy: 'function', Embedding Provider: 'openai'.

Discussion