BGE Reranker Large API Pricing & Rate Card
Complete 2026 developer pricing guide for BGE Reranker Large. Benchmark token economics, context window limits, prompt caching discounts, and live inference latency across official provider endpoints.
| Billing Dimension | Standard Rate | Cached / Optimized Rate | Batch (50% Off) | Context Window |
|---|---|---|---|---|
| Input Prompt | $0.04/1k / M | $0.00 / M | 50% discount (24h SLA) | 8,192 tokens |
| Output Generation | $0.00 / M | Standard metered | 50% discount (24h SLA) | Up to max model limit |
| Tool / Function Calling | Standard input tokens | Eligible for prefix cache | Batch supported | Shared context budget |
Interactive Monthly Cost Estimator
Estimate your total bill based on prompt size, response volume, and request volume.
Open-Source Relevance Scoring: BAAI BGE-Reranker-Large vs Hosted APIs
BGE-Reranker-Large (developed by the Beijing Academy of Artificial Intelligence / BAAI) is the premier open-weights cross-encoder model for semantic reranking in RAG pipelines. Operating with 560 million parameters based on XLM-RoBERTa, BGE-Reranker-Large processes candidate text-document pairs simultaneously, capturing nuanced semantic cross-attention that dense vector embeddings miss.
In benchmark evaluations on MTEB (Massive Text Embedding Benchmark), BGE-Reranker-Large matches or exceeds the relevance accuracy of proprietary commercial reranking APIs (like Cohere Rerank v3) while being 100% free under the MIT license for self-hosting on private cloud GPUs.
Self-Hosted GPU vs Hosted API Cost Benchmark (100,000 Search Queries)
| Reranker Architecture | Hardware / Deployment | Cost per 100k Searches (50 chunks each) | Latency (p50) |
|---|---|---|---|
| BGE-Reranker-Large (RunPod L4 GPU) | Self-hosted NVIDIA L4 ($0.75/hr) | ~$5.40 (Hardware run) | ~24 ms (Ultra-Fast) |
| Jina Reranker v2 API | Hosted Cloud API | $2.00 | ~65 ms |
| Cohere Rerank v3.5 API | Hosted Cloud API ($2.00/1k) | $200.00 | ~80 ms |
Frequently Asked Questions
Related Frontier Showdowns & Hubs
BAAI BGE Reranker Large: Cross-Attention Scoring & Self-Hosted Serving TCO
BAAI's bge-reranker-large is the open-source industry standard for cross-encoder retrieval re-scoring in production RAG systems. Unlike bi-encoder embeddings that encode queries and documents into independent vector points, cross-encoders feed the query and candidate passage jointly through 24 transformer layers, calculating full token-level cross-attention. This yields dramatic NDCG@10 recall improvements (typically +18% over raw vector similarity), while self-hosting on dedicated cloud GPUs costs under $0.02 per 1,000 queries.
| Deployment Model | Price per 1k Searches | Hardware Required | P90 Latency (50 Docs) | Monthly Cost (2M Searches) |
|---|---|---|---|---|
| Self-Hosted TEI (1x NVIDIA L4) | $0.018 / 1k | 1x 24GB L4 GPU ($0.30/hr) | 14–22 ms | $216.00 / month |
| Cohere Rerank 3.5 (Cloud API) | $1.000 / 1k | Serverless Managed API | 65–110 ms | $2,000.00 / month (9.2× higher) |
| CPU Inference (c6i.4xlarge AWS) | $0.045 / 1k | 16 vCPU, 32GB RAM ($0.68/hr) | 85–160 ms (High latency) | $489.60 / month |
Production Pipeline: Fast ONNX Runtime / PyTorch Scoring Function
Scoring candidate passages using Hugging Face Sentence Transformers with automatic FP16 acceleration:
from sentence_transformers import CrossEncoder
# Load model in half-precision (FP16) on GPU
reranker = CrossEncoder('BAAI/bge-reranker-large', max_length=512, automodel_args={"torch_dtype": "float16"})
def score_passages(query: str, passages: list[str]) -> list[tuple[int, float]]:
pairs = [[query, passage] for passage in passages]
scores = reranker.predict(pairs, batch_size=32)
# Sort passages by relevance score descending
ranked_indices = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)
return [(idx, float(scores[idx])) for idx in ranked_indices]Production Best Practices & Reranking Rules
- Truncate Context at 512 Tokens: Cross-encoders scale quadratically with sequence length. Bounding the passage length to 512 tokens preserves 99% of retrieval signal while preventing latency blowouts.
- Top-K Pipeline Pacing: Fetch the Top 40 to Top 60 candidate chunks from vector search (Pinecone/Milvus), pass them through bge-reranker-large, and feed only the Top 5 into the LLM context window.
- FP16 Quantization Parity: Running bge-reranker-large in FP16 or INT8 ONNX cuts inference latency by 60% with zero measurable loss in ranking accuracy.
Cross-Encoder Computational Complexity vs Bi-Encoder Retrieval
Deploying BGE-Reranker-Large requires a fundamental understanding of transformer attention dynamics. Unlike bi-encoder embedding models (such as BGE-Large or OpenAI text-embedding-3) which process queries and documents independently to compute cosine similarities, BGE-Reranker-Large is a cross-encoder. It performs full self-attention across the combined pair `[CLS] + Query + [SEP] + Document + [EOS]`. This quadratic computational complexity $O((N+M)^2)$ yields superior semantic ranking precision but imposes heavy GPU compute demands.
Two-Stage Retrieval Pipeline Compute & Latency Trade-offs
| Rerank Candidates (Top-K) | Hardware Runtime | P95 Latency | Throughput (QPS) | Cost / 1M Queries |
|---|---|---|---|---|
| Top-10 Documents | 1x NVIDIA T4 / L4 | 18 ms | 52 qps | $12.50 |
| Top-50 Documents | 1x NVIDIA L40S | 44 ms | 22 qps | $48.20 |
| Top-200 Documents | 2x NVIDIA A100 (TensorRT) | 165 ms | 6 qps | $185.00 |
High-Throughput Sentence-Transformers Scoring Script
Implement an efficient batched cross-encoder evaluation loop in Python with automatic device offloading:
import torch
from sentence_transformers import CrossEncoder
device = "cuda" if torch.cuda.is_available() else "cpu"
model = CrossEncoder("BAAI/bge-reranker-large", device=device, max_length=512)
def rerank_search_results(query: str, candidates: list[str], top_n = 5):
pairs = [[query, doc] for doc in candidates]
scores = model.predict(pairs, batch_size=32)
ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
return ranked[:top_n]Cross-Encoder Pipeline Sizing & Dynamic Document Pruning
To maintain strict P95 search latency SLAs under 50 milliseconds, production systems should dynamically adjust the candidate pool size passed to BGE-Reranker-Large. Restricting reranking to the top 25 bi-encoder results delivers 98% of the maximum theoretical NDCG score while avoiding quadratic compute degradation during heavy peak search traffic.