Groq API Pricing Sheet, LPU Inference Rates & Token Economics (2026)
Groq LPUs deliver 300 to 1,250 tokens per second across open-weight frontier models. Explore official per-token rates, rate limit tiers, LPU architecture, and Whisper audio transcription economics.
Groq LPU Monthly Cost Simulator
Calculate monthly bills across Groq Language Processing Units for high-speed AI applications.
Production Pricing & Model Tier Rates
Verified rates, context capacities, typical inference latency, and recommended enterprise workloads.
| Model / Tier | Input Rate (1M) | Output Rate (1M) | Context Window | Latency / Throughput | Recommended Workload |
|---|---|---|---|---|---|
| Llama 3.3 70B Versatile | $0.590 | $0.790 | 128,000 | 300+ tok/s | Flagship open reasoning, complex code generation, agents |
| Llama 3.1 8B Instant | $0.050 | $0.080 | 128,000 | 1,250 tok/s | Sub-100ms routing, classification, realtime autocomplete |
| DeepSeek-R1 Distill Llama 70B | $0.750 | $0.990 | 128,000 | 250 tok/s | Deep reasoning distilled with open weights |
| Whisper Large v3 (Audio) | $0.00185 / min | ($0.111 / hr) | N/A | 216x real-time | World's fastest speech-to-text batch transcription |
⚡ Architectural Deep Dive & Unit Economics Analysis
Groq fundamentally redefined LLM inference by inventing the Language Processing Unit (LPU). While traditional GPUs (like the Nvidia H100) rely on external High Bandwidth Memory (HBM) that suffers from memory-bus latency and unpredictable queuing, Groq's Tensor Streaming Processor features 230 MB of ultra-fast on-chip SRAM directly interconnected with 80 Terabytes per second of bandwidth.
By executing computations deterministically without runtime memory contention, Groq serves Llama 3.3 70B at 300+ tokens per second and Llama 3.1 8B at over 1,250 tokens per second. Concurrently, Groq maintains market-leading pricing ($0.59 input and $0.79 output per million tokens), proving that extreme throughput does not require a pricing premium.
Production Workloads & Monthly Cost Modeling
Evaluating practical unit economics across real-world application architectures.
Sub-200ms Conversational Voice Agent
Streaming LLM dialogue into voice pipelines with human-like conversation latency. Groq's 140ms TTFT eliminates unnatural conversational pauses.
High-Volume Speech Audio Transcription
Transcribing 5,000 hours of monthly customer calls with Whisper Large v3 at 216x real-time. Groq costs $555 versus OpenAI's $1,800, saving 69%.
Instant IDE Code Auto-Complete Engine
Generating multi-line code recommendations in under 150ms using Llama 3.1 8B at 1,250 tokens/sec. Costs just $0.08 per million output tokens.
import os
from groq import Groq
# Initialize official Groq client
client = Groq(api_key=os.environ.get("GROQ_API_KEY"))
def stream_groq_response(prompt: str):
"""Stream completions at 300+ tokens/sec."""
completion = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": prompt}],
temperature=0.2,
stream=True
)
for chunk in completion:
print(chunk.choices[0].delta.content or "", end="", flush=True)Operational Gotchas & Failure Modes
Engineering realities, throttling thresholds, and hidden surcharge traps to monitor.
Related Models, Showdowns & Benchmarks
Cross-reference provider rates against hosted models, direct head-to-head battles, and calculators.
Groq vs Together AI Showdown
Compare Groq LPUs against Together AI serverless GPUs.
Groq vs Cerebras Showdown
Compare LPUs against wafer-scale CS-3 silicon.
OpenRouter vs Groq Showdown
Compare direct Groq pricing against multi-provider routing.
Cheapest LLM APIs of 2026
Rankings of all low-cost token providers on the market.
Frequently Asked Questions
What makes Groq LPUs faster than GPUs?
Groq uses Language Processing Units (LPUs) built with 230 MB of high-speed on-chip SRAM per chip, delivering 80 TB/s of memory bandwidth and eliminating the memory-bus bottleneck of standard GPUs.
What is Groq's pricing for Llama 3.3 70B?
Groq charges $0.59 per million input tokens and $0.79 per million output tokens for Llama 3.3 70B Versatile, making it one of the most affordable 70B endpoints available.
How fast is Groq's Whisper transcription?
Groq processes audio at roughly 216x real-time speed. A 1-hour audio recording is transcribed in approximately 16 seconds for $0.111.
Is Groq API compatible with the OpenAI SDK?
Yes! Setting base_url='https://api.groq.com/openai/v1' makes Groq a 100% drop-in replacement with the official OpenAI Python and Node.js SDKs.
Does Groq offer a free tier?
Yes! Groq provides a free developer tier with strict rate limits (TPM and RPM) for evaluation and prototyping.
Can I run my own fine-tuned weights on Groq?
Not directly on serverless endpoints. Because LPUs require ahead-of-time compiler optimization, custom fine-tuned weights require an enterprise commitment.
Groq LPU Tensor Architecture: Sub-80ms TTFT & Token Economics
Groq's Language Processing Unit (LPU) architecture is engineered specifically for sequential auto-regressive generation. Built around Tensor Streaming Processors with 230MB of ultra-fast on-chip SRAM per chip, Groq bypasses standard GPU HBM memory bandwidth constraints. On Llama 3.1 8B, Groq generates over 800 tokens per second with a Time-to-First-Token (TTFT) under 70ms at an astonishingly low price: $0.05/M input tokens and $0.08/M output tokens. On Llama 3.3 70B, it sustains 280 tok/s at $0.59/M in and $0.79/M out.
Groq Production Model Rate Card
| Model ID | Input ($/M) | Output ($/M) | Generation Speed | Context Window |
|---|---|---|---|---|
| llama-3.1-8b-instant | $0.05 | $0.08 | 820 tok/s | 128,000 tokens |
| llama-3.3-70b-versatile | $0.59 | $0.79 | 280 tok/s | 128,000 tokens |
| whisper-large-v3 | $0.00185 / min | N/A | 250x Real-Time | Audio transcription |
| gemma2-9b-it | $0.20 | $0.20 | 450 tok/s | 8,192 tokens |
Groq SDK Audio Transcription & Chat Stream (Python)
from groq import Groq
client = Groq()
# Transcribe 10-minute audio file in under 2.5 seconds with Whisper Large-v3 on LPU
with open("meeting_audio.m4a", "rb") as file:
transcription = client.audio.transcriptions.create(
file=(file.name, file.read()),
model="whisper-large-v3",
response_format="text"
)
print("Transcribed:", transcription[:200])Engineering Q&A: Groq Architecture
Why is Groq LPU faster than Nvidia H100 for token generation?
Nvidia GPUs are optimized for batch parallel compute (matrix multiplication), making them ideal for training. During single-user token generation, GPUs stall waiting for memory transfers from HBM. Groq LPUs store weights in deterministic on-chip SRAM with zero memory stalls, yielding 4x higher token generation throughput.
What are the daily rate limits on Groq free tiers?
Groq offers a generous free tier (30 requests/min and 14,400 requests/day on Llama 3.1 8B). Production applications requiring high enterprise concurrency should add a credit card to unlock Tier 1/2 pay-as-you-go capacity.