Novita AI Pricing & Developer Hub

Novita AI Infrastructure: Serverless LLMs & Ultra-Low-Cost GPU Cloud

Novita AI offers two primary services: Serverless AI APIs for LLMs (Llama 3.3, DeepSeek-V3, DeepSeek-R1) and image diffusion (Flux.1, Stable Diffusion XL), alongside an on-demand GPU Instance Cloud. By leveraging distributed mining clusters and high-density datacenter colocation, Novita delivers some of the lowest compute rates in the market: Nvidia RTX 4090 instances at $0.35/hour and serverless DeepSeek-V3 inference at $0.14/M input tokens, cutting prototype and synthetic data generation costs by 80%.

Novita AI Production Model Rate Card

Novita AI Python Image Generation Client

import requests

# Generate high-resolution Flux images for $0.0025 per generation
def generate_flux_image(prompt: str):
    res = requests.post(
        "https://api.novita.ai/v3/async/txt2img",
        headers={"Authorization": f"Bearer {os.environ['NOVITA_API_KEY']}"},
        json={
            "model_name": "flux_1_schnell",
            "prompt": prompt,
            "width": 1024,
            "height": 1024,
            "steps": 4
        }
    )
    return res.json()

Engineering Q&A: Novita AI Economics

Batch Synthetic Data Generation Economics on RTX 4090 Clusters

When generating large synthetic datasets (fine-tuning datasets, agent trajectory logs, or multilingual translations), commercial frontier APIs like GPT-4o cost over $10 per million tokens. Renting a cluster of 8x Nvidia RTX 4090 instances on Novita AI at $0.35/hour per GPU ($2.80/hour total) running DeepSeek-R1-Distill-70B via vLLM generates over 600 tokens per second. Over a 24-hour run, the cluster produces 50 million tokens for approximately $67.20—an effective cost of $1.34 per million tokens.

Production Novita SRE Guardrails

• Monitor CUDA OOM on Long Contexts: Consumer RTX 4090 GPUs have 24GB VRAM without unified memory pooling. When running 70B models in 4-bit quantization with context lengths exceeding 16k tokens, monitor KV cache allocation to prevent CUDA memory crashes.
• Asynchronous Task Polling: For image diffusion tasks (Flux.1), always use Novita's async task submission endpoint with webhook callbacks rather than holding open HTTP sockets, preventing gateway timeouts during high cluster load.

Novita AI Multi-GPU Model Serving & Distributed vLLM Clusters

Scaling custom open-weight models on Novita AI leverages distributed vLLM execution across high-performance Nvidia GPU pools. By utilizing pipeline and tensor parallelism, models with up to 70 billion parameters run smoothly across dual or quad GPU instances with continuous request batching and PagedAttention memory management. The platform provides automated health monitoring that restarts failed worker containers in under 15 seconds, ensuring high availability for user-facing production APIs.

Production API Tip: Use Novita's serverless pay-as-you-go endpoints during initial user acquisition. Once monthly generation volume stabilizes above 150 million tokens, switch to dedicated GPU instances to maximize gross margins.

Enterprise Security & Isolation: Novita AI enforces complete tenant isolation across GPU nodes. Ephemeral memory allocations are cleared and flushed to zero between subsequent batch generation sessions to prevent residual data leakage.