LiteLLM + Neosantara Provider Guide

LiteLLM is the most popular open-source AI gateway — with over 50,900 GitHub stars (GitHub — BerriAI/litellm, 2026), 140+ supported providers (LiteLLM Providers, 2026), and 96M monthly PyPI downloads (PyPI — litellm, 2026). Neosantara is available as a native provider using the neosantara/ model prefix, giving you full access to Neosantara's model catalog through LiteLLM's unified interface without custom wiring.

In this guide, you'll set up LiteLLM with Neosantara from scratch: authentication, chat completions, streaming, tool calling, the Responses API, and proxy configuration. You'll go from pip install to routing production traffic in under 10 minutes.

Unlike OpenAI-compatible adapters that require manual base URL configuration on every request, Neosantara's native LiteLLM integration works through BerriAI's official provider registry. LiteLLM resolves the correct base URL, authentication, and routing automatically — the same developer experience as using OpenAI or Anthropic directly through LiteLLM.

#What Prerequisites Do You Need?

Before you start, make sure you have the following:

• Python 3.9+ installed on your system
• A Neosantara API key — sign up at app.neosantara.xyz to get Rp 10,000 free credit
• Basic familiarity with Python and command-line tools
• About 10 minutes to complete this guide

Tested on: Python 3.12, LiteLLM v1.89.2+, Ubuntu 24.04

#What Will You Build?

You'll build a working Python application that uses LiteLLM to route requests through Neosantara. By the end, you'll have:

• A single Python script that calls any Neosantara model through LiteLLM
• Streaming responses with real-time output
• Tool/function calling for structured agent workflows
• Integration with the LiteLLM Proxy server for production deployments

The same code works with any LiteLLM-supported provider — just change the model string.

#How Do You Set Up LiteLLM and Neosantara?

Installation and authentication take under 2 minutes. LiteLLM handles all the provider resolution once your environment is configured.

#Step 1: Install LiteLLM

Install LiteLLM in your Python environment:

pip install -U litellm

The -U flag ensures you get the latest version (v1.89.2+ as of June 2026). LiteLLM is a single dependency with no required extras for the SDK mode.

#Step 2: Set Your API Key

Set your Neosantara API key as an environment variable:

export NEOSANTARA_API_KEY="nsk_..."

Get your API key from the Neosantara dashboard. The key format starts with nsk_.

Verify your setup:

import litellm
print(f"LiteLLM version: {litellm.__version__}")

Expected output:

LiteLLM version: 1.89.2

#How Do You Make Your First Chat Completion?

With LiteLLM installed and your API key set, making your first request is a single function call.

from litellm import completion

response = completion(
    model="neosantara/gemini-3.5-flash",
    messages=[
        {
            "role": "user",
            "content": "Explain what Neosantara does in one sentence.",
        }
    ],
    max_tokens=120,
)

print(response.choices[0].message.content)

What just happened: The neosantara/ prefix tells LiteLLM to route the request through the Neosantara provider. LiteLLM automatically resolves NEOSANTARA_API_KEY and the default base URL (https://api.neosantara.xyz/v1). The response comes back as a standard OpenAI ChatCompletion object.

Expected output:

Neosantara is an Indonesian AI gateway that provides unified API access to 40+ leading models — including Claude, Gemini, GPT, and DeepSeek — with Rupiah billing and low-latency local infrastructure.

Try it yourself. Get your free Neosantara API key and run this code in under 5 minutes — Rp 10,000 free credit included.

In our testing, the first response from LiteLLM + Neosantara takes roughly 800-1200ms including the provider resolution handshake. Subsequent requests to the same model average 300-600ms through Neosantara's Indonesia-local gateway — significantly faster than routing through US or EU-based providers for ASEAN developers.

#How Does Streaming Work with LiteLLM?

For real-time applications like chatbots or code assistants, streaming delivers tokens as they're generated. LiteLLM makes this a one-parameter change.

from litellm import completion

response = completion(
    model="neosantara/claude-sonnet-4-6",
    messages=[
        {"role": "user", "content": "Write a short haiku about APIs."}
    ],
    stream=True,
    max_tokens=200,
)

for chunk in response:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

What just happened: Setting stream=True switches LiteLLM to return an iterator of chunks. Each chunk contains a delta with the new token. The end="" and flush=True ensure tokens print as they arrive, mimicking a real chatbot experience.

How streaming differs across providers through LiteLLM:

• Neosantara/OpenAI models: Token-by-token text deltas
• Anthropic models: Content block start/end markers plus text deltas
• LiteLLM normalizes all of them to OpenAI-compatible chunk objects, so your application code never changes

#How Do You Use Tool Calling?

Tool calling lets your LLM request external data or actions — looking up order status, querying databases, or calling APIs. Neosantara models support this through LiteLLM, with one extra parameter.

LiteLLM rejects tools and tool_choice parameters for non-OpenAI providers by default — a safety guard that prevents parameter leaks to providers that don't support them. Neosantara supports tool calling, so you must explicitly opt in with allowed_openai_params. Most developers miss this on their first attempt.

from litellm import completion

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_order_status",
            "description": "Look up the delivery status for an order.",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {
                        "type": "string",
                        "description": "The customer order ID.",
                    }
                },
                "required": ["order_id"],
            },
        },
    }
]

response = completion(
    model="neosantara/gemini-3.5-flash",
    messages=[
        {"role": "user", "content": "Check order ID INV-2045."}
    ],
    tools=tools,
    tool_choice="auto",
    allowed_openai_params=["tools", "tool_choice"],
    max_tokens=200,
)

tool_calls = response.choices[0].message.tool_calls
if tool_calls:
    for tc in tool_calls:
        print(f"Tool: {tc.function.name}")
        print(f"Args: {tc.function.arguments}")

What just happened: LiteLLM routes the tool definitions to Neosantara's OpenAI-compatible endpoint. The model returns a tool_calls array with the function name and arguments. Your application then executes the actual function and returns the result in a follow-up message.

Expected output:

Tool: get_order_status
Args: {"order_id": "INV-2045"}

#Step 4: Responses API

LiteLLM v1.79+ added native support for OpenAI's Responses API format. Neosantara supports this through both the standard completion() path and a dedicated responses() helper.

from litellm import responses

response = responses(
    model="neosantara/gemini-3.5-flash",
    input="Reply with a short launch checklist.",
    max_completion_tokens=200,
)

print(response.output[0].content[0].text)

When to use the Responses API:

• Your application already uses OpenAI's /v1/responses format
• You want to stay close to Neosantara's modern API surface
• You're building a new application and prefer the Responses-style request/response pattern

LiteLLM bridges /responses requests to Neosantara's underlying chat completions endpoint when needed, so you can use either style interchangeably.

#Step 5: LiteLLM Proxy with Neosantara

For production deployments, LiteLLM's Proxy server adds virtual keys, rate limiting, cost tracking, and team management. Here's how to configure Neosantara as a provider behind the proxy.

#Create a Proxy Config

Create a config.yaml file:

model_list:
  - model_name: neosantara-gemini-flash
    litellm_params:
      model: neosantara/gemini-3.5-flash
      api_key: os.environ/NEOSANTARA_API_KEY

  - model_name: neosantara-claude-sonnet
    litellm_params:
      model: neosantara/claude-sonnet-4-6
      api_key: os.environ/NEOSANTARA_API_KEY

  - model_name: neosantara-deepseek-r1
    litellm_params:
      model: neosantara/deepseek-r1
      api_key: os.environ/NEOSANTARA_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

#Start the Proxy

export NEOSANTARA_API_KEY="nsk_..."
export LITELLM_MASTER_KEY="sk-1234"

litellm --config config.yaml --port 4000

#Call Through the Proxy

With the proxy running, point your OpenAI SDK at http://localhost:4000:

from openai import OpenAI

client = OpenAI(
    api_key="sk-1234",  # LiteLLM master key
    base_url="http://localhost:4000"
)

response = client.chat.completions.create(
    model="neosantara-gemini-flash",
    messages=[{"role": "user", "content": "Hello from the proxy!"}],
)

print(response.choices[0].message.content)

What just happened: The proxy accepts standard OpenAI SDK calls, applies your virtual key authentication and rate limits, then routes the request to Neosantara via the neosantara/ prefix. Spend tracking, logging, and team management all work out of the box.

The proxy setup is where LiteLLM really shines for teams. We've seen teams route 60-70% of their traffic through Neosantara for ASEAN users (for latency) and fall back to OpenAI or Anthropic for workloads that need US/EU endpoints — all handled transparently by the proxy's routing and fallback policies.

#Which Model Should You Choose?

LiteLLM supports every model in Neosantara's catalog through the neosantara/<model-id> prefix. Use the Neosantara Models Overview to find model IDs.

#How Do You Troubleshoot Common Issues?

Still stuck? Open an issue on LiteLLM GitHub or contact Neosantara support.

#Frequently Asked Questions

#Does LiteLLM add latency to Neosantara requests?

Minimal overhead — LiteLLM's routing adds 5-50ms per call depending on whether it hits routing logic, the fallback chain, or a retry path (LiteLLM Benchmarks, 2026). For latency-sensitive apps, Neosantara's Indonesia-local gateway is the dominant factor, not the thin LiteLLM wrapper.

#Can I mix Neosantara with other providers in one request?

Yes. LiteLLM lets you configure multiple providers and route by model name. You can set Neosantara as the primary and fall back to OpenAI or Anthropic automatically on failure — all in a single config.yaml.

#What models are available through Neosantara in LiteLLM?

All 40+ models in Neosantara's catalog — including Claude Opus 4.6, Gemini 3 Flash, Kimi K2, DeepSeek R1, Grok 4.1 Fast, Archipelago 70B, and GPT-5.4 — are accessible through the neosantara/ prefix. See the full list at app.neosantara.xyz/models.

#How do LiteLLM's Neosantara costs compare to direct API calls?

LiteLLM charges no additional fees — you pay Neosantara's standard per-token pricing plus LiteLLM's infrastructure if self-hosting the proxy. LiteLLM Enterprise adds per-seat licensing for team features like SSO and audit logs.

#Is LiteLLM Production-Ready with Neosantara?

LiteLLM is used by Stripe, Netflix, OpenAI Agents SDK, and Google ADK (LiteLLM README, 2026). With 1,364+ releases and 8ms P95 latency at 1k RPS, it's battle-tested. Pin your LiteLLM version in production — supply-chain incidents like CVE-2026-42208 were patched within hours.

#Next Steps

You now have a working LiteLLM + Neosantara setup that handles chat completions, streaming, tool calling, and proxy-based production routing.

Extend this setup:

• Add load balancing across Neosantara regions with multiple model entries in config.yaml
• Enable cost tracking with LiteLLM's success_callback: ["langfuse"]
• Set up fallback models so traffic routes to OpenAI or Anthropic if Neosantara is unavailable
• Explore building AI agents using Neosantara with Agno

Related resources:

• LiteLLM Providers Documentation
• using Neosantara with any-llm
• building production AI agents with Neosantara and Agno

#Source References

• BerriAI. "LiteLLM." GitHub, 2026. https://github.com/BerriAI/litellm (Retrieved June 2026)
• BerriAI. "LiteLLM Providers Documentation." 2026. https://docs.litellm.ai/docs/providers (Retrieved June 2026)
• "litellm." PyPI, 2026. https://pypi.org/project/litellm/ (Retrieved June 2026)
• Neosantara. "LiteLLM Integration Guide." 2026. https://docs.neosantara.xyz/en/litellm (Retrieved June 2026)
• Neosantara. "Models Overview." 2026. https://docs.neosantara.xyz/en/models-overview (Retrieved June 2026)