Introducing Neosantara in any-llm

any-llm is an open-source Python library from Mozilla.ai that gives you a single, unified interface to 40+ LLM providers (Mozilla.ai, 2026). Neosantara is now available as a native provider with full feature support — one of only three providers in the entire ecosystem with 100% coverage across all capabilities.

In this guide, we'll show you how to use Neosantara through any-llm: from basic completions and streaming to reasoning, tool calling, embeddings, and batch processing.

#What is any-llm and Why Does It Matter?

any-llm has grown to 2.1k GitHub stars (as of June 2026) since Mozilla.ai first introduced it in July 2025 (Mozilla.ai, July 2025). The library now supports 42 providers across cloud, local, and hybrid deployments.

any-llm solves a real problem: every LLM provider has its own SDK, its own API format, and its own quirks. If you want to switch from OpenAI to Anthropic to a local model, you're rewriting integration code every time.

Mozilla.ai built any-llm to fix this. One import, one function call, one consistent response format — regardless of which provider runs behind it.

Key design principles:

• Uses official provider SDKs — not reimplementations that break on updates
• No proxy server required — it's a library, not a gateway service
• OpenAI-compatible output — all responses normalize to OpenAI ChatCompletion objects
• Production-ready — v1.0 shipped November 2025 (Mozilla.ai, November 2025) with async-first APIs, reusable connections, and stable interfaces

Unlike gateway solutions like OpenRouter or Portkey that route traffic through a proxy server, any-llm runs entirely in your process. Your data never passes through a third-party intermediary — a critical distinction for teams handling sensitive data or operating under data residency requirements.

The library is actively maintained as part of Mozilla.ai's broader any-suite ecosystem (any-agent, any-guardrail, Otari gateway).

#How Does Neosantara Compare to Other any-llm Providers?

According to the any-llm provider matrix (updated June 2026) (Mozilla.ai, 2026), Neosantara is one of only three providers supporting all 8 capabilities — alongside OpenAI and Otari. Most providers support 4-6 features, leaving gaps in batch processing, reasoning traces, or the Responses API.

Neosantara achieves full coverage because it implements the complete OpenAI-compatible API surface, plus provider-specific extensions for reasoning and the Responses API.

Other benefits:

• 40+ models — Model catalog (Neosantara, 2026): Claude Opus 4.6, Gemini 3 Flash, Kimi K2, Archipelago 70B, DeepSeek R1, and more
• Low latency — Indonesia-local gateway for fast response times
• Rupiah billing — transparent IDR pricing
• Single API key — one key unlocks all models in the catalog

If you've already explored building an LLM stack, any-llm handles the provider abstraction layer so you can focus on application logic instead of SDK plumbing.

#Getting Started with any-llm + Neosantara

Setting up takes under 2 minutes. The any-llm quickstart walks through the full setup — here's the Neosantara-specific path.

#Installation

Install any-llm with Neosantara support:

pip install any-llm-sdk[neosantara]

Or install with all providers:

pip install any-llm-sdk[all]

#Authentication

Set your Neosantara API key:

export NEOSANTARA_API_KEY="nsk_..."

The provider uses https://api.neosantara.xyz/v1 by default. Override with NEOSANTARA_API_BASE if needed. Don't have a key yet? Sign up and get Rp 10,000 free credit to start.

#Your First Completion

from any_llm import completion

response = completion(
    model="claude-opus-4-6",
    provider="neosantara",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(response.choices[0].message.content)

That's it. Every any-llm feature builds on this foundation.

Try it yourself in under 2 minutes. Get your free Neosantara API key — Rp 10,000 free credit, no credit card required.

#What Can You Build with any-llm + Neosantara?

From our integration testing, Neosantara's any-llm provider handles all 8 API surfaces without fallback or custom adapters. Here's each capability with working code you can run today.

#1. The AnyLLM Class

For applications making multiple requests, use the AnyLLM class to avoid repeated provider instantiation:

import os
from any_llm import AnyLLM

llm = AnyLLM.create("neosantara")

response = llm.completion(
    model="claude-opus-4-6",
    messages=[{"role": "user", "content": "Explain AI gateways in two sentences."}],
)

print(response.choices[0].message.content)

# Check provider capabilities
metadata = llm.get_provider_metadata()
print(f"Supports streaming: {metadata.streaming}")
print(f"Supports tools: {metadata.completion}")
print(f"Supports batch: {metadata.batch}")

#2. Streaming

For real-time interfaces — chatbots, dashboards, terminal tools:

from any_llm import completion

output = ""
for chunk in completion(
    model="gemini-3-flash",
    provider="neosantara",
    messages=[{"role": "user", "content": "Write a haiku about Indonesia."}],
    stream=True,
):
    chunk_content = chunk.choices[0].delta.content or ""
    print(chunk_content, end="")
    output += chunk_content

Streaming works with all Neosantara models. Each chunk follows the OpenAI streaming format, so existing code that handles OpenAI streams works unchanged.

#3. Reasoning

Get thinking traces alongside responses using reasoning_effort. This lets models like Claude and DeepSeek R1 show their chain-of-thought:

from any_llm import completion

response = completion(
    model="claude-opus-4-6",
    provider="neosantara",
    messages=[{"role": "user", "content": "How many r's are in strawberry?"}],
    reasoning_effort="high",
)

# Access the model's thinking trace
if response.choices[0].message.reasoning:
    print("Thinking:", response.choices[0].message.reasoning.content)

# The final answer
print("Answer:", response.choices[0].message.content)

Reasoning also works with streaming — each chunk may include chunk.choices[0].delta.reasoning.

#4. Tool Calling

Pass Python functions directly. any-llm automatically converts them to the provider's tool format:

from any_llm import completion

def search_web(query: str) -> str:
    """Search the web for information.

    Args:
        query: The search query to look up

    Returns:
        Search results as text
    """
    return f"Results for: {query}"

def get_weather(city: str, unit: str = "C") -> str:
    """Get current weather for a city.

    Args:
        city: City name to check weather for
        unit: Temperature unit, 'C' for Celsius or 'F' for Fahrenheit

    Returns:
        Current weather description
    """
    return f"Weather in {city}: sunny, 32°{unit}"

response = completion(
    model="claude-opus-4-6",
    provider="neosantara",
    messages=[{"role": "user", "content": "What's the weather in Jakarta?"}],
    tools=[search_web, get_weather],
)

# Handle tool calls
tool_calls = response.choices[0].message.tool_calls
if tool_calls:
    for call in tool_calls:
        print(f"Tool: {call.function.name}")
        print(f"Args: {call.function.arguments}")

Functions must have type annotations and docstrings. any-llm handles the rest. If you're building WhatsApp bots or chat assistants, this pairs well with the Neosantara Hitori plugin for deploying tool-using agents to messaging platforms.

#5. Embeddings

Generate vector embeddings for RAG, semantic search, and similarity matching:

from any_llm import embedding

result = embedding(
    model="text-embedding-3-small",
    provider="neosantara",
    inputs="Neosantara is Indonesia's fastest AI gateway",
)

vector = result.data[0].embedding
print(f"Vector dimensions: {len(vector)}")
print(f"Tokens used: {result.usage.total_tokens}")

Pass a list of strings for batch embedding:

result = embedding(
    model="text-embedding-3-small",
    provider="neosantara",
    inputs=[
        "First document to embed",
        "Second document to embed",
        "Third document to embed",
    ],
)

for item in result.data:
    print(f"Index {item.index}: {len(item.embedding)} dimensions")

#6. Batch Processing

For high-volume workloads where you don't need real-time responses, batch processing reduces costs:

from any_llm import AnyLLM

llm = AnyLLM.create("neosantara")

# Submit a batch of requests
batch = llm.batch(
    model="gemini-3-flash",
    requests=[
        {"messages": [{"role": "user", "content": "Summarize AI trends in 2026"}]},
        {"messages": [{"role": "user", "content": "What is retrieval-augmented generation?"}]},
        {"messages": [{"role": "user", "content": "Explain transformer architecture"}]},
    ],
)

print(f"Batch ID: {batch.id}")
print(f"Status: {batch.status}")

#7. List Models

Discover available models programmatically:

from any_llm import list_models

models = list_models(provider="neosantara")

for model in models.data:
    print(f"  {model.id}")

#8. Which Model Should You Choose?

Neosantara's catalog includes models optimized for different tasks. Here's a practical selection guide:

For multi-agent applications that need different models per agent role, see our Agno + Neosantara deep dive which covers team-based model selection patterns.

#Switching Between Providers

The core value of any-llm: switch between providers without rewriting your app. Same code, different provider parameter:

from any_llm import completion

messages = [{"role": "user", "content": "What is an AI gateway?"}]

# Use Neosantara
response = completion(model="claude-opus-4-6", provider="neosantara", messages=messages)

# Switch to OpenAI — only the provider and model change
response = completion(model="gpt-4o", provider="openai", messages=messages)

# Switch to local Ollama
response = completion(model="llama3", provider="ollama", messages=messages)

Your application logic stays identical. Only the provider and model parameters change. This makes Neosantara an excellent default for production — with instant fallback to other providers if needed.

#What About Async and High-Throughput Use Cases?

For applications handling concurrent requests, use the async variants:

import asyncio
from any_llm import acompletion

async def main():
    response = await acompletion(
        model="gemini-3-flash",
        provider="neosantara",
        messages=[{"role": "user", "content": "Hello from async!"}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())

Every function has an async counterpart: acompletion, aembedding, etc. Combined with the AnyLLM class's reusable connections, this handles high-throughput workloads efficiently.

#How Does Error Handling Work?

any-llm provides unified exceptions across all providers. Enable them with an environment variable:

import os
os.environ["ANY_LLM_UNIFIED_EXCEPTIONS"] = "1"

from any_llm import completion
from any_llm.exceptions import (
    RateLimitError,
    AuthenticationError,
    ModelNotFoundError,
)

try:
    response = completion(
        model="claude-opus-4-6",
        provider="neosantara",
        messages=[{"role": "user", "content": "Hello!"}],
    )
except AuthenticationError as e:
    print(f"Check your NEOSANTARA_API_KEY: {e.message}")
except RateLimitError as e:
    print(f"Rate limited — retry after backoff: {e.message}")
except ModelNotFoundError as e:
    print(f"Model not available: {e.message}")

#When Should You Use any-llm + Neosantara?

any-llm gives you one Python interface to every major LLM provider. With Neosantara as a native provider with 100% feature coverage across all 8 capabilities, you get access to top-tier models through Indonesia's fastest gateway — without sacrificing any functionality. The combination works: any-llm's unified interface + Neosantara's full model catalog + Rupiah pricing + low-latency local gateway. Start with a simple completion, add streaming and tools as your app grows, and switch models freely without touching your application code.

#Frequently Asked Questions

#Does any-llm add latency compared to calling Neosantara directly?

Minimal overhead. any-llm uses Neosantara's official SDK under the hood and reuses connections via the AnyLLM class. The normalization layer adds microseconds, not milliseconds. For latency-sensitive applications, the Indonesia-local gateway matters far more than the thin client wrapper.

#Can I use any-llm with existing OpenAI-compatible code?

Yes. any-llm returns standard OpenAI ChatCompletion Pydantic models. Code that already processes response.choices[0].message.content works without changes. You're adding provider flexibility, not replacing your response handling.

#What happens if I exceed my Neosantara rate limit?

With unified exceptions enabled, any-llm raises a RateLimitError that you can catch and retry — either with exponential backoff on the same provider, or by failing over to another provider with one parameter change.

#Is any-llm suitable for production applications?

The v1.0 release (November 2025) explicitly targets production stability: async-first APIs, reusable client connections, clear deprecation notices, and standardized output across all providers. Mozilla.ai's Otari gateway itself is built on any-llm.

#How does this compare to using Agno with Neosantara?

Different tools for different jobs. Agno is a full agent platform (tools, memory, teams, workflows). any-llm is a lightweight provider abstraction. Use any-llm when you need direct LLM calls with provider flexibility. Use Agno when you're building stateful agents with multi-step workflows.

#Source References

1. Mozilla.ai, "any-llm Provider Matrix," 2026. https://docs.mozilla.ai/any-llm/providers/
2. Mozilla.ai, "Introducing any-llm: A Unified API to Access Any LLM Provider," July 2025. https://blog.mozilla.ai/introducing-any-llm-a-unified-api-to-access-any-llm-provider/
3. Mozilla.ai, "Run Any LLM with a Single API: Introducing any-llm v1.0," November 2025. https://blog.mozilla.ai/run-any-llm-with-a-single-api-introducing-any-llm-v1-0/
4. Mozilla.ai, "any-llm GitHub Repository," 2025. https://github.com/mozilla-ai/any-llm
5. Neosantara, "Models Overview," 2026. https://docs.neosantara.xyz/en/models-overview
6. Neosantara, "Quickstart Guide," 2026. https://docs.neosantara.xyz/en/quickstart

All URLs retrieved June 2026.