MeshAgent LLM Proxy

The MeshAgent LLM Proxy lets OpenAI-, Anthropic-, and Grok-compatible HTTP and WebSocket clients send requests through MeshAgent. Instead of pointing your app, SDK, or framework directly at a provider, you point it at MeshAgent. MeshAgent authenticates the request, applies project-level routing and policy, sends the request to the selected provider, and records usage against the right project and user. With the LLM Proxy you can:

Manage provider configuration centrally at the project level in MeshAgent.
Track usage, billing, and budget controls across tools and team members in one place.
Use the same integration pattern for raw HTTP requests, official SDKs, and higher-level frameworks.

Before you start

MeshAgent provides managed OpenAI and Anthropic access by default. To use your own provider credentials, configure them per project in MeshAgent Accounts under Integrations. See Integrations for details. For localhost URLs and temporary local credentials on your machine, use the Local CLI Proxy. For Codex or Claude, meshagent setup can configure them to use MeshAgent directly. To inspect your own LLM usage for the current project, open the LLM Proxy page in MeshAgent Studio and check the My Usage tab. Use MeshAgent Accounts to manage billing, model and app restrictions, and quotas.

How it works

Your client sends a normal OpenAI-, Anthropic-, or Grok-compatible request to MeshAgent.
Your client authenticates with a MeshAgent OAuth access token or participant token.
MeshAgent validates the provider path and model, then forwards the request.
MeshAgent returns the provider-compatible response and records usage for the project resolved from the credential.

This is the same route used by meshagent ask and the MeshAgent Codex and Claude integrations. For OpenAI-compatible clients, the router supports Chat Completions and Responses over HTTP, including streamed responses. It also proxies WebSocket upgrades for the Responses API and Realtime API. The legacy /v1/completions endpoint is not part of the supported route set; use /v1/chat/completions or /v1/responses.

Select a deployment and project

meshagent setup
meshagent project list
meshagent project activate PROJECT_ID # optional: switch the active project

meshagent setup signs you in and stores an OAuth session locally. meshagent project list shows the projects you can use and their IDs. The current project is marked with *. Use meshagent project activate PROJECT_ID to switch projects first.

Find the proxy URLs

Read the URLs from the active MeshAgent CLI profile. This works for meshagent.com and self-hosted deployments:

meshagent config get openai.url
meshagent config get anthropic.url
meshagent config get grok.url

The meshagent.com defaults are:

Client	Base URL
OpenAI-compatible	`https://api.meshagent.com/openai/v1`
Anthropic SDK	`https://api.meshagent.com/anthropic`
Grok/OpenAI-compatible	`https://api.meshagent.com/grok/v1`

The Anthropic value is the SDK base URL. For raw Anthropic HTTP requests, append the normal endpoint path, such as /v1/messages.

Authentication

The proxy accepts either of these values in Authorization: Bearer <token>:

A MeshAgent OAuth access token, such as the value printed by meshagent auth token. The token must include the llm:invoke scope. OAuth requests must also include Meshagent-Project-Id, and the signed-in user must have LLM proxy access to that project.
A MeshAgent participant token whose API grant permits LLM requests. The project is embedded in the participant token, so Meshagent-Project-Id is optional; when supplied, it must match the token’s project.

A room connection is not required to call the proxy. Participant tokens may associate usage with a room, but the HTTP request still goes directly to the proxy URL.

Make a raw HTTP request

Set the project ID shown by meshagent project list, then call the deployment-specific URL directly.

Send an OpenAI-compatible request

export MESHAGENT_PROJECT_ID="YOUR_PROJECT_ID"

curl "$(meshagent config get openai.url)/responses" \
  -H "Authorization: Bearer $(meshagent auth token)" \
  -H "Meshagent-Project-Id: $MESHAGENT_PROJECT_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "input": "Tell me a fun fact about AI."
  }'

Send an Anthropic-compatible request

export MESHAGENT_PROJECT_ID="YOUR_PROJECT_ID"

curl "$(meshagent config get anthropic.url)/v1/messages" \
  -H "Authorization: Bearer $(meshagent auth token)" \
  -H "Meshagent-Project-Id: $MESHAGENT_PROJECT_ID" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 512,
    "messages": [
      {"role": "user", "content": "Tell me a fun fact about AI."}
    ]
  }'

To use a participant token instead, replace $(meshagent auth token) with the participant token and omit the Meshagent-Project-Id header.

Use the OpenAI and Anthropic SDKs

Official SDKs use provider environment variables, so set them from the active MeshAgent deployment and OAuth session before running the examples:

export MESHAGENT_PROJECT_ID="YOUR_PROJECT_ID"
export OPENAI_BASE_URL="$(meshagent config get openai.url)"
export ANTHROPIC_BASE_URL="$(meshagent config get anthropic.url)"
export OPENAI_API_KEY="$(meshagent auth token)"
export ANTHROPIC_API_KEY="$OPENAI_API_KEY"

The examples add Meshagent-Project-Id as a default SDK header because they use an OAuth token.

OpenAI SDK

# python3 llm-proxy-openai-sdk.py

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
    default_headers={"Meshagent-Project-Id": os.environ["MESHAGENT_PROJECT_ID"]},
)

response = client.responses.create(
    model="gpt-5.4",
    input="Tell me a fun fact about AI.",
)

print(response.output_text)

// npx tsx llm-proxy-openai-sdk.ts

import OpenAI from "openai";

async function main() {
  const client = new OpenAI({
    baseURL: process.env.OPENAI_BASE_URL!,
    apiKey: process.env.OPENAI_API_KEY!,
    defaultHeaders: {
      "Meshagent-Project-Id": process.env.MESHAGENT_PROJECT_ID!,
    },
  });

  const response = await client.responses.create({
    model: "gpt-5.4",
    input: "Tell me a fun fact about AI.",
  });

  console.log(response.output_text);
}

void main();

Anthropic SDK

# python3 llm-proxy-anthropic-sdk.py

import os
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ["ANTHROPIC_BASE_URL"],
    api_key=os.environ["ANTHROPIC_API_KEY"],
    default_headers={"Meshagent-Project-Id": os.environ["MESHAGENT_PROJECT_ID"]},
)

message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=512,
    messages=[
        {
            "role": "user",
            "content": "Tell me a fun fact about AI.",
        }
    ],
)

print(message.content[0].text)

// npx tsx llm-proxy-anthropic-sdk.ts

import Anthropic from "@anthropic-ai/sdk";

async function main() {
  const client = new Anthropic({
    baseURL: process.env.ANTHROPIC_BASE_URL!,
    apiKey: process.env.ANTHROPIC_API_KEY!,
    defaultHeaders: {
      "Meshagent-Project-Id": process.env.MESHAGENT_PROJECT_ID!,
    },
  });

  const message = await client.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 512,
    messages: [
      {
        role: "user",
        content: "Tell me a fun fact about AI.",
      },
    ],
  });

  console.log(message.content[0]);
}

void main();

Use other frameworks

Configure an OpenAI-compatible framework with the value from meshagent config get openai.url, or an Anthropic SDK with the value from meshagent config get anthropic.url. Use an OAuth or participant token as the framework’s API key. When using OAuth, configure Meshagent-Project-Id as a default request header. For example, LangChain can use the OpenAI-compatible route directly:

# python3 llm-proxy-langchain.py

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-5.4",
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
    default_headers={"Meshagent-Project-Id": os.environ["MESHAGENT_PROJECT_ID"]},
)

result = llm.invoke("Tell me a fun fact about AI.")
print(result.content)

For frameworks or local tools that require localhost provider endpoints, use Local CLI Proxy. The local proxy gives you temporary localhost OpenAI and Anthropic endpoints that forward through MeshAgent while the meshagent llm proxy process is running.

Automatic environment variables in room services

Service containers using container.template: agent receive the standard provider variables automatically. agent is the default template for containers declared in a service manifest.

OPENAI_BASE_URL and ANTHROPIC_BASE_URL point to the MeshAgent proxy reachable by the room runtime.
OPENAI_API_KEY and ANTHROPIC_API_KEY contain the container’s MeshAgent participant token. MESHAGENT_TOKEN contains the same token.
Grok-compatible services also receive GROK_BASE_URL, XAI_BASE_URL, GROK_API_KEY, and XAI_API_KEY.
Values explicitly set in container.environment override these defaults.
container.template: none disables all template-provided values. In that case, configure the URLs and credentials yourself.

The injected token has the container identity, room grant, agent role, and default agent API permissions. The LLM variables are therefore available only when the agent template can create that runtime participant identity. See Deploy Services for the complete template behavior.

Configure models, apps, and quotas

Open the project in MeshAgent Accounts, then select Models:

Allowed Models restricts proxy traffic to selected provider/model pairs. If the restriction is off, the project is not model-allowlisted.
Apps allows or blocks requests using glob-style User-Agent patterns. You can also reject requests that omit User-Agent.
Quotas sets default monthly dollar limits for users and service accounts. Blank means unlimited. Limits reset at the start of each UTC month and are soft limits, so an in-flight request may finish after the balance is exhausted.
A quota manager can set per-user and per-service-account overrides from Members, return a subject to the project default, or reset its current balance.

Project admins manage model and app restrictions. Project admins and members with the LLM Quota Manager role can manage quotas.

Log proxy traffic to a feed

Use meshagent llm logger to manage project loggers that copy matching LLM proxy events into a destination feed. Create the destination feed first, then create a logger with a JMESPath metadata filter:

bash

meshagent feed create \
  --name llm-proxy-logs \
  --description "LLM proxy request and response events"

# Use the returned feed id as FEED_ID.
meshagent llm logger create \
  --feed-id FEED_ID \
  --filter-expression '`true`'

Use meshagent llm logger list, meshagent llm logger get LOGGER_ID, meshagent llm logger update LOGGER_ID, and meshagent llm logger delete LOGGER_ID to manage existing loggers.

Supported Provider Paths

MeshAgent exposes these provider-compatible paths. MeshAgent proxies OpenAI, Anthropic, and Grok endpoints.

OpenAI-Compatible

The main generation and realtime transports are:

API	HTTP	WebSocket
Chat Completions	`/v1/chat/completions`	No
Responses	`/v1/responses`	`/v1/responses`
Realtime	`/v1/realtime` and `/v1/realtime/*`	`/v1/realtime`

For WebSocket clients, use the same proxy host and change the URL scheme from https to wss (or http to ws). Send the same MeshAgent bearer credential and, for OAuth, the same project header during the WebSocket handshake.

/v1/chat/completions
/v1/responses
/v1/responses/compact
/v1/responses/input_tokens
/v1/embeddings
/v1/audio/speech
/v1/audio/transcriptions
/v1/audio/translations
/v1/models and /v1/models/*
/v1/images/*
/v1/realtime and /v1/realtime/*

Anthropic-Compatible

/v1/messages
/v1/messages/count_tokens
/v1/messages/batches*
/v1/complete
/v1/models and /v1/models/*

Grok/OpenAI-Compatible

/v1/messages
/v1/responses and /v1/responses/*
/v1/responses/compact
/v1/responses/input_tokens
/v1/models and /v1/models/*

Introduction

Rooms

Agents & Tools

Deploy & Manage

Projects

Product Interfaces

Developer Reference

MeshAgent LLM Proxy

Before you start

How it works

Select a deployment and project

Find the proxy URLs

Authentication

Make a raw HTTP request

Send an OpenAI-compatible request

Send an Anthropic-compatible request

Use the OpenAI and Anthropic SDKs

OpenAI SDK

Anthropic SDK

Use other frameworks

Automatic environment variables in room services

Configure models, apps, and quotas

Log proxy traffic to a feed

Supported Provider Paths

OpenAI-Compatible

Anthropic-Compatible

Grok/OpenAI-Compatible

​Before you start

​How it works

​Select a deployment and project

​Find the proxy URLs

​Authentication

​Make a raw HTTP request

​Send an OpenAI-compatible request

​Send an Anthropic-compatible request

​Use the OpenAI and Anthropic SDKs

​OpenAI SDK

​Anthropic SDK

​Use other frameworks

​Automatic environment variables in room services

​Configure models, apps, and quotas

​Log proxy traffic to a feed

​Supported Provider Paths

​OpenAI-Compatible

​Anthropic-Compatible

​Grok/OpenAI-Compatible

​Related Docs

Before you start

How it works

Select a deployment and project

Find the proxy URLs

Authentication

Make a raw HTTP request

Send an OpenAI-compatible request

Send an Anthropic-compatible request

Use the OpenAI and Anthropic SDKs

OpenAI SDK

Anthropic SDK

Use other frameworks

Automatic environment variables in room services

Configure models, apps, and quotas

Log proxy traffic to a feed

Supported Provider Paths

OpenAI-Compatible

Anthropic-Compatible

Grok/OpenAI-Compatible

Related Docs