> ## Documentation Index
> Fetch the complete documentation index at: https://docs.meshagent.com/llms.txt
> Use this file to discover all available pages before exploring further.

# MeshAgent LLM Proxy

> Centralize provider setup, usage, billing, and budget controls for OpenAI-, Anthropic-, and Grok-compatible traffic routed through MeshAgent.

The MeshAgent LLM Proxy lets OpenAI-, Anthropic-, and Grok-compatible HTTP and WebSocket clients send requests through MeshAgent. Instead of pointing your app, SDK, or framework directly at a provider, you point it at MeshAgent. MeshAgent authenticates the request, applies project-level routing and policy, sends the request to the selected provider, and records usage against the right project and user.

With the LLM Proxy you can:

* Manage provider configuration centrally at the project level in MeshAgent.
* Track usage, billing, and budget controls across tools and team members in one place.
* Use the same integration pattern for raw HTTP requests, official SDKs, and higher-level frameworks.

## Before you start

MeshAgent provides managed OpenAI and Anthropic access by default. To use your own provider credentials, configure them per project in [MeshAgent Accounts](https://accounts.meshagent.com) under **Integrations**. See [Integrations](../../project_admin/integrations) for details.

For localhost URLs and temporary local credentials on your machine, use the [Local CLI Proxy](./local_cli_proxy). For [Codex or Claude](./use_codex_claude), `meshagent setup` can configure them to use MeshAgent directly.

To inspect your own LLM usage for the current project, open the **LLM Proxy** page in [MeshAgent Studio](../../interfaces/meshagent_studio) and check the **My Usage** tab. Use [MeshAgent Accounts](https://accounts.meshagent.com) to manage billing, model and app restrictions, and quotas.

## How it works

1. Your client sends a normal OpenAI-, Anthropic-, or Grok-compatible request to MeshAgent.
2. Your client authenticates with a MeshAgent OAuth access token or participant token.
3. MeshAgent validates the provider path and model, then forwards the request.
4. MeshAgent returns the provider-compatible response and records usage for the project resolved from the credential.

This is the same route used by `meshagent ask` and the [MeshAgent Codex and Claude](./use_codex_claude) integrations.

For OpenAI-compatible clients, the router supports Chat Completions and Responses over HTTP, including streamed responses. It also proxies WebSocket upgrades for the Responses API and Realtime API. The legacy `/v1/completions` endpoint is not part of the supported route set; use `/v1/chat/completions` or `/v1/responses`.

## Select a deployment and project

```bash theme={null}
meshagent setup
meshagent project list
meshagent project activate PROJECT_ID # optional: switch the active project
```

`meshagent setup` signs you in and stores an OAuth session locally. `meshagent project list` shows the projects you can use and their IDs. The current project is marked with `*`. Use `meshagent project activate PROJECT_ID` to switch projects first.

## Find the proxy URLs

Read the URLs from the active MeshAgent CLI profile. This works for meshagent.com and self-hosted deployments:

```bash theme={null}
meshagent config get openai.url
meshagent config get anthropic.url
meshagent config get grok.url
```

The meshagent.com defaults are:

| Client | Base URL |
| - | - |
| OpenAI-compatible | `https://api.meshagent.com/openai/v1` |
| Anthropic SDK | `https://api.meshagent.com/anthropic` |
| Grok/OpenAI-compatible | `https://api.meshagent.com/grok/v1` |

The Anthropic value is the SDK base URL. For raw Anthropic HTTP requests, append the normal endpoint path, such as `/v1/messages`.

## Authentication

The proxy accepts either of these values in `Authorization: Bearer <token>`:

* A MeshAgent OAuth access token, such as the value printed by `meshagent auth token`. The token must include the `llm:invoke` scope. OAuth requests must also include `Meshagent-Project-Id`, and the signed-in user must have LLM proxy access to that project.
* A MeshAgent participant token whose API grant permits LLM requests. The project is embedded in the participant token, so `Meshagent-Project-Id` is optional; when supplied, it must match the token's project.

A room connection is not required to call the proxy. Participant tokens may associate usage with a room, but the HTTP request still goes directly to the proxy URL.

## Make a raw HTTP request

Set the project ID shown by `meshagent project list`, then call the deployment-specific URL directly.

### Send an OpenAI-compatible request

```bash theme={null}
export MESHAGENT_PROJECT_ID="YOUR_PROJECT_ID"

curl "$(meshagent config get openai.url)/responses" \
  -H "Authorization: Bearer $(meshagent auth token)" \
  -H "Meshagent-Project-Id: $MESHAGENT_PROJECT_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "input": "Tell me a fun fact about AI."
  }'
```

### Send an Anthropic-compatible request

```bash theme={null}
export MESHAGENT_PROJECT_ID="YOUR_PROJECT_ID"

curl "$(meshagent config get anthropic.url)/v1/messages" \
  -H "Authorization: Bearer $(meshagent auth token)" \
  -H "Meshagent-Project-Id: $MESHAGENT_PROJECT_ID" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 512,
    "messages": [
      {"role": "user", "content": "Tell me a fun fact about AI."}
    ]
  }'
```

To use a participant token instead, replace `$(meshagent auth token)` with the participant token and omit the `Meshagent-Project-Id` header.

## Use the OpenAI and Anthropic SDKs

Official SDKs use provider environment variables, so set them from the active MeshAgent deployment and OAuth session before running the examples:

```bash theme={null}
export MESHAGENT_PROJECT_ID="YOUR_PROJECT_ID"
export OPENAI_BASE_URL="$(meshagent config get openai.url)"
export ANTHROPIC_BASE_URL="$(meshagent config get anthropic.url)"
export OPENAI_API_KEY="$(meshagent auth token)"
export ANTHROPIC_API_KEY="$OPENAI_API_KEY"
```

The examples add `Meshagent-Project-Id` as a default SDK header because they use an OAuth token.

### OpenAI SDK

<CodeGroup>
  ```python Python theme={null}
  # python3 llm-proxy-openai-sdk.py

  import os
  from openai import OpenAI

  client = OpenAI(
      base_url=os.environ["OPENAI_BASE_URL"],
      api_key=os.environ["OPENAI_API_KEY"],
      default_headers={"Meshagent-Project-Id": os.environ["MESHAGENT_PROJECT_ID"]},
  )

  response = client.responses.create(
      model="gpt-5.4",
      input="Tell me a fun fact about AI.",
  )

  print(response.output_text)

  ```

  ```typescript TypeScript theme={null}
  // npx tsx llm-proxy-openai-sdk.ts

  import OpenAI from "openai";

  async function main() {
    const client = new OpenAI({
      baseURL: process.env.OPENAI_BASE_URL!,
      apiKey: process.env.OPENAI_API_KEY!,
      defaultHeaders: {
        "Meshagent-Project-Id": process.env.MESHAGENT_PROJECT_ID!,
      },
    });

    const response = await client.responses.create({
      model: "gpt-5.4",
      input: "Tell me a fun fact about AI.",
    });

    console.log(response.output_text);
  }

  void main();

  ```
</CodeGroup>

### Anthropic SDK

<CodeGroup>
  ```python Python theme={null}
  # python3 llm-proxy-anthropic-sdk.py

  import os
  from anthropic import Anthropic

  client = Anthropic(
      base_url=os.environ["ANTHROPIC_BASE_URL"],
      api_key=os.environ["ANTHROPIC_API_KEY"],
      default_headers={"Meshagent-Project-Id": os.environ["MESHAGENT_PROJECT_ID"]},
  )

  message = client.messages.create(
      model="claude-sonnet-4-6",
      max_tokens=512,
      messages=[
          {
              "role": "user",
              "content": "Tell me a fun fact about AI.",
          }
      ],
  )

  print(message.content[0].text)

  ```

  ```typescript TypeScript theme={null}
  // npx tsx llm-proxy-anthropic-sdk.ts

  import Anthropic from "@anthropic-ai/sdk";

  async function main() {
    const client = new Anthropic({
      baseURL: process.env.ANTHROPIC_BASE_URL!,
      apiKey: process.env.ANTHROPIC_API_KEY!,
      defaultHeaders: {
        "Meshagent-Project-Id": process.env.MESHAGENT_PROJECT_ID!,
      },
    });

    const message = await client.messages.create({
      model: "claude-sonnet-4-6",
      max_tokens: 512,
      messages: [
        {
          role: "user",
          content: "Tell me a fun fact about AI.",
        },
      ],
    });

    console.log(message.content[0]);
  }

  void main();

  ```
</CodeGroup>

## Use other frameworks

Configure an OpenAI-compatible framework with the value from `meshagent config get openai.url`, or an Anthropic SDK with the value from `meshagent config get anthropic.url`. Use an OAuth or participant token as the framework's API key. When using OAuth, configure `Meshagent-Project-Id` as a default request header.

For example, LangChain can use the OpenAI-compatible route directly:

<CodeGroup>
  ```python Python theme={null}
  # python3 llm-proxy-langchain.py

  import os
  from langchain_openai import ChatOpenAI

  llm = ChatOpenAI(
      model="gpt-5.4",
      base_url=os.environ["OPENAI_BASE_URL"],
      api_key=os.environ["OPENAI_API_KEY"],
      default_headers={"Meshagent-Project-Id": os.environ["MESHAGENT_PROJECT_ID"]},
  )

  result = llm.invoke("Tell me a fun fact about AI.")
  print(result.content)

  ```
</CodeGroup>

For frameworks or local tools that require localhost provider endpoints, use [Local CLI Proxy](./local_cli_proxy). The local proxy gives you temporary localhost OpenAI and Anthropic endpoints that forward through MeshAgent while the `meshagent llm proxy` process is running.

## Automatic environment variables in room services

Service containers using `container.template: agent` receive the standard provider variables automatically. `agent` is the default template for containers declared in a service manifest.

* `OPENAI_BASE_URL` and `ANTHROPIC_BASE_URL` point to the MeshAgent proxy reachable by the room runtime.
* `OPENAI_API_KEY` and `ANTHROPIC_API_KEY` contain the container's MeshAgent participant token. `MESHAGENT_TOKEN` contains the same token.
* Grok-compatible services also receive `GROK_BASE_URL`, `XAI_BASE_URL`, `GROK_API_KEY`, and `XAI_API_KEY`.
* Values explicitly set in `container.environment` override these defaults.
* `container.template: none` disables all template-provided values. In that case, configure the URLs and credentials yourself.

The injected token has the container identity, room grant, `agent` role, and default agent API permissions. The LLM variables are therefore available only when the `agent` template can create that runtime participant identity. See [Deploy Services](../../services/deployment/deploy_services#containertemplate) for the complete template behavior.

## Configure models, apps, and quotas

Open the project in [MeshAgent Accounts](https://accounts.meshagent.com), then select **Models**:

* **Allowed Models** restricts proxy traffic to selected provider/model pairs. If the restriction is off, the project is not model-allowlisted.
* **Apps** allows or blocks requests using glob-style `User-Agent` patterns. You can also reject requests that omit `User-Agent`.
* **Quotas** sets default monthly dollar limits for users and service accounts. Blank means unlimited. Limits reset at the start of each UTC month and are soft limits, so an in-flight request may finish after the balance is exhausted.
* A quota manager can set per-user and per-service-account overrides from **Members**, return a subject to the project default, or reset its current balance.

Project admins manage model and app restrictions. Project admins and members with the **LLM Quota Manager** role can manage quotas.

## Log proxy traffic to a feed

Use `meshagent llm logger` to manage project loggers that copy matching LLM proxy events into a destination feed. Create the destination feed first, then create a logger with a JMESPath metadata filter:

```bash bash theme={null}
meshagent feed create \
  --name llm-proxy-logs \
  --description "LLM proxy request and response events"

# Use the returned feed id as FEED_ID.
meshagent llm logger create \
  --feed-id FEED_ID \
  --filter-expression '`true`'
```

Use `meshagent llm logger list`, `meshagent llm logger get LOGGER_ID`, `meshagent llm logger update LOGGER_ID`, and `meshagent llm logger delete LOGGER_ID` to manage existing loggers.

## Supported Provider Paths

MeshAgent exposes these provider-compatible paths.

MeshAgent proxies OpenAI, Anthropic, and Grok endpoints.

### OpenAI-Compatible

The main generation and realtime transports are:

| API | HTTP | WebSocket |
| - | - | - |
| Chat Completions | `/v1/chat/completions` | No |
| Responses | `/v1/responses` | `/v1/responses` |
| Realtime | `/v1/realtime` and `/v1/realtime/*` | `/v1/realtime` |

For WebSocket clients, use the same proxy host and change the URL scheme from `https` to `wss` (or `http` to `ws`). Send the same MeshAgent bearer credential and, for OAuth, the same project header during the WebSocket handshake.

* `/v1/chat/completions`
* `/v1/responses`
* `/v1/responses/compact`
* `/v1/responses/input_tokens`
* `/v1/embeddings`
* `/v1/audio/speech`
* `/v1/audio/transcriptions`
* `/v1/audio/translations`
* `/v1/models` and `/v1/models/*`
* `/v1/images/*`
* `/v1/realtime` and `/v1/realtime/*`

### Anthropic-Compatible

* `/v1/messages`
* `/v1/messages/count_tokens`
* `/v1/messages/batches*`
* `/v1/complete`
* `/v1/models` and `/v1/models/*`

### Grok/OpenAI-Compatible

* `/v1/messages`
* `/v1/responses` and `/v1/responses/*`
* `/v1/responses/compact`
* `/v1/responses/input_tokens`
* `/v1/models` and `/v1/models/*`

## Related Docs

* [Local CLI Proxy](./local_cli_proxy)
* [Use Codex and Claude with MeshAgent](./use_codex_claude)
* [Ask MeshAgent from the CLI](./ask_meshagent_cli)
* [OAuth Clients](../../project_admin/oauth)
* [Billing and Usage](../../project_admin/billing)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.