Quickstart

Create an API key, send a chat completion through the chokepoint, and find the record it left behind.

1. Create an API key

Each key carries its own scopes, model allowlist and daily/monthly spend caps.

1. Open API Keys

   In the sidebar, expand Admin and choose API Keys.

2. Add a provider key first, if you have not

   Developer keys hang off a provider credential (your OpenAI, Anthropic or Gemini key), which Forgebench stores encrypted and injects at call time. Your code never holds the provider key, so revoking it is one action here rather than a redeploy.

3. Mint a developer key

   Give it a name, set a daily and monthly cap, and tick the models it may reach. Both caps and the model list stay editable afterwards.

4. Copy the secret now

   The sk_... secret is shown once, at creation. Forgebench stores only a hash of it, so it cannot be shown again later.

2. Install an SDK

pip install "git+https://github.com/seedlinglabs/forgebench-sdk.git#subdirectory=sdk-py"

npm install "github:seedlinglabs/forgebench-sdk#path:/sdk-ts"

3. Send the call

Point the client at your workspace's API URL and pass the key. mock-gpt is a built-in test model that costs almost nothing, use it to prove the path works before you spend real provider money.

from forgebench import Forgebench

# api_key defaults to $FORGEBENCH_API_KEY, base_url to $FORGEBENCH_BASE_URL
client = Forgebench(api_key="sk_...", base_url="https://api.forgebench.ai")

resp = client.chat.completions.create(
  model="mock-gpt",
  messages=[{"role": "user", "content": "Prove the governed path works."}],
)

print(resp.choices[0].message.content)
print(resp.usage.total_tokens)

import { Forgebench } from "@seedlinglabs/forgebench-sdk";

const forgebench = new Forgebench({
baseUrl: "https://api.forgebench.ai",
apiKey: process.env.FORGEBENCH_API_KEY!,
});

const res = await forgebench.chat.create({
model: "mock-gpt",
messages: [{ role: "user", content: "Prove the governed path works." }],
});

console.log(res.choices[0]?.message.content);
console.log("tokens:", res.usage.total_tokens);

curl -X POST https://api.forgebench.ai/v1/chat/completions \
-H "Authorization: Bearer sk_..." \
-H "Content-Type: application/json" \
-d '{
  "model": "mock-gpt",
  "messages": [{"role": "user", "content": "Prove the governed path works."}]
}'

4. Read the response

The shape is OpenAI-compatible, so existing code mostly works unchanged, with one addition:

{
  "id": "chatcmpl-a5a25bcd-3632-4b60-9964-a0828565cdbc",
  "object": "chat.completion",
  "created": 1789101061,
  "model": "mock-gpt",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hello from the Forgebench mock model …" },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30 },
  "trace_id": "600902b2bd484291b7433f10ac4465f1"
}

trace_id is the addition, and it is the thread to pull on. It ties this response to its audit entry, its metering record and its trace, so "what did the agent do at 14:32" has a single answer you can look up rather than reconstruct.

5. Find the record it left

That one call wrote to several places at once. Each is a different question:

What happens when it is refused

Budgets are checked before the call is forwarded, so a refused call costs nothing: it never reaches the provider.

Over the cap, the chokepoint returns 402:

{
  "detail": {
    "code": "budget_exceeded",
    "message": "monthly budget exceeded",
    "limit_usd": "0.000100",
    "spent_usd": "0.000385",
    "attempted_cost_usd": "0.005000"
  }
}

The body tells you the cap, the spend against it, and what this call would have cost, enough to decide whether to raise the cap or fix the caller. The Python SDK raises this as BudgetExceededError.

Troubleshooting

Next