AWS Bedrock Integration

Overview

Track costs and monitor usage for Amazon Bedrock by sending Converse requests through LLM Ops. This integration is not a generic “swap the OpenAI base URL” flow: you call a dedicated path on the LLM Ops API that forwards to bedrock-runtime.{region}.amazonaws.com with AWS Signature Version 4 signing. This guide shows Python (requests), JavaScript (fetch), and cURL. Use the official AWS Converse API request body shape (messages, inferenceConfig, etc.).

Security: LLM Ops does not store your Cloudidr tracking token, AWS secret keys, request prompts, or response content in the analytics database—only usage metadata needed for cost analytics. The proxy must forward the JSON body to AWS to complete the call. AWS credentials are used in memory for the request to sign the upstream call; they are not written to our analytics tables. Never commit real keys; use IAM least privilege and environment variables.

Quick Start

LLM Ops API host: https://api.llm-ops.cloudidr.com
Endpoint pattern: POST /bedrock/model/{modelId}/converse
Example model ID in the path: amazon.nova-lite-v1:0 (dots and colons stay in the path segment).
Upstream API: Converse only—not InvokeModel or third-party SDK defaults that talk to AWS without the proxy.

You need two kinds of credentials on every request:

X-Cloudidr-Key — LLM Ops tracking token (trk_...).
AWS credentials — X-Aws-Access-Key-Id and X-Aws-Secret-Access-Key (and optionally STS session token) so the proxy can sign the Bedrock request.

API Keys and configuration

Credential	Purpose
Cloudidr Key	From the LLM Ops dashboard; typically `trk_...`
AWS Access Key ID	IAM user or role with `bedrock:InvokeModel` (and model access) in the target account
AWS Secret Access Key	Paired with the access key
AWS Session Token (optional)	For temporary STS credentials

The marketing site llmfinops.ai points at the same product; the dashboard URL above is the canonical app host. Environment variables (recommended):

export CLOUDIDR_KEY="trk_..."
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="us-east-1"
# Optional for STS:
# export AWS_SESSION_TOKEN="..."

Integration Examples

Python
JavaScript
cURL

Install

pip install requests

The AWS SDK for Python (boto3) calls AWS directly. To route through LLM Ops, use HTTP (e.g. requests) with the URL and headers below—matching how our smoke tests exercise the proxy.

Basic Example

import os
import requests

BASE = "https://api.llm-ops.cloudidr.com"
MODEL_ID = "amazon.nova-lite-v1:0"

url = f"{BASE}/bedrock/model/{MODEL_ID}/converse"
headers = {
    "Content-Type": "application/json",
    "X-Cloudidr-Key": os.environ["CLOUDIDR_KEY"],
    "X-Aws-Access-Key-Id": os.environ["AWS_ACCESS_KEY_ID"],
    "X-Aws-Secret-Access-Key": os.environ["AWS_SECRET_ACCESS_KEY"],
    "X-Aws-Region": os.environ.get("AWS_REGION", "us-east-1"),
}
# Optional: "X-Aws-Session-Token": os.environ["AWS_SESSION_TOKEN"],

body = {
    "messages": [
        {"role": "user", "content": [{"text": "What is the capital of France?"}]}
    ],
    "inferenceConfig": {"maxTokens": 256},
}

resp = requests.post(url, headers=headers, json=body, timeout=120)
resp.raise_for_status()
data = resp.json()
text = data["output"]["message"]["content"][0].get("text", "")
print(text)
print("usage:", data.get("usage"))

With Metadata (Department / Project / Agent)

import os
import requests

BASE = "https://api.llm-ops.cloudidr.com"
MODEL_ID = "amazon.nova-lite-v1:0"
url = f"{BASE}/bedrock/model/{MODEL_ID}/converse"

headers = {
    "Content-Type": "application/json",
    "X-Cloudidr-Key": os.environ["CLOUDIDR_KEY"],
    "X-Aws-Access-Key-Id": os.environ["AWS_ACCESS_KEY_ID"],
    "X-Aws-Secret-Access-Key": os.environ["AWS_SECRET_ACCESS_KEY"],
    "X-Aws-Region": os.environ.get("AWS_REGION", "us-east-1"),
    "X-Department": "engineering",
    "X-Project": "backend",
    "X-Agent": "bedrock-assistant",
}

body = {
    "messages": [
        {"role": "user", "content": [{"text": "Say hello in one sentence."}]}
    ],
    "inferenceConfig": {"maxTokens": 128},
}

resp = requests.post(url, headers=headers, json=body, timeout=120)

Parsing the response (Converse shape)

Successful responses include output, usage, and stopReason, for example:

data["usage"]["inputTokens"], data["usage"]["outputTokens"]
Assistant text: data["output"]["message"]["content"] (list of blocks; often a block with "text")

Some models may return additional block types (e.g. reasoning); iterate over content as needed.

Basic Example (Node 18+ or browser with `fetch`)

const BASE = 'https://api.llm-ops.cloudidr.com';
const MODEL_ID = 'amazon.nova-lite-v1:0';

const url = `${BASE}/bedrock/model/${MODEL_ID}/converse`;

const body = {
  messages: [
    { role: 'user', content: [{ text: 'What is the capital of France?' }] },
  ],
  inferenceConfig: { maxTokens: 256 },
};

const res = await fetch(url, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-Cloudidr-Key': process.env.CLOUDIDR_KEY,
    'X-Aws-Access-Key-Id': process.env.AWS_ACCESS_KEY_ID,
    'X-Aws-Secret-Access-Key': process.env.AWS_SECRET_ACCESS_KEY,
    'X-Aws-Region': process.env.AWS_REGION || 'us-east-1',
  },
  body: JSON.stringify(body),
});

if (!res.ok) {
  throw new Error(await res.text());
}
const data = await res.json();
const text = data.output?.message?.content?.find((b) => b.text)?.text;
console.log(text);
console.log('usage', data.usage);

In browsers, avoid embedding long-lived AWS keys; prefer a backend that injects headers or short-lived STS credentials.

With Metadata

const res = await fetch(url, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-Cloudidr-Key': process.env.CLOUDIDR_KEY,
    'X-Aws-Access-Key-Id': process.env.AWS_ACCESS_KEY_ID,
    'X-Aws-Secret-Access-Key': process.env.AWS_SECRET_ACCESS_KEY,
    'X-Aws-Region': process.env.AWS_REGION || 'us-east-1',
    'X-Department': 'engineering',
    'X-Project': 'backend',
    'X-Agent': 'bedrock-assistant',
  },
  body: JSON.stringify(body),
});

Basic Example

curl -sS -X POST \
  "https://api.llm-ops.cloudidr.com/bedrock/model/amazon.nova-lite-v1:0/converse" \
  -H "Content-Type: application/json" \
  -H "X-Cloudidr-Key: trk_..." \
  -H "X-Aws-Access-Key-Id: AKIA..." \
  -H "X-Aws-Secret-Access-Key: ..." \
  -H "X-Aws-Region: us-east-1" \
  -d '{
    "messages": [
      {
        "role": "user",
        "content": [{"text": "What is the capital of France?"}]
      }
    ],
    "inferenceConfig": {"maxTokens": 256}
  }'

With Metadata

curl -sS -X POST \
  "https://api.llm-ops.cloudidr.com/bedrock/model/amazon.nova-lite-v1:0/converse" \
  -H "Content-Type: application/json" \
  -H "X-Cloudidr-Key: trk_..." \
  -H "X-Aws-Access-Key-Id: AKIA..." \
  -H "X-Aws-Secret-Access-Key: ..." \
  -H "X-Aws-Region: us-east-1" \
  -H "X-Department: engineering" \
  -H "X-Project: backend" \
  -H "X-Agent: bedrock-assistant" \
  -d '{
    "messages": [
      {"role": "user", "content": [{"text": "Say hello in one sentence."}]}
    ],
    "inferenceConfig": {"maxTokens": 128}
  }'

Temporary AWS credentials (STS)

# Add:
# -H "X-Aws-Session-Token: ..."

Required and optional headers

Header	Required	Description
`X-Cloudidr-Key`	Yes	LLM Ops tracking token (`trk_...`)
`X-Aws-Access-Key-Id`	Yes	AWS access key ID
`X-Aws-Secret-Access-Key`	Yes	AWS secret access key
`X-Aws-Session-Token`	No	STS session token when using temporary credentials
`X-Aws-Region`	No	AWS region for Bedrock runtime (default `us-east-1` if omitted)
`X-Department`	No	Cost attribution: department
`X-Project`	No	Cost attribution: project/team (preferred)
`X-Team`	No	Legacy alias for the same tag as `X-Project`
`X-Agent`	No	Cost attribution: agent or app name

Model IDs and cross-region inference

The {modelId} in the URL is passed to AWS (after normalization). For some providers, Bedrock expects a cross-region inference profile ID (prefix us., eu., or ap.). The proxy may automatically prepend the right prefix when you use a plain ID for:

anthropic.*
meta.*
deepseek.*

The geo prefix is derived from X-Aws-Region (e.g. eu-* → eu., ap-* → ap., otherwise often us.). If your model ID already starts with us., eu., or ap., it is left unchanged. Many Amazon models (e.g. Amazon Nova) use the plain model ID without a us. prefix. Always confirm the exact ID in the AWS Bedrock console for your account and region. Anthropic models on Bedrock: Your account must have model access enabled; Anthropic often requires completing the use-case form under Bedrock → Model access → Anthropic in the AWS console.

Converse features (streaming, tools)

The proxy forwards the Converse JSON body to AWS. Features such as streaming (stream in the Converse request) or tool configuration follow the AWS Converse API specification. Refer to the latest AWS documentation for field names and behavior; LLM Ops records usage from successful responses when token counts are present.

Supported Models

Models available to your AWS account in the chosen region can be used. See the Supported Models page for pricing alignment in LLM Ops.

What Gets Tracked

LLM Ops automatically captures: ✅ Token usage — Input and output tokens from the Converse usage object where available
✅ Cost — Estimated cost from LLM Ops pricing
✅ Latency — Request duration and provider timing
✅ Model — Effective model ID used for Bedrock
✅ Metadata — Department, project/team, agent
✅ Errors — HTTP status and summarized error messages
✅ Optimizer — When enabled, routing metadata for cost-optimizer decisions

What we do not persist in the analytics database:

❌ Cloudidr or AWS secret keys
❌ Full prompt or completion text
❌ Raw request/response bodies as searchable content

Operational logging in your environment may still exist; treat headers and bodies as sensitive in transit.

View Your Data

After making requests, view costs in the LLM Ops Dashboard:

Agent Explorer — Costs by agent
Department Breakdown — Department spending
Team Analysis — Project/team-level costs
Model Comparison — Compare models (including Bedrock IDs)
Time Series — Spend over time

Migration from calling Bedrock directly

Previously you might have called: POST https://bedrock-runtime.{region}.amazonaws.com/model/{modelId}/converse with SigV4 from your code. With LLM Ops:

Change the URL to https://api.llm-ops.cloudidr.com/bedrock/model/{modelId}/converse.
Add X-Cloudidr-Key: trk_... on every request.
Pass AWS credentials using the headers above so the proxy can sign the upstream Bedrock call (you can remove local SigV4 signing when using the proxy, unless you keep a different architecture).

Keep the same Converse JSON body you used with AWS.

# Conceptual diff
- POST https://bedrock-runtime.us-east-1.amazonaws.com/model/MODEL/converse  (signed by you)
+ POST https://api.llm-ops.cloudidr.com/bedrock/model/MODEL/converse
+   Headers: X-Cloudidr-Key, X-Aws-Access-Key-Id, X-Aws-Secret-Access-Key, X-Aws-Region, ...

Troubleshooting

400 Missing X-Cloudidr-Key

Every request must include X-Cloudidr-Key with a valid LLM Ops tracking token. Get or rotate tokens in the dashboard.

401 AWS authentication failed

Check X-Aws-Access-Key-Id, X-Aws-Secret-Access-Key, and optional X-Aws-Session-Token. Ensure the IAM principal can invoke Bedrock in X-Aws-Region. Keys must not be swapped with the Cloudidr token.

403 / 404 Model access

Enable the model in Amazon Bedrock → Model access for your account and region. For Anthropic, complete the console use-case step if required.

Wrong region or model ID

Set X-Aws-Region to the region where the model is available. Verify the model ID string (including version suffixes like :0) matches AWS documentation.

Cost data not appearing

Allow 10–30 seconds for dashboard updates. Confirm HTTP 200 from the proxy and a valid X-Cloudidr-Key so the request is attributed to your org.

Next Steps

View Dashboard

See your OpenAI API costs in real-time

Supported Models

View all supported OpenAI models

Anthropic Integration

Add cost tracking for Claude models

Set Budgets

Configure spending alerts and limits

Get Started

LLM Ops

Flex Compute

Overview

Quick Start

API Keys and configuration

Integration Examples

Install

Basic Example

With Metadata (Department / Project / Agent)

Parsing the response (Converse shape)

Basic Example (Node 18+ or browser with `fetch`)

With Metadata

Basic Example

With Metadata

Temporary AWS credentials (STS)

Required and optional headers

Model IDs and cross-region inference

Converse features (streaming, tools)

Supported Models

What Gets Tracked

View Your Data

Migration from calling Bedrock directly

Troubleshooting

Next Steps

View Dashboard

Supported Models

Anthropic Integration

Set Budgets

Get Started

LLM Ops

Flex Compute

​Overview

​Quick Start

​API Keys and configuration

​Integration Examples

​Install

​Basic Example

​With Metadata (Department / Project / Agent)

​Parsing the response (Converse shape)

​Basic Example (Node 18+ or browser with fetch)

​With Metadata

​Basic Example

​With Metadata

​Temporary AWS credentials (STS)

​Required and optional headers

​Model IDs and cross-region inference

​Converse features (streaming, tools)

​Supported Models

​What Gets Tracked

​View Your Data

​Migration from calling Bedrock directly

​Troubleshooting

​Next Steps

View Dashboard

Supported Models

Anthropic Integration

Set Budgets

Overview

Quick Start

API Keys and configuration

Integration Examples

Install

Basic Example

With Metadata (Department / Project / Agent)

Parsing the response (Converse shape)

Basic Example (Node 18+ or browser with `fetch`)

With Metadata

Basic Example

With Metadata

Temporary AWS credentials (STS)

Required and optional headers

Model IDs and cross-region inference

Converse features (streaming, tools)

Supported Models

What Gets Tracked

View Your Data

Migration from calling Bedrock directly

Troubleshooting

Next Steps