> ## Documentation Index > Fetch the complete documentation index at: https://docs.cloudidr.com/llms.txt > Use this file to discover all available pages before exploring further. # AWS Bedrock Integration > Learn how to integrate LLM Ops with AWS Bedrock (Converse API) for cost tracking and monitoring ## Overview Track costs and monitor usage for **Amazon Bedrock** by sending **Converse** requests through LLM Ops. This integration is **not** a generic “swap the OpenAI base URL” flow: you call a dedicated path on the LLM Ops API that forwards to `bedrock-runtime.{region}.amazonaws.com` with AWS Signature Version 4 signing. This guide shows **Python** (`requests`), **JavaScript** (`fetch`), and **cURL**. Use the official [AWS Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) request body shape (`messages`, `inferenceConfig`, etc.). **Security:** LLM Ops does not store your **Cloudidr** tracking token, **AWS secret keys**, request prompts, or response content in the **analytics database**—only usage metadata needed for cost analytics. The proxy must forward the JSON body to AWS to complete the call. AWS credentials are used **in memory for the request** to sign the upstream call; they are **not** written to our analytics tables. Never commit real keys; use IAM least privilege and environment variables. ## Quick Start * **LLM Ops API host:** `https://api.llm-ops.cloudidr.com` * **Endpoint pattern:** `POST /bedrock/model/{modelId}/converse`\ Example model ID in the path: `amazon.nova-lite-v1:0` (dots and colons stay in the path segment). * **Upstream API:** [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) only—not `InvokeModel` or third-party SDK defaults that talk to AWS without the proxy. You need **two** kinds of credentials on every request: 1. **`X-Cloudidr-Key`** — LLM Ops tracking token (`trk_...`). 2. **AWS credentials** — `X-Aws-Access-Key-Id` and `X-Aws-Secret-Access-Key` (and optionally STS session token) so the proxy can sign the Bedrock request. ## API Keys and configuration | Credential | Purpose | | -------------------------------- | ----------------------------------------------------------------------------------------- | | **Cloudidr Key** | From the [LLM Ops dashboard](https://llm-ops.cloudidr.com/dashboard); typically `trk_...` | | **AWS Access Key ID** | IAM user or role with `bedrock:InvokeModel` (and model access) in the target account | | **AWS Secret Access Key** | Paired with the access key | | **AWS Session Token** (optional) | For temporary STS credentials | The marketing site [llmfinops.ai](https://llmfinops.ai) points at the same product; the dashboard URL above is the canonical app host. Environment variables (recommended): ```bash theme={null} export CLOUDIDR_KEY="trk_..." export AWS_ACCESS_KEY_ID="AKIA..." export AWS_SECRET_ACCESS_KEY="..." export AWS_REGION="us-east-1" # Optional for STS: # export AWS_SESSION_TOKEN="..." ``` *** ## Integration Examples ### Install ```bash theme={null} pip install requests ``` The AWS SDK for Python (`boto3`) calls AWS **directly**. To route through LLM Ops, use HTTP (e.g. `requests`) with the URL and headers below—matching how our [smoke tests](../test/smoke/test_bedrock_models.py) exercise the proxy. ### Basic Example ```python theme={null} import os import requests BASE = "https://api.llm-ops.cloudidr.com" MODEL_ID = "amazon.nova-lite-v1:0" url = f"{BASE}/bedrock/model/{MODEL_ID}/converse" headers = { "Content-Type": "application/json", "X-Cloudidr-Key": os.environ["CLOUDIDR_KEY"], "X-Aws-Access-Key-Id": os.environ["AWS_ACCESS_KEY_ID"], "X-Aws-Secret-Access-Key": os.environ["AWS_SECRET_ACCESS_KEY"], "X-Aws-Region": os.environ.get("AWS_REGION", "us-east-1"), } # Optional: "X-Aws-Session-Token": os.environ["AWS_SESSION_TOKEN"], body = { "messages": [ {"role": "user", "content": [{"text": "What is the capital of France?"}]} ], "inferenceConfig": {"maxTokens": 256}, } resp = requests.post(url, headers=headers, json=body, timeout=120) resp.raise_for_status() data = resp.json() text = data["output"]["message"]["content"][0].get("text", "") print(text) print("usage:", data.get("usage")) ``` ### With Metadata (Department / Project / Agent) ```python theme={null} import os import requests BASE = "https://api.llm-ops.cloudidr.com" MODEL_ID = "amazon.nova-lite-v1:0" url = f"{BASE}/bedrock/model/{MODEL_ID}/converse" headers = { "Content-Type": "application/json", "X-Cloudidr-Key": os.environ["CLOUDIDR_KEY"], "X-Aws-Access-Key-Id": os.environ["AWS_ACCESS_KEY_ID"], "X-Aws-Secret-Access-Key": os.environ["AWS_SECRET_ACCESS_KEY"], "X-Aws-Region": os.environ.get("AWS_REGION", "us-east-1"), "X-Department": "engineering", "X-Project": "backend", "X-Agent": "bedrock-assistant", } body = { "messages": [ {"role": "user", "content": [{"text": "Say hello in one sentence."}]} ], "inferenceConfig": {"maxTokens": 128}, } resp = requests.post(url, headers=headers, json=body, timeout=120) ``` ### Parsing the response (Converse shape) Successful responses include `output`, `usage`, and `stopReason`, for example: * `data["usage"]["inputTokens"]`, `data["usage"]["outputTokens"]` * Assistant text: `data["output"]["message"]["content"]` (list of blocks; often a block with `"text"`) Some models may return additional block types (e.g. reasoning); iterate over `content` as needed. ### Basic Example (Node 18+ or browser with `fetch`) ```javascript theme={null} const BASE = 'https://api.llm-ops.cloudidr.com'; const MODEL_ID = 'amazon.nova-lite-v1:0'; const url = `${BASE}/bedrock/model/${MODEL_ID}/converse`; const body = { messages: [ { role: 'user', content: [{ text: 'What is the capital of France?' }] }, ], inferenceConfig: { maxTokens: 256 }, }; const res = await fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Cloudidr-Key': process.env.CLOUDIDR_KEY, 'X-Aws-Access-Key-Id': process.env.AWS_ACCESS_KEY_ID, 'X-Aws-Secret-Access-Key': process.env.AWS_SECRET_ACCESS_KEY, 'X-Aws-Region': process.env.AWS_REGION || 'us-east-1', }, body: JSON.stringify(body), }); if (!res.ok) { throw new Error(await res.text()); } const data = await res.json(); const text = data.output?.message?.content?.find((b) => b.text)?.text; console.log(text); console.log('usage', data.usage); ``` In **browsers**, avoid embedding long-lived AWS keys; prefer a backend that injects headers or short-lived STS credentials. ### With Metadata ```javascript theme={null} const res = await fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Cloudidr-Key': process.env.CLOUDIDR_KEY, 'X-Aws-Access-Key-Id': process.env.AWS_ACCESS_KEY_ID, 'X-Aws-Secret-Access-Key': process.env.AWS_SECRET_ACCESS_KEY, 'X-Aws-Region': process.env.AWS_REGION || 'us-east-1', 'X-Department': 'engineering', 'X-Project': 'backend', 'X-Agent': 'bedrock-assistant', }, body: JSON.stringify(body), }); ``` ### Basic Example ```bash theme={null} curl -sS -X POST \ "https://api.llm-ops.cloudidr.com/bedrock/model/amazon.nova-lite-v1:0/converse" \ -H "Content-Type: application/json" \ -H "X-Cloudidr-Key: trk_..." \ -H "X-Aws-Access-Key-Id: AKIA..." \ -H "X-Aws-Secret-Access-Key: ..." \ -H "X-Aws-Region: us-east-1" \ -d '{ "messages": [ { "role": "user", "content": [{"text": "What is the capital of France?"}] } ], "inferenceConfig": {"maxTokens": 256} }' ``` ### With Metadata ```bash theme={null} curl -sS -X POST \ "https://api.llm-ops.cloudidr.com/bedrock/model/amazon.nova-lite-v1:0/converse" \ -H "Content-Type: application/json" \ -H "X-Cloudidr-Key: trk_..." \ -H "X-Aws-Access-Key-Id: AKIA..." \ -H "X-Aws-Secret-Access-Key: ..." \ -H "X-Aws-Region: us-east-1" \ -H "X-Department: engineering" \ -H "X-Project: backend" \ -H "X-Agent: bedrock-assistant" \ -d '{ "messages": [ {"role": "user", "content": [{"text": "Say hello in one sentence."}]} ], "inferenceConfig": {"maxTokens": 128} }' ``` ### Temporary AWS credentials (STS) ```bash theme={null} # Add: # -H "X-Aws-Session-Token: ..." ``` *** ## Required and optional headers | Header | Required | Description | | ------------------------- | -------- | ------------------------------------------------------------------- | | `X-Cloudidr-Key` | Yes | LLM Ops tracking token (`trk_...`) | | `X-Aws-Access-Key-Id` | Yes | AWS access key ID | | `X-Aws-Secret-Access-Key` | Yes | AWS secret access key | | `X-Aws-Session-Token` | No | STS session token when using temporary credentials | | `X-Aws-Region` | No | AWS region for Bedrock runtime (default **`us-east-1`** if omitted) | | `X-Department` | No | Cost attribution: department | | `X-Project` | No | Cost attribution: project/team (preferred) | | `X-Team` | No | Legacy alias for the same tag as `X-Project` | | `X-Agent` | No | Cost attribution: agent or app name | *** ## Model IDs and cross-region inference The `{modelId}` in the URL is passed to AWS (after normalization). For some providers, Bedrock expects a **cross-region inference profile** ID (prefix `us.`, `eu.`, or `ap.`). The proxy may **automatically prepend** the right prefix when you use a plain ID for: * `anthropic.*` * `meta.*` * `deepseek.*` The geo prefix is derived from **`X-Aws-Region`** (e.g. `eu-*` → `eu.`, `ap-*` → `ap.`, otherwise often `us.`). If your model ID **already** starts with `us.`, `eu.`, or `ap.`, it is left unchanged. Many Amazon models (e.g. **Amazon Nova**) use the **plain** model ID without a `us.` prefix. Always confirm the exact ID in the [AWS Bedrock console](https://console.aws.amazon.com/bedrock) for your account and region. **Anthropic models on Bedrock:** Your account must have model access enabled; Anthropic often requires completing the **use-case** form under **Bedrock → Model access → Anthropic** in the AWS console. *** ## Converse features (streaming, tools) The proxy forwards the **Converse** JSON body to AWS. Features such as **streaming** (`stream` in the Converse request) or **tool configuration** follow the **AWS Converse API** specification. Refer to the latest AWS documentation for field names and behavior; LLM Ops records usage from successful responses when token counts are present. *** ## Supported Models Models available to your AWS account in the chosen region can be used. See the [Supported Models](/guides/llm-ops/supported-models) page for pricing alignment in LLM Ops. *** ## What Gets Tracked LLM Ops automatically captures: ✅ **Token usage** — Input and output tokens from the Converse `usage` object where available\ ✅ **Cost** — Estimated cost from LLM Ops pricing\ ✅ **Latency** — Request duration and provider timing\ ✅ **Model** — Effective model ID used for Bedrock\ ✅ **Metadata** — Department, project/team, agent\ ✅ **Errors** — HTTP status and summarized error messages\ ✅ **Optimizer** — When enabled, routing metadata for cost-optimizer decisions **What we do not persist in the analytics database:** * ❌ Cloudidr or AWS secret keys * ❌ Full prompt or completion text * ❌ Raw request/response bodies as searchable content Operational logging in your environment may still exist; treat headers and bodies as sensitive in transit. *** ## View Your Data After making requests, view costs in the [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard): * **Agent Explorer** — Costs by agent * **Department Breakdown** — Department spending * **Team Analysis** — Project/team-level costs * **Model Comparison** — Compare models (including Bedrock IDs) * **Time Series** — Spend over time *** ## Migration from calling Bedrock directly Previously you might have called: `POST https://bedrock-runtime.{region}.amazonaws.com/model/{modelId}/converse` with SigV4 from your code. With LLM Ops: 1. **Change the URL** to `https://api.llm-ops.cloudidr.com/bedrock/model/{modelId}/converse`. 2. **Add** `X-Cloudidr-Key: trk_...` on every request. 3. **Pass AWS credentials** using the headers above so the **proxy** can sign the upstream Bedrock call (you can remove local SigV4 signing when using the proxy, unless you keep a different architecture). Keep the **same Converse JSON body** you used with AWS. ```text theme={null} # Conceptual diff - POST https://bedrock-runtime.us-east-1.amazonaws.com/model/MODEL/converse (signed by you) + POST https://api.llm-ops.cloudidr.com/bedrock/model/MODEL/converse + Headers: X-Cloudidr-Key, X-Aws-Access-Key-Id, X-Aws-Secret-Access-Key, X-Aws-Region, ... ``` *** ## Troubleshooting Every request must include **`X-Cloudidr-Key`** with a valid LLM Ops tracking token. Get or rotate tokens in the dashboard. Check **`X-Aws-Access-Key-Id`**, **`X-Aws-Secret-Access-Key`**, and optional **`X-Aws-Session-Token`**. Ensure the IAM principal can invoke Bedrock in **`X-Aws-Region`**. Keys must not be swapped with the Cloudidr token. Enable the model in **Amazon Bedrock → Model access** for your account and region. For Anthropic, complete the console use-case step if required. Set **`X-Aws-Region`** to the region where the model is available. Verify the model ID string (including version suffixes like `:0`) matches AWS documentation. Allow **10–30 seconds** for dashboard updates. Confirm HTTP **200** from the proxy and a valid **`X-Cloudidr-Key`** so the request is attributed to your org. *** ## Next Steps See your OpenAI API costs in real-time View all supported OpenAI models Add cost tracking for Claude models Configure spending alerts and limits