# Cancel Capacity
Source: https://docs.cloudidr.com/api-reference/cancel-capacity

/api-reference/flex-compute/flex-api.json post /external/api/v1/flexcompute/cancel
Cancel a capacity reservation policy with applicable refunds based on cancellation timing.



# Configure Capacity
Source: https://docs.cloudidr.com/api-reference/configure-capacity

/api-reference/flex-compute/flex-api.json put /external/api/v1/flexcompute/capacity-configuration
Update capacity configuration settings for a policy.



# Describe Capacity
Source: https://docs.cloudidr.com/api-reference/describe-capacity

/api-reference/flex-compute/flex-api.json get /external/api/v1/flexcompute/describe/{cidr_id}
This response provides detailed view information for a schedule policy, including policy details, region settings, instances, dates, disaster recovery events, and pricing.



# Describe Cyber Policy
Source: https://docs.cloudidr.com/api-reference/describe-cyber-policy

/api-reference/disaster-recovery/dr-api.json get /external/api/v1/policies/cyber/{cidr_id}/view
This response provides detailed view information for a cyber policy, including policy details, region settings, instances, dates, disaster recovery events, and pricing.



# Describe Linked Accounts
Source: https://docs.cloudidr.com/api-reference/describe-linked-accounts

/api-reference/auth-api.json get /external/api/v1/my-linked-accounts
Retrieve a list of all cloud provider accounts linked to the authenticated user.



# Using DR/Cyber APIs
Source: https://docs.cloudidr.com/api-reference/disaster-recovery/dr-api-usage

This document serves as a comprehensive guide for utilizing the Cloudidr Cyber Policy APIs. It outlines the necessary steps to view cyber policy details, list policies for customers, initiate disaster recovery events, and terminate ongoing events. Each section provides clear instructions and highlights potential errors that may arise during API interactions, ensuring users can effectively manage their cyber policies.

> ##### **Viewing Cyber Policy Details**

To view details of a specific cyber policy, make a **GET** request to the endpoint with the required `cidr_id`. Include the authorization token in the request headers. If the `cidr_id` is valid, the response will contain detailed policy information, including region settings, instances, and disaster recovery events. If the `cidr_id` is incorrect, you will receive a **404 Not Found** error. Ensure that the access token is valid to avoid a **401 Unauthorized** error.

> ##### **Listing Cyber Policies for a Customer**

To fetch all cyber policies associated with a customer, send a **GET** request to the endpoint using the `cid_ext` identifier. You can include optional query parameters like `page`, `pageSize`, `sort`, `policyStatus`, and `policyName` to refine the results. If policies exist for the customer, they will be listed in the response. If none are found, an empty result set will be returned. Invalid or missing authentication will lead to a **401 Unauthorized** error, and server issues may result in a **500 Internal Server Error**.

> ##### **Initiating a Cyber Policy Event**

To initiate a disaster recovery event, send a **POST** request to the endpoint with the `cidr_id`. The authorization token must be included in the headers. If the event is successfully created, the response will confirm its initiation. If an event has already been triggered, the response will indicate that no new event is required. If the `cidr_id` does not exist, a **404 Not Found** error will be returned. Invalid authentication will lead to a **401 Unauthorized** error.

> ##### **Terminating a Cyber Policy Event**

To terminate an ongoing disaster recovery event, send a **POST** request to the endpoint with the `cidr_id`. Ensure the request includes a valid authorization token. A successful response confirms event termination. If the event has already been terminated, the response will indicate that no further action is needed. If the `cidr_id` is incorrect, a **404 Not Found** error will occur. An invalid token will result in a **401 Unauthorized** error.

Ensure all API requests include valid authentication and correct identifiers to avoid errors.


# Using Flex Compute APIs
Source: https://docs.cloudidr.com/api-reference/flex-compute/flex-api-usage

This document provides a comprehensive guide for users on how to effectively utilize the APIs for signing in, searching for instances, reserving, and purchasing them. By following the outlined steps, users can seamlessly navigate through the API functionalities to manage their cloud resources.

> ##### **Step 1: Sign in to the system**

To begin using the APIs, users must first authenticate themselves by making a call to the **Sign In API**. This step is crucial as it establishes a secure session for the user, allowing access to the subsequent functionalities.

> ##### **Step 2: Search for Instances**

Once signed in, users can utilize the **Search API** to explore the various instances available. This API call enables users to filter and find instances that meet their specific requirements, ensuring they have a range of options to choose from. You will need to specify the AWS designation for region with zone id, instance family, instance size and when your start and end times. End time can be changed during instance reservation and afterwards as well.

> ##### **Step 3: Reserve an Instance**

After identifying a suitable instance, users can proceed to check its availability by invoking the **Reserve API**. This API will confirm whether the selected instance can be reserved for purchase. The user needs to specify the AWS regions/zone, Platform (Linux/Unix), start and end time.

##### **If the Instance is Available**

If the instance is available for reservation, users can move forward to purchase it by using the **Purchase API**. This step finalizes the transaction, allowing users to acquire the instance for their cloud account.

##### **If the Instance is Not Available**

In the event that the instance is not available for reservation, users will need to select an alternative instance. They can initiate the Reserve API again to check the availability of the new selection.

> ##### **Step 4: Advanced Configuration (Optional)**

Several parameters exist to configure security groups, VPC, and EBS volume sizes before the instance is purchased. Not these parameters cannot be changed after the instance has been purchased.

> ##### **Step 5: Accessing the Purchased Instance**

Once the purchase is successfully completed, the instance will be added to the user's AWS cloud account. Users can then access and manage their newly acquired resources as needed.

> ##### **Step 6: Cancel Purchased Instance**

After the instance is purchased, user can cancel for free within 10 minutes of purchase and before 24 hours of start time (free of charge). After that time a penalty of 20% is deducted for the remaining operating duration left for the instance.

> ##### **Step 7: One time renewal or Auto-renewal**

During reservation and after purchase, customer have option to renew the instances for 24 hours, 7 days or 30 days. Use can also turn on auto-renewal for these duration. Renewal is only honored and charged if the instance is available for the new duration.


# Initiate DR Event
Source: https://docs.cloudidr.com/api-reference/initiate-dr-event

/api-reference/disaster-recovery/dr-api.json post /external/api/v1/policies/cyber/initiate/{cidr_id}
Initiate a disaster recovery event for a cyber policy. A unique identifier representing the cyber policy or customer view.



# List of Capacities
Source: https://docs.cloudidr.com/api-reference/list-of-capacities

/api-reference/flex-compute/flex-api.json get /external/api/v1/flexcompute/capacities
Retrieve a paginated list of FlexCompute capacity policies with optional filtering and sorting.



# List of Cyber Policies
Source: https://docs.cloudidr.com/api-reference/list-of-cyber-policies

/api-reference/disaster-recovery/dr-api.json get /external/api/v1/policies/cyber/{cid_ext}/customer
Retrieve a paginated list of cyber/DR policies for a specific customer with optional filtering and sorting.



# Purchase Capacity
Source: https://docs.cloudidr.com/api-reference/purchase-capacity

/api-reference/flex-compute/flex-api.json post /external/api/v1/flexcompute/purchase
Purchase a reserved capacity policy.



# Renew Capacity
Source: https://docs.cloudidr.com/api-reference/renew-capacity

/api-reference/flex-compute/flex-api.json put /external/api/v1/flexcompute/capacity/auto-renewal
Update auto-renewal configuration for a capacity policy.



# Reserve Capacity
Source: https://docs.cloudidr.com/api-reference/reserve-capacity

/api-reference/flex-compute/flex-api.json post /external/api/v1/flexcompute/reserve
Create a reservation policy. Times must be in 15-minute increments.



# Search Capacity
Source: https://docs.cloudidr.com/api-reference/search-capacity

/api-reference/flex-compute/flex-api.json get /external/api/v1/flexcompute/search
Search for available EC2 instance type offerings within a specific availability zone and time window.



# Sign In
Source: https://docs.cloudidr.com/api-reference/sign-in

/api-reference/auth-api.json post /external/api/v1/sign-in
Authenticate and obtain an access token (JWT) for API requests. The access token must be included in the Authorization header as 'Bearer <token>' for all subsequent API calls.



# Terminate DR Event
Source: https://docs.cloudidr.com/api-reference/terminate-dr-event

/api-reference/disaster-recovery/dr-api.json post /external/api/v1/policies/cyber/terminate/{cidr_id}
Terminate a disaster recovery event for a cyber policy. A unique identifier representing the cyber policy or customer view.



# Evaluations
Source: https://docs.cloudidr.com/evaluations



Use this tab to **measure routing in practice**: the same prompt runs on your chosen **baseline** model and on a second model in parallel, with **LLM-as-judge** scoring and an **eval history** of your last 20 runs per user in the org.

## What this tab is for

* See whether a cheaper routed model stays "good enough" for your prompts.
* See **cost and latency savings** when routing applies.
* Keep a short **history** of runs (with delete) for demos or regression checks.

The Evaluations tab has two modes selectable at the top:

| Mode                        | Description                                                                                                                                |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| **Smart Routing**           | Cloudidr automatically picks the comparison model based on your routing strategy. Requires LLM Optimization to be **on**.                  |
| **Manual Model Comparison** | You choose both the baseline and comparison models from any provider. Useful for evaluating specific model pairs independently of routing. |

<Frame>
  <img alt="Screenshot 2026 05 08 At 21 12 54" />
</Frame>

## LLM Optimization status — required for Smart Routing

A status card shows whether **LLM Optimization** is on and which **strategy** applies (e.g. **Intra Provider** vs **Flexible**).

> **Important:** If Model Routing is **off**, **Run Eval** is disabled in Smart Routing mode — there is no routed path to compare. Turn optimization on under **LLM** **Optimizer Settings**, or switch to **Manual Model Comparison** mode.

## Configuration

* **Baseline model** — same model picker as Try a Model.
* **Comparison model** — in **Manual** mode, you pick this yourself from any provider. In **Smart Routing** mode, Cloudidr selects it automatically.
* **Judge model** — pick who scores the two answers:
  * **Cloudidr** (free): Gemma 3 27B or Qwen 3.5 27B or similar model is used — no API key needed.
  * **Your provider key**: top-tier options per provider, for example:
    * OpenAI: GPT-5.5 Pro / GPT-5.5 / GPT-5.4
    * Anthropic: Claude Opus 4.6 / Claude Sonnet 4.6
    * Google: Gemini 3.1 Pro / Gemini 2.5 Flash

Judge scores are **subjective**. The UI notes that **accuracy** can be unreliable for **very recent events** because models have knowledge cutoffs.

## Run Eval

Runs both models **in parallel**, then runs the judge. Results appear side by side; savings percentage and verdict show below.

<Frame>
  <img alt="Screenshot 2026 05 01 At 21 46 51" />
</Frame>

## When routing does not apply — Smart Routing mode only

The UI explains two cases where no routing substitute is used:

1. **Recency protection** — the prompt matched recency signals; Cloudidr keeps the baseline model. You can turn **Recency protection** off under **Optimizer Settings** if you accept routing for those prompts.
2. **Complex / no substitute** — the prompt is classified as too complex for routing, or no cheaper mapped model exists. **Flexible** routing does **not** override the "complex" classification; simplifying the prompt is the practical path.

## Verdict and scores

* **Verdict** — whether the comparison answer is **Better**, **Equivalent**, or **Worse** (derived from the score delta), or **No routing** / **Too complex to route** in Smart Routing mode.
* **Score** — overall 1–10 score with a **criteria breakdown** (Accuracy, Completeness, Clarity, Practical usefulness) when the judge returns structured axes.
* If the **judge fails** (API error, parse error), an explicit error message is shown instead of silent neutral scores.

## Eval History

Table of recent runs showing: time, prompt snippet, models used, savings percentage, verdict, and score. Expand any row for the full responses, judge reasoning, and criteria breakdown.

* **Delete** removes a single run (with confirmation).
* **Bulk delete** — select multiple rows with the checkboxes and delete them all at once.
* At most **20** runs per organization are kept — the oldest run is trimmed automatically when a new one is inserted.


# Getting Started 
Source: https://docs.cloudidr.com/guides/flex-compute/getting-started

Set up AWS GPU & CPU instances at 30-40% discount in 15 minutes

Get access to AWS EC2 instances at 30-40% discount in four simple steps. Total setup time: 15 minutes.

## Prerequisites

Before you begin, make sure you have:

* **AWS Account** - Active AWS account with billing enabled
* **AWS IAM Access** - Admin permissions to create IAM roles
* **Email Address** - For your FlexCompute account
* **Use Case** - Know which instance types you need (optional)

**Don't have AWS admin access?** Contact your AWS administrator or [email us](mailto:hello@cloudidr.com) for alternative setup options.

***

## Step 1: Create FlexCompute Account

**Time required: 2 minutes**

1. Go to [cloudidr.com](https://www.cloudidr.com)
2. Click **"Start Free Trial"** or **"Get Started"**
3. Fill in your details:
   * Full name
   * Work email
   * Company name
   * Phone number (optional)
4. Click **"Create Account"**
5. Verify your email (check inbox for verification link)

**What happens next:** You'll receive a welcome email with your FlexCompute dashboard login credentials.

***

## Step 2: Connect Your AWS Account

**Time required: 5 minutes**

FlexCompute needs permission to provision instances in your AWS account. We use AWS IAM roles (industry standard, secure, read-only access).

### Option A: Automated Setup (Recommended)

1. Log in to your FlexCompute dashboard
2. Click **"Connect AWS Account"**
3. Click **"Launch CloudFormation Stack"**
   * This opens AWS Console in new tab
   * Pre-configured CloudFormation template loads
4. In AWS Console:
   * Check the box: "I acknowledge that AWS CloudFormation might create IAM resources"
   * Click **"Create Stack"**
5. Wait 2-3 minutes for stack creation
6. Return to FlexCompute dashboard
7. Click **"Verify Connection"**

**Done!** Your AWS account is now connected.

### Option B: Manual Setup

If you prefer manual IAM role creation:

1. Download our IAM policy template: [Download JSON](https://www.cloudidr.com/iam-policy.json)
2. In AWS Console:
   * Go to IAM → Roles → Create Role
   * Select "Another AWS Account"
   * Enter FlexCompute Account ID: `123456789012` (shown in your dashboard)
   * Attach the downloaded policy
   * Name role: `FlexComputeAccess`
3. Copy the Role ARN
4. Paste Role ARN in FlexCompute dashboard
5. Click **"Verify Connection"**

### What Permissions Does FlexCompute Get?

FlexCompute IAM role has **limited permissions**:

**Can do:**

* Launch EC2 instances in your account
* Terminate instances (only ones we provisioned)
* View instance status
* Attach to your existing VPC/security groups

**Cannot do:**

* Access your data or applications
* Modify existing resources
* Change IAM policies
* Access other AWS services (S3, RDS, etc.)

**Security:** Role uses AWS STS with time-limited credentials. You can revoke access anytime by deleting the IAM role.

***

## Step 3: Request Your First Instance

**Time required: 2 minutes**

Now that your AWS account is connected, request instances through FlexCompute.

### Via Dashboard (Web Interface)

1. In FlexCompute dashboard, click **"Request Instance"**
2. Configure your instance:
   * **Instance Type:** Select from dropdown (e.g., `c5.2xlarge`, `g5.xlarge`)
   * **Region:** Choose AWS region (e.g., `us-east-1`, `us-west-2`)
   * **Availability Zone:** Select AZ or leave as "Any"
   * **Quantity:** Number of instances (1-100)
   * **Duration:** How long you need them (hours, days, or ongoing)
3. Review discounted pricing (shows savings vs AWS On-Demand)
4. Click **"Launch Instances"**

**Processing time:** 2-5 minutes for instances to appear in your AWS Console.

### Via API (Programmatic)

For automated provisioning:

```bash theme={null}
curl -X POST https://api.cloudidr.com/v1/instances \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "instance_type": "c5.2xlarge",
    "region": "us-east-1",
    "quantity": 5,
    "duration": "8h"
  }'
```

**API Documentation:** [View full API reference →](/flexcompute/api-reference)

***

## Step 4: Access Your Instances

**Time required: 1 minute**

Once instances are provisioned, access them exactly like regular AWS EC2 instances.

### Via AWS Console

1. Log in to AWS Console
2. Navigate to **EC2 → Instances**
3. Filter by tag: `Provider: FlexCompute`
4. Your FlexCompute instances appear in the list
5. Select instance → **Connect** (SSH, RDP, Session Manager)

### Via AWS CLI

```bash theme={null}
# List FlexCompute instances
aws ec2 describe-instances \
  --filters "Name=tag:Provider,Values=FlexCompute"

# SSH to instance
ssh -i your-key.pem ec2-user@<instance-ip>
```

### Via Terraform/Infrastructure-as-Code

FlexCompute instances integrate with your existing IaC:

```hcl theme={null}
# Reference FlexCompute-provisioned instances
data "aws_instances" "flexcompute" {
  filter {
    name   = "tag:Provider"
    values = ["FlexCompute"]
  }
}
```

***

## Monitoring & Management

### View Instance Status

**In FlexCompute Dashboard:**

* See all active instances
* View current costs
* Check remaining hours (if time-limited)
* Monitor utilization

**In AWS Console:**

* Instances appear like regular EC2
* Use CloudWatch for metrics
* Configure alarms normally

### Terminate Instances

**When you're done:**

**Option 1:** Via FlexCompute dashboard

* Click **"Active Instances"**
* Select instance → **"Terminate"**

**Option 2:** Via AWS Console

* Navigate to EC2 → Instances
* Select FlexCompute instance → **"Terminate"**
* Termination allowed for FlexCompute-provisioned instances

**Billing stops immediately upon termination.**

***

## Common Setup Issues

### Issue: CloudFormation Stack Fails

**Solution:**

* Ensure you have IAM admin permissions
* Check AWS region matches FlexCompute dashboard
* Try manual IAM role setup instead

### Issue: Instances Not Appearing in AWS Console

**Solution:**

* Wait 5 minutes (provisioning takes 2-5 min)
* Check correct AWS region selected
* Verify IAM role connection in FlexCompute dashboard

### Issue: Can't Connect to Instance

**Solution:**

* Ensure your security group allows SSH/RDP access
* Verify key pair associated with instance
* Check instance is in running state

**Still stuck?** Email [support@cloudidr.com](mailto:support@cloudidr.com) with your account ID.

***

## What's Next?

### Explore Features

* **Auto-Scaling:** Set rules to scale instances automatically
* **Scheduled Provisioning:** Reserve capacity in advance
* **Team Management:** Add team members to your account
* **Cost Alerts:** Get notified when spending exceeds threshold

### Optimize Further

* **Right-Size Instances:** Analyze usage, switch to smaller/larger types
* **Reserved Capacity:** For predictable workloads, pre-book capacity
* **Spot-Like Pricing:** Access even deeper discounts for interruptible workloads

### Get Support

* **Email:** [hello@cloudidr.com](mailto:hello@cloudidr.com)
* **Schedule Demo:** [Book 15-minute call](https://meetings.hubspot.com/khursheed-hassan)
* **Documentation:** Browse [setup guides](/flexcompute/guides)

***

## Free Trial Details

**\$100 in free credits** to test FlexCompute:

* Valid for 14 days
* No credit card required
* Test any instance type
* Full support included
* Credits expire after 14 days

**Start trial:** [cloudidr.com](https://www.cloudidr.com)

***

## Ready to Save?

You're all set! You now have:

✅ FlexCompute account created\
✅ AWS account connected\
✅ First instances provisioned\
✅ Access via AWS Console/CLI

**Start requesting instances and save 30-40% on your AWS bill.**

**Questions?** [Contact our team →](mailto:hello@cloudidr.com)

**Need help?** [Schedule a walkthrough →](https://meetings.hubspot.com/khursheed-hassan)

```

---

## **File Structure:**
```

/flexcompute
├── overview\.mdx          (File 1: What it is + benefits)
├── use-cases.mdx         (File 2: How teams use it)
└── getting-started.mdx   (File 3: Setup guide)
title: "Getting Started"
------------------------


# Use cases
Source: https://docs.cloudidr.com/guides/flex-compute/use-cases

How teams use FlexCompute to save 30-40% on AWS infrastructure

FlexCompute works for any workload that runs on AWS EC2—from AI training to production web servers. Here's how different teams use FlexCompute to cut costs while maintaining performance and reliability.

## AI/ML Training & Inference

### LLM Training & Fine-Tuning

**Challenge:** Training large language models on GPU instances (p4, p5) costs thousands per hour. Reserved Instances require 1-3 year commitments, but training needs fluctuate project-by-project.

**Solution:** Use FlexCompute to access p5 instances at 30-40% discount with zero commitment. Spin up capacity when starting a training run, release when complete.

**Example:**

* Instance: 8x p5.48xlarge
* Training duration: 120 hours
* AWS On-Demand cost: \$76,800
* FlexCompute cost: \$51,840
* **Savings: \$24,960 per training run**

**Who uses this:**

* AI research labs
* LLM development teams
* Companies fine-tuning foundation models

***

### Computer Vision Model Development

**Challenge:** Training vision models requires GPU capacity (g5, g6 instances) but workload is inconsistent—intense during active development, idle between projects.

**Solution:** Pay for GPU instances only when training, not 24/7. FlexCompute provides g5/g6 instances at 30-40% discount without Reserved Instance lock-in.

**Example:**

* Instance: 4x g5.12xlarge
* Usage: 10 hours/day, 20 days/month
* AWS On-Demand cost: \$22,000/month
* FlexCompute cost: \$14,300/month
* **Savings: \$7,700/month**

**Who uses this:**

* Autonomous vehicle teams
* Medical imaging startups
* Security/surveillance companies

***

### Real-Time Inference Serving

**Challenge:** Serving LLM or vision model inferences at scale requires always-on GPU capacity, making Reserved Instances attractive—but capacity needs vary with user growth.

**Solution:** Start with FlexCompute instances, scale up as traffic grows, no upfront commitment. Maintain same performance as Reserved Instances at similar pricing but with full flexibility.

**Example:**

* Instance: 6x g6e.2xlarge (24/7)
* AWS On-Demand: \$12,096/month
* AWS Reserved (1yr): \$8,467/month (locked in)
* FlexCompute: \$7,862/month (no lock-in)
* **Savings vs On-Demand: \$4,234/month**
* **Flexibility vs Reserved: Cancel anytime**

**Who uses this:**

* AI API services
* Chatbot platforms
* Real-time recommendation engines

***

### Batch Inference Processing

**Challenge:** Processing large batches of inference requests (e.g., analyzing millions of images overnight) requires massive GPU capacity temporarily.

**Solution:** Spin up 20-50 GPU instances for the batch job, run for a few hours, release capacity. Pay only for hours used at 30-40% discount.

**Example:**

* Instance: 50x g5.xlarge
* Runtime: 6 hours
* AWS On-Demand cost: \$303
* FlexCompute cost: \$195
* **Savings: \$108 per batch job**
* Run 30 batches/month = \$3,240/month savings

**Who uses this:**

* Content moderation platforms
* Document processing services
* Data labeling companies

***

## Development & Testing

### CI/CD Pipeline Infrastructure

**Challenge:** Running comprehensive test suites requires substantial compute capacity, but tests run intermittently throughout the day, not 24/7.

**Solution:** Provision FlexCompute instances for CI/CD runners, scale to match build queue, release capacity during off-hours.

**Example:**

* Instance: 20x c5.4xlarge (running 8 hours/day)
* AWS On-Demand: \$1,835/month
* FlexCompute: \$1,193/month
* **Savings: \$642/month**

**Who uses this:**

* Engineering teams with frequent deployments
* Startups optimizing dev costs
* Companies with large test suites

***

### Staging & QA Environments

**Challenge:** Staging environments mirror production but don't need to run 24/7. Paying full price for environments used only during business hours wastes money.

**Solution:** Run staging on FlexCompute instances, schedule automatic shutdown outside business hours, pay only for uptime at discounted rates.

**Example:**

* Instance: 15x m6i.2xlarge (10 hours/day, weekdays only)
* AWS On-Demand: \$1,152/month
* FlexCompute: \$749/month
* **Savings: \$403/month**

**Who uses this:**

* Product teams with extensive QA needs
* Agencies managing multiple client environments
* SaaS companies with staging requirements

***

### Load Testing Infrastructure

**Challenge:** Load testing production systems requires spinning up hundreds of instances temporarily to simulate user traffic—expensive if using On-Demand rates.

**Solution:** Use FlexCompute to provision load testing infrastructure for hours or days, not months. Pay 30-40% less for the same capacity.

**Example:**

* Instance: 100x c5.2xlarge
* Duration: 8 hours (quarterly load test)
* AWS On-Demand cost: \$272 per test
* FlexCompute cost: \$177 per test
* \*\*Savings: $95 per test** ($380/year)

**Who uses this:**

* E-commerce platforms (pre-Black Friday testing)
* Gaming companies (launch testing)
* FinTech (stress testing)

***

## Batch Processing & Data Workloads

### Data Analytics Pipelines

**Challenge:** Running nightly ETL jobs on large datasets requires compute-optimized or memory-optimized instances, but jobs complete in 2-6 hours—not 24 hours.

**Solution:** Schedule FlexCompute instances to provision before job start, process data, release capacity when complete. No paying for 18+ idle hours daily.

**Example:**

* Instance: 10x c5.9xlarge (4 hours/day)
* AWS On-Demand: \$2,448/month
* FlexCompute: \$1,591/month
* **Savings: \$857/month**

**Who uses this:**

* Data engineering teams
* Business intelligence platforms
* Analytics-heavy SaaS companies

***

### Video Rendering & Transcoding

**Challenge:** Rendering video content requires high-compute instances but demand fluctuates—heavy during project work, idle between clients.

**Solution:** Provision rendering farm with FlexCompute instances on-demand, scale to match project queue, release when queue clears.

**Example:**

* Instance: 30x c6i.8xlarge
* Usage: Variable (15 hours/day average)
* AWS On-Demand: \$12,240/month
* FlexCompute: \$7,956/month
* **Savings: \$4,284/month**

**Who uses this:**

* Video production studios
* Streaming platforms
* Social media content creators

***

### Scientific Computing

**Challenge:** Research simulations require massive compute capacity for days or weeks, but projects are episodic—not continuous.

**Solution:** Access high-performance computing instances through FlexCompute for simulation runs without multi-year commitments.

**Example:**

* Instance: 20x c5.18xlarge
* Duration: 2 weeks per quarter
* AWS On-Demand: \$4,896 per simulation
* FlexCompute: \$3,182 per simulation
* \*\*Savings: $1,714 per simulation** ($6,856/year)

**Who uses this:**

* Research institutions
* Pharmaceutical companies (drug modeling)
* Engineering firms (finite element analysis)

***

## Production Workloads

### Web Application Servers

**Challenge:** Production web servers need high availability but capacity requirements vary with traffic patterns (daily/weekly fluctuations).

**Solution:** Run baseline capacity on FlexCompute instances, scale up during traffic peaks, maintain same reliability as Reserved Instances without commitment.

**Example:**

* Instance: 25x m6i.xlarge (24/7)
* AWS On-Demand: \$4,800/month
* AWS Reserved (1yr): \$3,360/month (locked in)
* FlexCompute: \$3,120/month (no lock-in)
* **Savings vs On-Demand: \$1,680/month**
* **Flexibility vs Reserved: Scale freely**

**Who uses this:**

* SaaS platforms
* E-commerce sites
* API services

***

### Microservices & API Backends

**Challenge:** Microservice architectures run dozens of small-to-medium instances. Reserved Instances make capacity planning difficult as service needs change.

**Solution:** Run all microservices on FlexCompute instances, adjust capacity per service independently, no commitment required.

**Example:**

* Instances: 50 mixed (t3, c5, m6i)
* AWS On-Demand: \$8,500/month
* FlexCompute: \$5,525/month
* **Savings: \$2,975/month**

**Who uses this:**

* Cloud-native applications
* Startups scaling rapidly
* Companies modernizing legacy apps

***

### Database Instances

**Challenge:** Running databases on EC2 (PostgreSQL, MySQL, MongoDB) requires memory-optimized instances running 24/7—expensive on On-Demand, inflexible with Reserved.

**Solution:** Use FlexCompute for database instances, maintain same performance and uptime, save 30-40% without multi-year commitment.

**Example:**

* Instance: 3x r7i.4xlarge (24/7, Multi-AZ)
* AWS On-Demand: \$7,272/month
* FlexCompute: \$4,727/month
* **Savings: \$2,545/month**

**Who uses this:**

* Companies running self-managed databases
* Teams avoiding managed database costs
* Applications with specific DB requirements

***

## Common Patterns Across Use Cases

### Pattern 1: Variable Workloads

**Best for:** Workloads that scale up/down daily, weekly, or seasonally\
**Savings:** Pay only for hours used, save 30-40% vs On-Demand\
**Examples:** Dev/test, CI/CD, batch processing

### Pattern 2: Project-Based Capacity

**Best for:** Capacity needed for specific projects, not continuously\
**Savings:** No idle costs between projects, no long-term commitments\
**Examples:** AI training, load testing, video rendering

### Pattern 3: Growing Production

**Best for:** Production workloads with unpredictable growth\
**Savings:** Reserved pricing without capacity lock-in\
**Examples:** SaaS apps, microservices, databases

***

**Ready to save on your workloads?** [Get started with FlexCompute →](/flexcompute/getting-started)

**Questions about your specific use case?** [Contact us →](mailto:hello@cloudidr.com)


# What is Flex Compute?
Source: https://docs.cloudidr.com/guides/flex-compute/what-is-flex-compute

AWS GPU & CPU instances at 30-40% discount, directly in your AWS account

FlexCompute gives you access to AWS EC2 instances (GPU and CPU) at 30-40% lower prices than standard AWS On-Demand rates—with no long-term contracts, no commitments, and no migration required. Instances appear directly in your existing AWS account, working seamlessly with your current infrastructure.

## The Problem We Solve

AWS Reserved Instances force you to commit for 1-3 years and pay 24/7, even when you're not using capacity. AWS On-Demand pricing is flexible but expensive, with no volume discounts no matter how much you spend.

**The dilemma every engineering team faces:**

* Pay full price with On-Demand (expensive, flexible)
* Lock into Reserved Instances (cheaper, but wasteful when idle)
* Use Spot Instances (cheapest, but unreliable and can terminate anytime)

**FlexCompute solves this:** Pay-as-you-go pricing with 30-40% savings and zero commitments.

## How FlexCompute Works

We purchase compute capacity in bulk directly from AWS and pass the savings to you through our platform. When you request instances through FlexCompute, they appear as standard EC2 instances in your own AWS account—fully integrated with your existing VPC, IAM policies, security groups, and billing.

### Your Infrastructure Stays in AWS

**Nothing changes about how you work:**

* Instances run in **your AWS account**
* You keep **your existing IAM roles** and permissions
* You use **your existing VPC** and security configurations
* You access instances via **AWS Console** or CLI
* You maintain **your current workflows**

The only difference: You're paying 30-40% less.

## Key Benefits

### 30-40% Cost Savings

Pay significantly less than AWS On-Demand rates across all major instance families:

* **GPU instances** (g5, g6, p4, p5) - Perfect for AI training and inference
* **Compute-optimized** (c5, c6, c7) - High-performance computing
* **Memory-optimized** (m6i, r7i, x2) - Large-scale data processing
* **General purpose** (t3, t4g) - Web servers, dev environments

### No Long-Term Commitments

* **Pay as you go** - Hourly billing, cancel anytime
* **No Reserved Instance lock-in** - No 1-year or 3-year commitments
* **Scale up or down freely** - Add capacity when needed, release when done
* **No penalties** - Stop using FlexCompute anytime without fees

### Runs in Your AWS Account

* **Zero migration** - No moving workloads to another cloud
* **Same AWS infrastructure** - Identical performance and reliability
* **Your security controls** - Keep existing IAM, VPC, compliance
* **AWS billing integration** - Charges appear on your AWS bill

### 15-Minute Setup

1. Sign up for FlexCompute
2. Connect your AWS account (one-time setup)
3. Request instances via our dashboard or API
4. Instances appear in your AWS Console

No DevOps work. No infrastructure changes. Just savings.

### High Availability

While we can't guarantee 100% availability (like all cloud providers), we maintain high availability through:

* Multiple availability zones
* Automatic failover options
* Real-time capacity monitoring
* Alternative instance type recommendations

**Unlike Spot Instances:** FlexCompute instances won't terminate unexpectedly. Once allocated, they're yours until you release them.

## How We Compare

| Feature              | AWS On-Demand | AWS Reserved (1yr) | AWS Spot      | FlexCompute |
| -------------------- | ------------- | ------------------ | ------------- | ----------- |
| **Discount**         | 0%            | \~30%              | \~70%         | **30-40%**  |
| **Commitment**       | None          | 1-3 years          | None          | **None**    |
| **Availability**     | High          | High               | Low           | **High**    |
| **Can terminate?**   | No            | No                 | Yes (anytime) | **No**      |
| **Runs in your AWS** | Yes           | Yes                | Yes           | **Yes**     |
| **Setup time**       | 5 min         | 5 min              | 5 min         | **15 min**  |
| **Billing**          | AWS           | AWS                | AWS           | **AWS**     |

**FlexCompute = Reserved Instance pricing without the commitment**

## Security & Compliance

### Data Never Leaves AWS

* All compute happens in **your AWS account**
* Your data stays in **your VPC**
* We never access your instances or data
* Full AWS compliance (HIPAA, SOC 2, GDPR)

### IAM Integration

* Use your existing **IAM roles and policies**
* No new security permissions required
* Maintain your access controls
* Audit trail in AWS CloudTrail

### No Code Changes

* Applications run unchanged
* Same AWS APIs and SDKs
* Same AWS services integration
* Same monitoring and logging

## Real-World Savings

### Startup Example

**Before FlexCompute:**

* 10x c5.2xlarge instances (24/7)
* AWS On-Demand: \$3,400/month
* Reserved Instances would require 1-year commitment

**With FlexCompute:**

* Same 10x c5.2xlarge instances
* FlexCompute: \$2,210/month
* **Savings: 14,280/year)**
* No commitment required

### AI Company Example

**Before FlexCompute:**

* Training models on 4x g5.12xlarge (10 hours/day)
* AWS On-Demand: \$22,000/month
* Spot Instances too unreliable for training jobs

**With FlexCompute:**

* Same 4x g5.12xlarge instances
* FlexCompute: \$14,300/month
* **Savings: 92,400/year)**
* Stable instances, no interruptions

### Enterprise Example

**Before FlexCompute:**

* Mixed workloads (50 instances, various types)
* AWS On-Demand: \$45,000/month
* Reserved Instances lock in capacity they don't always need

**With FlexCompute:**

* Flexible capacity, scale as needed
* FlexCompute: \$29,250/month
* **Savings: 189,000/year)**
* No commitment, full flexibility

***

**Ready to start saving?** [Get started with FlexCompute →](/flexcompute/getting-started)

**See how others use it:** [FlexCompute use cases →](/flexcompute/use-cases)


# Introduction
Source: https://docs.cloudidr.com/guides/get-started/introduction

Quick overview of platform

This guide will help you get started quickly and make the most of the Cloudidr platform. Here, you'll find step by-step instructions, best practices, and troubleshooting tips to ensure a smooth experience.

Cloudidr platform consists of several unique products:

* **LLM Ops** - to monitor, track and optimize multi-provider LLM API calls
* **Flex Compute** - Discounted AWS GPUs and CPUs without signing any long term contracts. Directly orhcestrated into your AWS native account
* **Disaster Recovery Compute** - 99.999% availability SLA for failover without paying the idle usage fees
* **GPU Service** - Same AWS GPUs at discounted rate with direct shell access to run train experiment and deploy into production. No commitments either
* **Hosted AI Models** - We manage open source LLM and other models for you. Fully secured for our enterprise environment without the paying the mark up fees with LLM providers.

Whether you're new or experienced, this documentation provides everything you need to navigate, manage resources, and optimize your usage efficiently.

Let’s get started! 🚀


# Agent Traces
Source: https://docs.cloudidr.com/guides/llm-ops/agent-traces



Cloudidr **Agent Traces** gives you cost visibility inside a single agent run — not just across all requests in aggregate, but step by step, in the exact sequence they executed.

When a user sends one question to your AI agent, the agent typically fires multiple LLM calls in sequence: a planner, a retriever, a summarizer, a formatter. Today those appear as four separate rows in the Activity log with no indication they belong together. Agent Traces groups them automatically and shows you which step cost the most, which one Cloudidr routed to a cheaper model, and where the time went.

> **Availability:** Agent Traces is available on **Growth, Scale, and Enterprise** plans. Not available on Starter.

> **This is not a debugging tool.** Cloudidr does not store prompts or completions. Traces answer one question: *"Which step in my agent costs the most and what did Cloudidr do about it?"* For prompt debugging and chain internals, use LangSmith or Braintrust alongside Cloudidr.

***

## How to instrument

No SDK install required. Add two HTTP headers to every LLM call in your agent run.

### Headers

| Header        | Purpose                                                                              | Example                           |
| ------------- | ------------------------------------------------------------------------------------ | --------------------------------- |
| `X-Trace-Id`  | A unique ID for the entire agent run — all calls sharing this ID appear as one trace | `"conv_abc123"` or a UUID         |
| `X-Span-Name` | A label for this individual step within the run                                      | `"query_planner"`, `"summarizer"` |

Both headers are **optional**. Requests without them continue to be logged in the Activity tab as before. They have no effect on routing or billing.

### Python (OpenAI SDK)

```python theme={null}
import uuid

trace_id = str(uuid.uuid4())  # generate once per agent run

# Step 1
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    extra_headers={
        "X-Cloudidr-Key": "trk_xxx",
        "X-Trace-Id":     trace_id,
        "X-Span-Name":    "query_planner"
    }
)

# Step 2 — same trace_id, different span name
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    extra_headers={
        "X-Cloudidr-Key": "trk_xxx",
        "X-Trace-Id":     trace_id,
        "X-Span-Name":    "summarizer"
    }
)
```

### Python (Anthropic SDK)

```python theme={null}
import uuid

trace_id = str(uuid.uuid4())

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[...],
    extra_headers={
        "X-Cloudidr-Key": "trk_xxx",
        "X-Trace-Id":     trace_id,
        "X-Span-Name":    "intent_classifier"
    }
)
```

### Using an existing session or conversation ID

You do not need to generate a new UUID. If your application already tracks a `session_id`, `conversation_id`, or `request_id`, pass it directly as `X-Trace-Id`. Any string works — Cloudidr groups all calls sharing the same value under one trace.

```python theme={null}
trace_id = session.id  # your existing session identifier
```

***

## Traces list page

Navigate to **Sidebar → Traces** (under Analytics).

<Frame>
  <img alt="Image" />
</Frame>

### Summary cards

Four cards at the top respond to your active filters:

| Card                 | Value                                                |
| -------------------- | ---------------------------------------------------- |
| **Total Traces**     | Count of distinct agent runs in the selected period  |
| **Total Cost**       | Combined LLM cost across all traces                  |
| **Avg Cost / Trace** | Mean cost per agent run                              |
| **Total Savings**    | Routing savings Cloudidr generated across all traces |

### Filters

| Filter         | Options                                                                |
| -------------- | ---------------------------------------------------------------------- |
| **Date range** | Today / Last 7 Days / Last 30 Days / Last 90 Days / 1 Year / Custom    |
| **Provider**   | All Providers / OpenAI / Anthropic / Google                            |
| **Search**     | Free-text search on Trace ID, Span Name, Agent, Project, or Department |

### Trace list table

Each row represents one agent run (one unique `X-Trace-Id`):

| Column         | Description                                                                                                                                                                                                       |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Time**       | When the first span in this trace arrived                                                                                                                                                                         |
| **Trace Name** | The first `X-Span-Name` value seen for this trace, or the Trace ID if no span name was set                                                                                                                        |
| **Agent**      | The `X-Agent` tag on the requests                                                                                                                                                                                 |
| **Project**    | The `X-Project` tag                                                                                                                                                                                               |
| **Dept**       | The `X-Department` tag                                                                                                                                                                                            |
| **Spans**      | Total number of LLM calls in this trace                                                                                                                                                                           |
| **Provider**   | Provider(s) used (shown as the dominant provider)                                                                                                                                                                 |
| **Cost**       | Total LLM cost for this trace                                                                                                                                                                                     |
| **Savings**    | Total routing savings Cloudidr generated across all spans in this trace                                                                                                                                           |
| **Latency**    | Wall-clock time from when the proxy received the first span to when it received the complete response for the last span — this is proxy-measured time and excludes the final network hop back to your application |

Click any row to open the Trace Detail view.

**Load more** — the list loads 20 traces at a time. Click **Load more** at the bottom to page through additional results.

***

## Trace detail — waterfall view

Clicking a row opens the trace detail view, showing a waterfall chart of all spans in the order they were received by the proxy.

### Waterfall columns

| Column       | Description                                                                                                                             |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------- |
| **Step**     | The `X-Span-Name` for this span; most-expensive step is marked with ⚡                                                                   |
| **Timeline** | Horizontal bar proportional to this span's latency relative to the total trace duration                                                 |
| **Latency**  | Time from when the proxy received this request to when it received the complete provider response                                       |
| **Model**    | The model requested and, if Cloudidr routed to a different model, the routed model shown with a `→` arrow (e.g. `gpt-4o → gpt-4o-mini`) |
| **Cost**     | Actual cost for this span at the routed model's price                                                                                   |
| **Savings**  | Cost saved vs. the originally requested model's price (shown when Cloudidr routed)                                                      |

### Tool call rows

When an LLM call invokes a tool (web search, database lookup, function call), Cloudidr detects the tool invocation from the provider response and inserts a greyed-out row for each tool called, placed immediately after the LLM span that triggered it.

<Frame>
  <img alt="Image" />
</Frame>

Tool call rows show:

* The tool name (e.g. `web_search`, `get_weather`, `query_database`)
* A wrench icon to distinguish them from LLM spans
* Dashes for Cost and Latency — tool calls are detected from the provider response metadata; Cloudidr does not intercept the tool execution itself

> **Note:** Tool call rows are proxy-detected from the LLM response. They show which tools the model invoked, not how long the tool took to execute. For full tool execution timing and input/output, use a dedicated tracing tool like LangSmith or Arize Phoenix alongside Cloudidr.

***

## Span detail panel

Click any span row in the waterfall to open the span detail panel on the right side.

<Frame>
  <img alt="Image" />
</Frame>

### Panel contents

**For LLM spans:**

| Field                            | Description                                                       |
| -------------------------------- | ----------------------------------------------------------------- |
| **Span name**                    | The `X-Span-Name` value                                           |
| **Provider**                     | LLM provider (OpenAI, Anthropic, Google)                          |
| **Model requested**              | The model your application asked for                              |
| **Model used**                   | The model Cloudidr actually routed to (if different)              |
| **Input tokens**                 | Token count sent to the model                                     |
| **Output tokens**                | Token count returned by the model                                 |
| **Cost**                         | Actual cost at the routed model's price                           |
| **Savings**                      | Difference between requested model price and routed model price   |
| **Latency**                      | Proxy-measured response time for this span                        |
| **Timestamp**                    | When this span was received                                       |
| **Department / Project / Agent** | Org tags from the request headers                                 |
| **Tools called**                 | If this span invoked tools, a list of tool names appears as pills |

**For tool call rows:**

Tool rows show the tool name and its parent span name only. Cost, tokens, and latency are not available for tool calls (tool execution happens outside the proxy).

***

## Access control

| Role              | What they see                                                        |
| ----------------- | -------------------------------------------------------------------- |
| **Owner / Admin** | All traces from all users in the organisation                        |
| **Member**        | Only traces from their own requests (filtered by their user account) |

***

## Latency definition

All latency values in Agent Traces are **proxy-measured**: the wall-clock time from when Cloudidr's proxy received your request to when it received the complete response from the LLM provider. This excludes:

* Time your application spent preparing the request before it reached the proxy
* The final network hop from the proxy back to your application

For most agents this difference is negligible. If your agents involve long pre-processing steps or large response streaming, the actual end-to-end latency your users experience will be slightly higher than what Traces reports.

***

## Common patterns

### Identify which step costs the most

The most expensive span in each trace is marked with ⚡ in the Step column. Sort the trace list by **Cost** descending to find your most expensive agent runs, then click in to see which step is responsible.

### See where Cloudidr saved money

The **Savings** column in both the trace list and waterfall shows routing savings per run and per step. A non-zero savings value means Cloudidr detected that the original model was more expensive than needed for that prompt's complexity and routed it to a cheaper equivalent.

### Track a specific agent workflow

Use the **Search** filter to enter a span name (e.g. `summarizer`) or agent name to narrow the list to a specific workflow. Use the **Date range** filter alongside it to scope to a specific deployment window.

### Use your existing session ID as trace ID

If your application already generates a session, conversation, or request ID, pass it as `X-Trace-Id`. You get traces with zero additional ID management — just reuse what you already have.C


# API Keys
Source: https://docs.cloudidr.com/guides/llm-ops/api-keys



Every user gets a unique Cloudidr API Key. Cloudidr uses these unique API keys to track LLM API calls and associate to each user and organization.

* Go to **Settings** on left side panel and click **API Keys** page.

<Frame>
  <img alt="Screenshot 2026 04 10 At 21 54 14" />
</Frame>

* A user can create and manage their own API keys under the **My Keys** tab
* Owner can also see and revoke API Keys for their team members under the **Company** tab.
* Cloudidr API Key starts with prefix **trk\_**

Cloudidr API Key is required in the API calls with **X-Cloudidr-Key**

```text theme={null}
curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Authorization: Bearer your-openai-key" \
  -H "Content-Type: application/json" \
  -H "X-Cloudidr-Key: trk_9H4V2Bxxxxxxxxxxxxxx" \
  -H "X-Department: Engineering" \
  -H "X-Project: PoC-New" \
  -H "X-Agent: chatbot" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```


# Budget Protection
Source: https://docs.cloudidr.com/guides/llm-ops/budget-guard

Configure org-wide spend alerts, per-agent monthly budgets, enforcement, and email notifications. 

## Overview

| Feature                   | Where in the app                         | What it does                                                                                                                                        |
| ------------------------- | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Budget Alert Settings** | **Settings** → **Budget Alert Settings** | **Notifications**: email and in-app alerts when spend crosses thresholds—**org-wide** (Overall Budget tab) and/or **per-agent** (Agent Alerts tab). |
| **Budget Guard**          | **Actions** → **Budget Guard**           | **Limits**: set each agent’s **monthly budget** and **Enforcement** (optional hard stop on API traffic when spend exceeds that budget).             |

***

## 1. Set overall spend alerts (organization-wide)

Configure **Daily**, **Weekly**, **Monthly**, and/or **Total** org-wide spend thresholds in dollars. For each period you enable:

* Set a **threshold amount (\$)**.
* Toggle **email notifications** if you want email when that threshold is crossed.
* Click **Save** for that block.

<Frame>
  <img alt="Image" />
</Frame>

***

## 2. Set each agent’s monthly budget (and Enforcement on or off)

For each agent that appears (from tracked API usage with **X-Agent** and related headers):

1. Enter a **Monthly Budget** (\$) for that agent.
2. Choose **Enforcement Enabled**:
   * **On** — When **Current Month Spend** (UTC month) reaches the budget, qualifying requests for that agent are **blocked** (e.g. HTTP 429 from the proxy).
   * **Off** — Spend is **not** blocked; you can rely on **alerts** only (see step 1 for org-wide; step 3 for per-agent emails).
3. Click **Save** for that row.

**Summary cards** at the top show total monthly budget (for agents with enforcement), spend to date, and how many agents are under Budget Guard.

<Frame>
  <img alt="Image" />
</Frame>

> **Who can edit:** Both **organization owners and team member** can change budgets and enforcement; other roles may see read-only messaging.

> **After you save:** If you entered a **positive monthly budget** and/or turned **Enforcement** on, **per-agent email alerts** are turned **on** by default for that agent (80% / 90% / 100% of the monthly budget)—unless you had already turned alerts off for that agent on the Agent Alerts tab (see step 3).

***

## 3. Optionally turn off per-agent alerts (Budget Alert Settings → Agent Alerts)

You only need this step if you want **fewer emails** for an agent that already has a monthly budget set in **Budget Guard**.

* Find the agent in the table.
* Turn **Alerts Enabled** **off** for that agent if you do not want 80% / 90% / 100% emails.
* Use **Save** on that row if shown.
  <Frame>
    <img alt="Screenshot 2026 04 11 At 00 10 58" />
  </Frame>

> If an agent has **no** monthly budget yet, the UI may show **Set budget in Budget Guard first** and you cannot enable alerts until step 2 is done.

***

## Troubleshooting

| Issue                                   | What to check                                                                                                              |
| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| No agent in Budget Guard                | Agents appear after usage is tracked with **X-Agent** (and department/project) on API requests.                            |
| Cannot enable or configure Agent Alerts | Set a **monthly budget** in **Budget Guard** (step 2) first.                                                               |
| Enforcement does not block traffic      | **Enforcement** must be **ON**; **Monthly Budget** must be **greater than zero**; evaluation uses **UTC** month.           |
| Spend vs. calendar feels wrong          | Budgets and “current month” use **UTC**, not your local timezone.                                                          |
| Too many emails                         | Use step 3: **Settings** → **Budget Alert Settings** → **Agent Alerts** → turn **Alerts Enabled** off for specific agents. |

***


# Plans & Features
Source: https://docs.cloudidr.com/guides/llm-ops/features

Complete feature comparison across all LLM Ops tiers

## Overview

| Feature                                     | Starter (Free)   | Growth (\$99)     | Scale (\$299)   | Enterprise (Custom) |
| ------------------------------------------- | ---------------- | ----------------- | --------------- | ------------------- |
| **Core Limits**                             |                  |                   |                 |                     |
| Tracked AI Spend                            | Up to \$5K/mo    | Up to \$30K/mo    | Up to \$100K/mo | Unlimited           |
| Team Members                                | 3 (1 owner + 2)  | 10 (1 owner + 9)  | Unlimited       | Unlimited           |
| **Monitoring & Analytics**                  |                  |                   |                 |                     |
| LLM Usage                                   | ✅                | ✅                 | ✅               | ✅                   |
| Agent Explorer (unlimited agents)           | ✅                | ✅                 | ✅               | ✅                   |
| Team Analytics                              | ✅                | ✅                 | ✅               | ✅                   |
| Savings Insights                            | ✅                | ✅                 | ✅               | ✅                   |
| Cloud Usage (AWS, GCP)                      | ❌                | ❌                 | ✅               | ✅                   |
| *Forecasting (coming soon)*                 | ❌                | ❌                 | ✅               | ✅                   |
| **Cost Control**                            |                  |                   |                 |                     |
| Budget Alerts                               | ✅                | ✅                 | ✅               | ✅                   |
| Budget Control (Hard Caps)                  | ✅ Up to 5 agents | ✅ Up to 30 agents | ✅ Unlimited     | ✅ Unlimited         |
| **LLM Cost Optimizer - Provider Strategy**  |                  |                   |                 |                     |
| Intra-provider (route within same provider) | ✅                | ✅                 | ✅               | ✅                   |
| Flexible (route across providers)           | ❌                | ✅                 | ✅               | ✅                   |
| Fixed (select specific set of providers)    | ❌                | ✅                 | ✅               | ✅                   |
| **LLM Cost Optimizer - Domain Plugins**     |                  |                   |                 |                     |
| Legal, Healthcare, Banking,Engineering      | ✅                | ✅                 | ✅               | ✅                   |
| **LLM Cost Optimizer - Routing Strategy**   |                  |                   |                 |                     |
| Smart (pattern matching)                    | ✅                | ✅                 | ✅               | ✅                   |
| *Adaptive (coming soon)*                    | ❌                | ❌                 | ✅               | ✅                   |
| Custom ML logic                             | ❌                | ❌                 | ❌               | ✅                   |
| Safety Controls                             | ✅ Preset         | ✅ Configurable    | ✅ Configurable  | ✅ Configurable      |
| **Data & Retention**                        |                  |                   |                 |                     |
| Historical Data Storage                     | 90 days          | 1 year            | 3 years         | Unlimited           |
| *Data Export (coming soon)*                 | ✅ CSV (manual)   | ✅ CSV (manual)    | ✅ CSV + API     | ✅ Full + Custom     |
| **Integrations & Access**                   |                  |                   |                 |                     |
| Slack Integration                           | ❌                | ✅                 | ✅               | ✅                   |
| *SSO (SAML) (coming soon)*                  | ❌                | ❌                 | ✅               | ✅                   |
| Custom Integrations                         | ❌                | ❌                 | ❌               | ✅                   |
| **Support**                                 |                  |                   |                 |                     |
| Support Level                               | Community        | Email (48hr)      | Priority (24hr) | Dedicated CSM       |
| SLA Uptime                                  | ❌                | ❌                 | 99.9%           | 99.99%              |

## Feature Details

### LLM Cost Optimizer - Provider Strategy

**Intra-provider:** Routes within the same provider as requested

* Example: `gpt-4` → `gpt-4o-mini` (OpenAI only)
* Typical savings: 60-85%

**Flexible (any):** Routes to cheapest model across ALL providers

* Example: `gpt-4` → `qwen3.5` (any provider)
* Typical savings: 70-90%

**Fixed (select):** Only route to your selected provider(s)

* Use case: Contracts, data sovereignty, compliance

### LLM Cost Optimizer - Routing Strategy

**Smart (pattern matching):** Analyzes request complexity using multi-factor scoring

* Routes queries to cheaper models
* Preserves quality for complex tasks

**Adaptive (AI learning):** Machine learning continuously improves routing decisions

* Learns from your usage patterns
* Additional 10-20% savings vs Smart routing

**Custom ML logic:** Custom models trained on your data

* Enterprise-only feature
* Tailored to your specific use cases

### Support Levels

**Community:** Self-service documentation and community forums

**Email (48hr):** Email support with 48-hour response time

**Priority (24hr):** Priority email and chat support with 24-hour response time

**Dedicated CSM:** Named Customer Success Manager with direct access

### SLA Uptime

**99.9% SLA:** \~43 minutes downtime/month maximum

**99.99% SLA:** \~4 minutes downtime/month maximum

**99.999% SLA (add-on):** \~26 seconds downtime/month maximum


# Finance Reports
Source: https://docs.cloudidr.com/guides/llm-ops/finance-reports



Cloudidr's **Finance Reports** feature gives owners and CFOs a dedicated view of LLM spend, usage, and budget health — and delivers a curated summary to executives every Monday morning via email.

> **Access:** Finance Reports and Reporting Settings are available to **account owners** only. Team members with other roles do not see the Finance Reports link in the sidebar or the "View Finance Reports" button on the main dashboard.

***

Navigate to **Sidebar → Finance Reports** or click **View Finance Reports** from the main dashboard.

### Spend Summary

Four KPI cards that show spend against your configured budget thresholds:

| Card                 | Period                       | Comparison                                                 |
| -------------------- | ---------------------------- | ---------------------------------------------------------- |
| **Year to Date**     | Jan 1 → today                | Traffic light vs yearly budget                             |
| **Month to Date**    | 1st of month → today         | % vs same day last month + traffic light vs monthly budget |
| **This Week**        | Sat 00:00 → now              | % vs same point last week + traffic light vs weekly budget |
| **Projected Annual** | Annualised from YTD run-rate | ± vs yearly budget (if set)                                |

Traffic lights use the budgets you configure in **Settings → Alert Settings**:

* **Green** — below 80 % of budget
* **Yellow** — 80–99 % of budget
* **Red** — at or over budget

> All period boundaries are calculated in your org's reporting timezone. Configure this in **Settings → Profile → Reporting Timezone**.

<Frame>
  <img alt="Screenshot 2026 05 03 At 20 34 02" />
</Frame>

### Usage Summary

Four KPI cards that show request and token volume alongside spend, so you can distinguish a cost increase driven by *more usage* vs *more expensive requests*:

| Card                      | Value                      | Comparison                   |
| ------------------------- | -------------------------- | ---------------------------- |
| **This Week — Requests**  | Requests this week so far  | % vs last week's full total  |
| **This Week — Tokens**    | Tokens this week so far    | % vs last week's full total  |
| **This Month — Requests** | Requests this month so far | % vs last month's full total |
| **This Month — Tokens**   | Tokens this month so far   | % vs last month's full total |

Tokens are formatted as `89.2M`, `1.4B` etc. for readability.

### Budget Plan

A read-only view of your budget thresholds (Yearly, Monthly, Weekly) pulled directly from Alert Settings. Click **Edit Budget Plan** to update them in Alert Settings.

> Budget thresholds power the traffic lights on the Spend Summary cards. If no budget is set for a period, the traffic light badge is hidden.

### Smart Routing Savings

Shows the savings generated by Cloudidr's model routing for the current and previous month, plus all-time. See [LLM Cost Optimizer](DOC_llm_cost_optimizer.md) for details.

### 12-Month Spend Chart

A month-by-month bar chart of LLM spend for the past 12 months. The current (partial) month bar is shown in a lighter shade. Toggle between **Spend**, **Requests**, and **Tokens** views using the buttons above the chart.

### Spend Tables

Five ranked tables filtered by the period selector at the top (**Current Week**, **Last Week**, **Current Month**, **Last Month**, **Last 90 Days**, **Year to Date**). Default is **Current Week**.

| Table                  | Groups by                              | Rows |
| ---------------------- | -------------------------------------- | ---- |
| **Top 10 Departments** | `department` tag on API requests       | 10   |
| **Top 10 Projects**    | `team` tag on API requests             | 10   |
| **Top 10 Agents**      | `agent` tag on API requests            | 10   |
| **Top 5 Models**       | Model name                             | 5    |
| **Top 5 Providers**    | Provider (Anthropic, OpenAI, Google …) | 5    |

Toggle columns between **Spend**, **Requests**, and **Tokens** using the metric selector above the tables. A date-range sub-label below the period buttons confirms the exact dates being shown.

> **Disclaimer:** Spend figures are estimates based on published list prices. Provider discounts, marketplace credits, and rounding may cause differences from your actual invoice.

***

## Email Weekly Finance Summaries

Navigate to **Settings → Reporting** (visible to account owners only).

### Weekly Executive Emails

Enable a weekly HTML email sent every **Monday at 08:00 UTC** to a configurable list of recipients.

**To configure:**

1. Toggle **Enable Weekly Executive Emails** to on.
2. Add recipient email addresses (one per line or comma-separated).
3. Click **Save**.

Use **Send Test Email** to trigger an immediate test send to all configured recipients without waiting for Monday.

<Frame>
  <img alt="Screenshot 2026 05 03 At 20 33 42" />
</Frame>

### What the email contains

Each email covers the **previous full week (Saturday 00:00 → Friday 23:59)** in your org's reporting timezone and includes:

| Section                           | Details                                                                                                               |
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| **This Week Spend**               | Total spend with week-over-week % change and request count                                                            |
| **Month to Date vs Budget**       | MTD spend against your monthly budget with traffic light                                                              |
| **Year to Date vs Yearly Budget** | YTD spend against your yearly budget with traffic light                                                               |
| **Projected Annual Spend**        | Annualised YTD run-rate, shown ± vs yearly budget                                                                     |
| **Usage Summary**                 | Requests and tokens for the week with WoW % change vs last week's full total                                          |
| **Week-over-Week Movements**      | Bulleted list of departments and projects with ↑/↓ changes ≥ 10% vs the prior week, sorted by largest absolute change |
| **Top 3 Departments**             | Highest-spend departments for the week with spend and request count                                                   |
| **Top 3 Projects**                | Highest-spend projects (team tag) for the week with spend and request count                                           |

The email footer includes a link to the full Finance Reports page and a link to manage recipients in Reporting Settings.

### Budget thresholds in the email

The email pulls budget thresholds from your existing **Alert Settings** — there is no separate budget to configure in Reporting Settings. To change thresholds, go to **Settings → Alert Settings → Overall Budget**.

### Timezone

All period boundaries in both the Finance Reports page and the weekly email respect the org-level **Reporting Timezone** set in **Settings → Profile**. The timezone used is always stated in the email header and footer.


# Quick Start
Source: https://docs.cloudidr.com/guides/llm-ops/getting-started



1. [**Sign up**](http://llm-ops.cloudidr.com/signup) with your **company name**.
   * This is a unique company name that cannot be changed later.
   * If you are creating the account first time for the company, you will be the **Owner (super user).**
   * Owner can [invite ](/guides/llm-ops/team-members)team members after logging into the application.
     <Frame>
       <img alt="Screenshot 2026 04 10 At 21 33 32" />
     </Frame>
2. After you have logged in, you will see an empty dashboard (as you have not executed any API calls with cloudidr API key). You can locate your Cloudidr [API Key](/guides/llm-ops/api-keys) from left side panel For new users, API Key is also shown at the top of the page under **Get Started in 3 steps.**

<Frame>
  <img alt="Screenshot 2026 04 10 At 22 08 19" />
</Frame>

3. Use **Get Started in 3 steps** to execute prompts. You have 2 ways to run your requests. As soon as you send the request it will be logged in the dashboard.
   1. The easiest way to send a request is to use [Playground](/try-model).
   2. Alternatively, you can use your linux terminal or code to send the request.
      * As shown in previous image, first **copy** the Cloudidr API Key
      * Second, click **View steps** to open a **Setup Guide** modal as shown below on how to integrate your Cloudidr API key with your application (alternatively you can use curl as well as shown in next bullet below). You will need to modify **base\_url** and insert **X-Cloudidr-Key**.
        > NOTE: you will need to enter your Provider API Key where it shows (YOUR\_ANTHROPIC\_KEY). Cloudidr never touches nor store your provider key.
        >
        > NOTE: see also [Code Integration ](/guides/get-started/introduction)
        <Frame>
          <img alt="Image" />
        </Frame>
      * Curl is a quick way to execute the API call with Cloudidr API Key from your terminal
        <Frame>
          <img alt="Image" />
        </Frame>
4. After you execute the LLM call, your **Usage Overview** in dashboard will updated and the Get Started ribbon will disappear.

<Frame>
  <img alt="Image" />
</Frame>


# Anthropic Integration
Source: https://docs.cloudidr.com/guides/llm-ops/integrations/anthropic

Learn how to integrate LLM Ops with Anthropic's Claude API for cost tracking and monitoring

## Overview

Track costs and monitor usage for Anthropic's Claude API by routing your requests through LLM Ops. This guide shows you how to integrate using Python, JavaScript, or cURL.

<Note>
  **Security Guarantee**: LLM Ops does not store your API keys, request prompts, or response content in the **analytics database**—only metadata needed for cost analytics. The proxy must forward request bodies to Anthropic to complete the call; optional operational logging may exist in your deployment environment.
</Note>

## Quick Start

Point the Anthropic SDK at the LLM Ops **API host** (do **not** append `/v1`—the SDK adds `/v1/messages` itself). For raw HTTP (cURL), use the full path `https://api.llm-ops.cloudidr.com/v1/messages`.

* **Original API host:** `https://api.anthropic.com`
* **LLM Ops API host (SDK `base_url` / `baseURL`):** `https://api.llm-ops.cloudidr.com`
* **cURL URL:** `https://api.llm-ops.cloudidr.com/v1/messages`

## API Keys

You'll need two credentials:

1. **Anthropic API Key** - Your Claude API key from [console.anthropic.com](https://console.anthropic.com)
2. **Cloudidr Key** - Your tracking token from the [LLM Ops dashboard](https://llm-ops.cloudidr.com/dashboard) (tokens are typically prefixed with `trk_`)

The marketing site [llmfinops.ai](https://llmfinops.ai) points at the same product; the dashboard URL above is the canonical app host.

Set them as environment variables:

```bash theme={null}
export ANTHROPIC_API_KEY="sk-ant-..."
export CLOUDIDR_KEY="trk_..."
```

***

## Integration Examples

<Tabs>
  <Tab title="Python">
    ### Install SDK

    ```bash theme={null}
    pip install anthropic
    ```

    ### Basic Example

    ```python theme={null}
    from anthropic import Anthropic

    # Initialize client with LLM Ops proxy (no /v1 — SDK appends /v1/messages)
    client = Anthropic(
        api_key="sk-ant-...",  # Your Anthropic API key
        base_url="https://api.llm-ops.cloudidr.com",
        default_headers={
            "X-Cloudidr-Key": "trk_..."  # Required for cost tracking
        }
    )

    # Make API call - costs are automatically tracked
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[
            {"role": "user", "content": "What is the capital of France?"}
        ]
    )

    print(message.content[0].text)
    ```

    ### With Metadata (Department/Project/Agent Tracking)

    ```python theme={null}
    from anthropic import Anthropic

    client = Anthropic(
        api_key="sk-ant-...",
        base_url="https://api.llm-ops.cloudidr.com"
    )

    # Track costs by department, team, and agent
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[
            {"role": "user", "content": "Help me debug this code"}
        ],
        extra_headers={
            "X-Cloudidr-Key": "trk_...",
            "X-Department": "engineering",
            "X-Project": "backend",
            "X-Agent": "code-assistant"
        }
    )

    print(message.content[0].text)
    ```

    ### Streaming Example

    ```python theme={null}
    from anthropic import Anthropic

    client = Anthropic(
        api_key="sk-ant-...",
        base_url="https://api.llm-ops.cloudidr.com",
        default_headers={
            "X-Cloudidr-Key": "trk_...",
            "X-Agent": "streaming-bot"
        }
    )

    # Streaming is fully supported
    with client.messages.stream(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[
            {"role": "user", "content": "Write a short poem about AI"}
        ]
    ) as stream:
        for text in stream.text_stream:
            print(text, end="", flush=True)
    ```
  </Tab>

  <Tab title="JavaScript">
    ### Install SDK

    ```bash theme={null}
    npm install @anthropic-ai/sdk
    ```

    ### Basic Example

    ```javascript theme={null}
    import Anthropic from '@anthropic-ai/sdk';

    // Initialize client with LLM Ops proxy (no /v1 — SDK appends /v1/messages)
    const client = new Anthropic({
      apiKey: 'sk-ant-...',  // Your Anthropic API key
      baseURL: 'https://api.llm-ops.cloudidr.com',
      defaultHeaders: {
        'X-Cloudidr-Key': 'trk_...'  // Required for cost tracking
      }
    });

    // Make API call - costs are automatically tracked
    const message = await client.messages.create({
      model: 'claude-sonnet-4-20250514',
      max_tokens: 1024,
      messages: [
        { role: 'user', content: 'What is the capital of France?' }
      ]
    });

    console.log(message.content[0].text);
    ```

    ### With Metadata (Department/Project/Agent Tracking)

    ```javascript theme={null}
    import Anthropic from '@anthropic-ai/sdk';

    const client = new Anthropic({
      apiKey: 'sk-ant-...',
      baseURL: 'https://api.llm-ops.cloudidr.com'
    });

    // Track costs by department, team, and agent
    const message = await client.messages.create({
      model: 'claude-sonnet-4-20250514',
      max_tokens: 1024,
      messages: [
        { role: 'user', content: 'Help me debug this code' }
      ]
    }, {
      headers: {
        'X-Cloudidr-Key': 'trk_...',
        'X-Department': 'engineering',
        'X-Project': 'backend',
        'X-Agent': 'code-assistant'
      }
    });

    console.log(message.content[0].text);
    ```

    ### Streaming (Node.js)

    The snippet below uses `process.stdout`, which is available in Node.js. In the browser, append `chunk.delta.text` to your UI or a string buffer instead.

    ```javascript theme={null}
    import Anthropic from '@anthropic-ai/sdk';

    const client = new Anthropic({
      apiKey: 'sk-ant-...',
      baseURL: 'https://api.llm-ops.cloudidr.com',
      defaultHeaders: {
        'X-Cloudidr-Key': 'trk_...',
        'X-Agent': 'streaming-bot'
      }
    });

    const stream = await client.messages.stream({
      model: 'claude-sonnet-4-20250514',
      max_tokens: 1024,
      messages: [
        { role: 'user', content: 'Write a short poem about AI' }
      ]
    });

    for await (const chunk of stream) {
      if (chunk.type === 'content_block_delta' && 
          chunk.delta.type === 'text_delta') {
        process.stdout.write(chunk.delta.text);
      }
    }
    ```
  </Tab>

  <Tab title="cURL">
    ### Basic Example

    ```bash theme={null}
    curl https://api.llm-ops.cloudidr.com/v1/messages \
      -H "Content-Type: application/json" \
      -H "x-api-key: sk-ant-..." \
      -H "anthropic-version: 2023-06-01" \
      -H "X-Cloudidr-Key: trk_..." \
      -d '{
        "model": "claude-sonnet-4-20250514",
        "max_tokens": 1024,
        "messages": [
          {
            "role": "user",
            "content": "What is the capital of France?"
          }
        ]
      }'
    ```

    ### With Metadata (Department/Project/Agent Tracking)

    ```bash theme={null}
    curl https://api.llm-ops.cloudidr.com/v1/messages \
      -H "Content-Type: application/json" \
      -H "x-api-key: sk-ant-..." \
      -H "anthropic-version: 2023-06-01" \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Department: engineering" \
      -H "X-Project: backend" \
      -H "X-Agent: code-assistant" \
      -d '{
        "model": "claude-sonnet-4-20250514",
        "max_tokens": 1024,
        "messages": [
          {
            "role": "user",
            "content": "Help me debug this code"
          }
        ]
      }'
    ```

    ### Streaming Example

    ```bash theme={null}
    curl https://api.llm-ops.cloudidr.com/v1/messages \
      -H "Content-Type: application/json" \
      -H "x-api-key: sk-ant-..." \
      -H "anthropic-version: 2023-06-01" \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Agent: streaming-bot" \
      -d '{
        "model": "claude-sonnet-4-20250514",
        "max_tokens": 1024,
        "stream": true,
        "messages": [
          {
            "role": "user",
            "content": "Write a short poem about AI"
          }
        ]
      }'
    ```
  </Tab>
</Tabs>

***

## Cost Tracking Headers

Add these headers to organize your costs by department, team, or agent:

| Header           | Description                                 | Example                                           |
| ---------------- | ------------------------------------------- | ------------------------------------------------- |
| `X-Cloudidr-Key` | **Required** - Your Cloudidr tracking token | `trk_abc123...`                                   |
| `X-Department`   | Track costs by department                   | `engineering`, `sales`, `marketing`, `support`    |
| `X-Project`      | Track costs by team                         | `backend`, `frontend`, `ml`, `data`, `qa`         |
| `X-Agent`        | Track costs by agent/application            | `chatbot`, `summarizer`, `analyzer`, `translator` |

***

## Supported Models

All Anthropic Claude models are supported. See the [Supported Models](/guides/llm-ops/supported-models) page for the complete list of available models and pricing.

***

## What Gets Tracked

LLM Ops automatically captures:

✅ **Token usage** - Input and output tokens\
✅ **Cost** - Real-time cost calculation\
✅ **Latency** - Request duration\
✅ **Model** - Which Claude model was used\
✅ **Metadata** - Department, team, agent\
✅ **Errors** - Failed requests and error types

<Warning>
  **What We DON'T Track:**

  * ❌ Customer API keys
  * ❌ Request content (prompts)
  * ❌ Response content (completions)

  We only persist metadata needed for cost analytics in our application database.
</Warning>

***

## View Your Data

After making requests, view your costs in the [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard):

* **Agent Explorer** - See costs by agent/application
* **Department Breakdown** - Compare department spending
* **Team Analysis** - Track team-level costs
* **Model Comparison** - Compare costs across models
* **Time Series** - Track spending over time

***

## Migration from Direct API

Switching from direct Anthropic API to LLM Ops is a two-line change:

```python theme={null}
# Before
client = Anthropic(api_key="sk-ant-...")

# After - add base_url (API host only, no /v1) and X-Cloudidr-Key header
client = Anthropic(
    api_key="sk-ant-...",
    base_url="https://api.llm-ops.cloudidr.com",      # ← Add this
    default_headers={"X-Cloudidr-Key": "trk_..."}  # ← Add this
)
```

Everything else stays the same - no code changes needed!

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Requests not being tracked" icon="exclamation-triangle">
    **Check these common issues:**

    * ✅ For **SDKs**, set `base_url` / `baseURL` to `https://api.llm-ops.cloudidr.com` (do **not** include `/v1`; the SDK adds `/v1/messages`).
    * ✅ For **cURL**, call `https://api.llm-ops.cloudidr.com/v1/messages`.
    * ✅ Confirm `X-Cloudidr-Key` is included on every request (or in `extra_headers` / per-request `headers` where applicable).
    * ✅ Check that your Anthropic API key is valid.
  </Accordion>

  <Accordion title="Authentication errors" icon="key">
    **Two separate keys are needed:**

    * Your Anthropic API key (for Claude access)
    * Your Cloudidr tracking token (for cost tracking)

    Make sure both are set correctly and not swapped.
  </Accordion>

  <Accordion title="Cost data not appearing" icon="chart-line">
    **Wait a few moments:**

    * Cost data may take 10-30 seconds to appear in dashboard
    * Check the correct time range in dashboard filters
    * Verify requests are returning 200 OK status
  </Accordion>
</AccordionGroup>

***

## Next Steps

<CardGroup>
  <Card title="View Dashboard" icon="chart-line" href="https://llm-ops.cloudidr.com/login">
    See your Claude API costs in real-time
  </Card>

  <Card title="Supported Models" icon="list" href="/guides/llm-ops/supported-models">
    View all supported Claude models
  </Card>

  <Card title="OpenAI Integration" icon="brain" href="/guides/llm-ops/integrations/openai">
    Add cost tracking for GPT models
  </Card>

  <Card title="Set Budgets" icon="wallet" href="guides/llm-ops/budget-guard">
    Configure spending alerts and limits
  </Card>
</CardGroup>


# AWS Bedrock Integration
Source: https://docs.cloudidr.com/guides/llm-ops/integrations/aws-bedrock-integration

Learn how to integrate LLM Ops with AWS Bedrock (Converse API) for cost tracking and monitoring

## Overview

Track costs and monitor usage for **Amazon Bedrock** by sending **Converse** requests through LLM Ops. This integration is **not** a generic “swap the OpenAI base URL” flow: you call a dedicated path on the LLM Ops API that forwards to `bedrock-runtime.{region}.amazonaws.com` with AWS Signature Version 4 signing.

This guide shows **Python** (`requests`), **JavaScript** (`fetch`), and **cURL**. Use the official [AWS Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) request body shape (`messages`, `inferenceConfig`, etc.).

<Note>
  **Security:** LLM Ops does not store your **Cloudidr** tracking token, **AWS secret keys**, request prompts, or response content in the **analytics database**—only usage metadata needed for cost analytics. The proxy must forward the JSON body to AWS to complete the call. AWS credentials are used **in memory for the request** to sign the upstream call; they are **not** written to our analytics tables. Never commit real keys; use IAM least privilege and environment variables.
</Note>

## Quick Start

* **LLM Ops API host:** `https://api.llm-ops.cloudidr.com`
* **Endpoint pattern:** `POST /bedrock/model/{modelId}/converse`\
  Example model ID in the path: `amazon.nova-lite-v1:0` (dots and colons stay in the path segment).
* **Upstream API:** [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) only—not `InvokeModel` or third-party SDK defaults that talk to AWS without the proxy.

You need **two** kinds of credentials on every request:

1. **`X-Cloudidr-Key`** — LLM Ops tracking token (`trk_...`).
2. **AWS credentials** — `X-Aws-Access-Key-Id` and `X-Aws-Secret-Access-Key` (and optionally STS session token) so the proxy can sign the Bedrock request.

## API Keys and configuration

| Credential                       | Purpose                                                                                   |
| -------------------------------- | ----------------------------------------------------------------------------------------- |
| **Cloudidr Key**                 | From the [LLM Ops dashboard](https://llm-ops.cloudidr.com/dashboard); typically `trk_...` |
| **AWS Access Key ID**            | IAM user or role with `bedrock:InvokeModel` (and model access) in the target account      |
| **AWS Secret Access Key**        | Paired with the access key                                                                |
| **AWS Session Token** (optional) | For temporary STS credentials                                                             |

The marketing site [llmfinops.ai](https://llmfinops.ai) points at the same product; the dashboard URL above is the canonical app host.

Environment variables (recommended):

```bash theme={null}
export CLOUDIDR_KEY="trk_..."
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="us-east-1"
# Optional for STS:
# export AWS_SESSION_TOKEN="..."
```

***

## Integration Examples

<Tabs>
  <Tab title="Python">
    ### Install

    ```bash theme={null}
    pip install requests
    ```

    The AWS SDK for Python (`boto3`) calls AWS **directly**. To route through LLM Ops, use HTTP (e.g. `requests`) with the URL and headers below—matching how our [smoke tests](../test/smoke/test_bedrock_models.py) exercise the proxy.

    ### Basic Example

    ```python theme={null}
    import os
    import requests

    BASE = "https://api.llm-ops.cloudidr.com"
    MODEL_ID = "amazon.nova-lite-v1:0"

    url = f"{BASE}/bedrock/model/{MODEL_ID}/converse"
    headers = {
        "Content-Type": "application/json",
        "X-Cloudidr-Key": os.environ["CLOUDIDR_KEY"],
        "X-Aws-Access-Key-Id": os.environ["AWS_ACCESS_KEY_ID"],
        "X-Aws-Secret-Access-Key": os.environ["AWS_SECRET_ACCESS_KEY"],
        "X-Aws-Region": os.environ.get("AWS_REGION", "us-east-1"),
    }
    # Optional: "X-Aws-Session-Token": os.environ["AWS_SESSION_TOKEN"],

    body = {
        "messages": [
            {"role": "user", "content": [{"text": "What is the capital of France?"}]}
        ],
        "inferenceConfig": {"maxTokens": 256},
    }

    resp = requests.post(url, headers=headers, json=body, timeout=120)
    resp.raise_for_status()
    data = resp.json()
    text = data["output"]["message"]["content"][0].get("text", "")
    print(text)
    print("usage:", data.get("usage"))
    ```

    ### With Metadata (Department / Project / Agent)

    ```python theme={null}
    import os
    import requests

    BASE = "https://api.llm-ops.cloudidr.com"
    MODEL_ID = "amazon.nova-lite-v1:0"
    url = f"{BASE}/bedrock/model/{MODEL_ID}/converse"

    headers = {
        "Content-Type": "application/json",
        "X-Cloudidr-Key": os.environ["CLOUDIDR_KEY"],
        "X-Aws-Access-Key-Id": os.environ["AWS_ACCESS_KEY_ID"],
        "X-Aws-Secret-Access-Key": os.environ["AWS_SECRET_ACCESS_KEY"],
        "X-Aws-Region": os.environ.get("AWS_REGION", "us-east-1"),
        "X-Department": "engineering",
        "X-Project": "backend",
        "X-Agent": "bedrock-assistant",
    }

    body = {
        "messages": [
            {"role": "user", "content": [{"text": "Say hello in one sentence."}]}
        ],
        "inferenceConfig": {"maxTokens": 128},
    }

    resp = requests.post(url, headers=headers, json=body, timeout=120)
    ```

    ### Parsing the response (Converse shape)

    Successful responses include `output`, `usage`, and `stopReason`, for example:

    * `data["usage"]["inputTokens"]`, `data["usage"]["outputTokens"]`
    * Assistant text: `data["output"]["message"]["content"]` (list of blocks; often a block with `"text"`)

    Some models may return additional block types (e.g. reasoning); iterate over `content` as needed.
  </Tab>

  <Tab title="JavaScript">
    ### Basic Example (Node 18+ or browser with `fetch`)

    ```javascript theme={null}
    const BASE = 'https://api.llm-ops.cloudidr.com';
    const MODEL_ID = 'amazon.nova-lite-v1:0';

    const url = `${BASE}/bedrock/model/${MODEL_ID}/converse`;

    const body = {
      messages: [
        { role: 'user', content: [{ text: 'What is the capital of France?' }] },
      ],
      inferenceConfig: { maxTokens: 256 },
    };

    const res = await fetch(url, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'X-Cloudidr-Key': process.env.CLOUDIDR_KEY,
        'X-Aws-Access-Key-Id': process.env.AWS_ACCESS_KEY_ID,
        'X-Aws-Secret-Access-Key': process.env.AWS_SECRET_ACCESS_KEY,
        'X-Aws-Region': process.env.AWS_REGION || 'us-east-1',
      },
      body: JSON.stringify(body),
    });

    if (!res.ok) {
      throw new Error(await res.text());
    }
    const data = await res.json();
    const text = data.output?.message?.content?.find((b) => b.text)?.text;
    console.log(text);
    console.log('usage', data.usage);
    ```

    In **browsers**, avoid embedding long-lived AWS keys; prefer a backend that injects headers or short-lived STS credentials.

    ### With Metadata

    ```javascript theme={null}
    const res = await fetch(url, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'X-Cloudidr-Key': process.env.CLOUDIDR_KEY,
        'X-Aws-Access-Key-Id': process.env.AWS_ACCESS_KEY_ID,
        'X-Aws-Secret-Access-Key': process.env.AWS_SECRET_ACCESS_KEY,
        'X-Aws-Region': process.env.AWS_REGION || 'us-east-1',
        'X-Department': 'engineering',
        'X-Project': 'backend',
        'X-Agent': 'bedrock-assistant',
      },
      body: JSON.stringify(body),
    });
    ```
  </Tab>

  <Tab title="cURL">
    ### Basic Example

    ```bash theme={null}
    curl -sS -X POST \
      "https://api.llm-ops.cloudidr.com/bedrock/model/amazon.nova-lite-v1:0/converse" \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Aws-Access-Key-Id: AKIA..." \
      -H "X-Aws-Secret-Access-Key: ..." \
      -H "X-Aws-Region: us-east-1" \
      -d '{
        "messages": [
          {
            "role": "user",
            "content": [{"text": "What is the capital of France?"}]
          }
        ],
        "inferenceConfig": {"maxTokens": 256}
      }'
    ```

    ### With Metadata

    ```bash theme={null}
    curl -sS -X POST \
      "https://api.llm-ops.cloudidr.com/bedrock/model/amazon.nova-lite-v1:0/converse" \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Aws-Access-Key-Id: AKIA..." \
      -H "X-Aws-Secret-Access-Key: ..." \
      -H "X-Aws-Region: us-east-1" \
      -H "X-Department: engineering" \
      -H "X-Project: backend" \
      -H "X-Agent: bedrock-assistant" \
      -d '{
        "messages": [
          {"role": "user", "content": [{"text": "Say hello in one sentence."}]}
        ],
        "inferenceConfig": {"maxTokens": 128}
      }'
    ```

    ### Temporary AWS credentials (STS)

    ```bash theme={null}
    # Add:
    # -H "X-Aws-Session-Token: ..."
    ```
  </Tab>
</Tabs>

***

## Required and optional headers

| Header                    | Required | Description                                                         |
| ------------------------- | -------- | ------------------------------------------------------------------- |
| `X-Cloudidr-Key`          | Yes      | LLM Ops tracking token (`trk_...`)                                  |
| `X-Aws-Access-Key-Id`     | Yes      | AWS access key ID                                                   |
| `X-Aws-Secret-Access-Key` | Yes      | AWS secret access key                                               |
| `X-Aws-Session-Token`     | No       | STS session token when using temporary credentials                  |
| `X-Aws-Region`            | No       | AWS region for Bedrock runtime (default **`us-east-1`** if omitted) |
| `X-Department`            | No       | Cost attribution: department                                        |
| `X-Project`               | No       | Cost attribution: project/team (preferred)                          |
| `X-Team`                  | No       | Legacy alias for the same tag as `X-Project`                        |
| `X-Agent`                 | No       | Cost attribution: agent or app name                                 |

***

## Model IDs and cross-region inference

The `{modelId}` in the URL is passed to AWS (after normalization). For some providers, Bedrock expects a **cross-region inference profile** ID (prefix `us.`, `eu.`, or `ap.`). The proxy may **automatically prepend** the right prefix when you use a plain ID for:

* `anthropic.*`
* `meta.*`
* `deepseek.*`

The geo prefix is derived from **`X-Aws-Region`** (e.g. `eu-*` → `eu.`, `ap-*` → `ap.`, otherwise often `us.`). If your model ID **already** starts with `us.`, `eu.`, or `ap.`, it is left unchanged.

Many Amazon models (e.g. **Amazon Nova**) use the **plain** model ID without a `us.` prefix. Always confirm the exact ID in the [AWS Bedrock console](https://console.aws.amazon.com/bedrock) for your account and region.

**Anthropic models on Bedrock:** Your account must have model access enabled; Anthropic often requires completing the **use-case** form under **Bedrock → Model access → Anthropic** in the AWS console.

***

## Converse features (streaming, tools)

The proxy forwards the **Converse** JSON body to AWS. Features such as **streaming** (`stream` in the Converse request) or **tool configuration** follow the **AWS Converse API** specification. Refer to the latest AWS documentation for field names and behavior; LLM Ops records usage from successful responses when token counts are present.

***

## Supported Models

Models available to your AWS account in the chosen region can be used. See the [Supported Models](/guides/llm-ops/supported-models) page for pricing alignment in LLM Ops.

***

## What Gets Tracked

LLM Ops automatically captures:

✅ **Token usage** — Input and output tokens from the Converse `usage` object where available\
✅ **Cost** — Estimated cost from LLM Ops pricing\
✅ **Latency** — Request duration and provider timing\
✅ **Model** — Effective model ID used for Bedrock\
✅ **Metadata** — Department, project/team, agent\
✅ **Errors** — HTTP status and summarized error messages\
✅ **Optimizer** — When enabled, routing metadata for cost-optimizer decisions

<Warning>
  **What we do not persist in the analytics database:**

  * ❌ Cloudidr or AWS secret keys
  * ❌ Full prompt or completion text
  * ❌ Raw request/response bodies as searchable content

  Operational logging in your environment may still exist; treat headers and bodies as sensitive in transit.
</Warning>

***

## View Your Data

After making requests, view costs in the [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard):

* **Agent Explorer** — Costs by agent
* **Department Breakdown** — Department spending
* **Team Analysis** — Project/team-level costs
* **Model Comparison** — Compare models (including Bedrock IDs)
* **Time Series** — Spend over time

***

## Migration from calling Bedrock directly

Previously you might have called:

`POST https://bedrock-runtime.{region}.amazonaws.com/model/{modelId}/converse` with SigV4 from your code.

With LLM Ops:

1. **Change the URL** to `https://api.llm-ops.cloudidr.com/bedrock/model/{modelId}/converse`.
2. **Add** `X-Cloudidr-Key: trk_...` on every request.
3. **Pass AWS credentials** using the headers above so the **proxy** can sign the upstream Bedrock call (you can remove local SigV4 signing when using the proxy, unless you keep a different architecture).

Keep the **same Converse JSON body** you used with AWS.

```text theme={null}
# Conceptual diff
- POST https://bedrock-runtime.us-east-1.amazonaws.com/model/MODEL/converse  (signed by you)
+ POST https://api.llm-ops.cloudidr.com/bedrock/model/MODEL/converse
+   Headers: X-Cloudidr-Key, X-Aws-Access-Key-Id, X-Aws-Secret-Access-Key, X-Aws-Region, ...
```

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="400 Missing X-Cloudidr-Key" icon="exclamation-triangle">
    Every request must include **`X-Cloudidr-Key`** with a valid LLM Ops tracking token. Get or rotate tokens in the dashboard.
  </Accordion>

  <Accordion title="401 AWS authentication failed" icon="key">
    Check **`X-Aws-Access-Key-Id`**, **`X-Aws-Secret-Access-Key`**, and optional **`X-Aws-Session-Token`**. Ensure the IAM principal can invoke Bedrock in **`X-Aws-Region`**. Keys must not be swapped with the Cloudidr token.
  </Accordion>

  <Accordion title="403 / 404 Model access" icon="server">
    Enable the model in **Amazon Bedrock → Model access** for your account and region. For Anthropic, complete the console use-case step if required.
  </Accordion>

  <Accordion title="Wrong region or model ID" icon="globe">
    Set **`X-Aws-Region`** to the region where the model is available. Verify the model ID string (including version suffixes like `:0`) matches AWS documentation.
  </Accordion>

  <Accordion title="Cost data not appearing" icon="chart-line">
    Allow **10–30 seconds** for dashboard updates. Confirm HTTP **200** from the proxy and a valid **`X-Cloudidr-Key`** so the request is attributed to your org.
  </Accordion>
</AccordionGroup>

***

## Next Steps

<CardGroup>
  <Card title="View Dashboard" icon="chart-line" href="https://llm-ops.cloudidr.com/dashboard">
    See your OpenAI API costs in real-time
  </Card>

  <Card title="Supported Models" icon="list" href="/guides/llm-ops/supported-models">
    View all supported OpenAI models
  </Card>

  <Card title="Anthropic Integration" icon="sparkles" href="/guides/llm-ops/integrations/anthropic">
    Add cost tracking for Claude models
  </Card>

  <Card title="Set Budgets" icon="wallet" href="guides/llm-ops/budget-guard">
    Configure spending alerts and limits
  </Card>
</CardGroup>


# Google Integration
Source: https://docs.cloudidr.com/guides/llm-ops/integrations/google

Learn how to integrate LLM Ops with Google's Gemini API for cost tracking and monitoring

## Overview

Track costs and monitor usage for Google's Gemini API by routing your requests through LLM Ops. This guide shows you how to integrate using Python, JavaScript, or cURL.

<Note>
  **Security Guarantee**: LLM Ops does not store your API keys, request prompts, or response content in the **analytics database**—only metadata needed for cost analytics. The proxy must forward request bodies to Google to complete the call; optional operational logging may exist in your deployment environment.
</Note>

## Quick Start

Point the Gemini client at the LLM Ops **API host** (same path layout as Google: `/v1beta/models/...`). The proxy serves:

* **Original API host:** `https://generativelanguage.googleapis.com`
* **LLM Ops API host:** `https://api.llm-ops.cloudidr.com` (paths such as `/v1beta/models/{model}:generateContent` stay the same; only the host changes)

## API Keys

You'll need two credentials:

1. **Google API Key** - Your Gemini API key from [aistudio.google.com](https://aistudio.google.com) (or your Google Cloud project)
2. **Cloudidr Key** - Your tracking token from the [LLM Ops dashboard](https://llm-ops.cloudidr.com/dashboard) (tokens are typically prefixed with `trk_`)

The marketing site [llmfinops.ai](https://llmfinops.ai) points at the same product; the dashboard URL above is the canonical app host.

Set them as environment variables:

```bash theme={null}
export GOOGLE_API_KEY="AIzaSy..."
export CLOUDIDR_KEY="trk_..."
```

***

## Integration Examples

<Tabs>
  <Tab title="Python">
    ### Install SDK

    ```bash theme={null}
    pip install google-generativeai
    ```

    ### Basic Example

    ```python theme={null}
    import google.generativeai as genai

    genai.configure(
        api_key="AIzaSy...",  # Your Google API key
        transport="rest",
        client_options={
            "api_endpoint": "https://api.llm-ops.cloudidr.com"
        }
    )

    model = genai.GenerativeModel('gemini-2.0-flash-exp')

    response = model.generate_content(
        "What is the capital of France?",
        request_options={
            "headers": {
                "X-Cloudidr-Key": "trk_..."
            }
        }
    )

    print(response.text)
    ```

    ### With Metadata (Department/Team/Agent Tracking)

    ```python theme={null}
    import google.generativeai as genai

    genai.configure(
        api_key="AIzaSy...",
        transport="rest",
        client_options={
            "api_endpoint": "https://api.llm-ops.cloudidr.com"
        }
    )

    model = genai.GenerativeModel('gemini-2.0-flash-exp')

    # X-Project is preferred for team/project; X-Team is a legacy alias
    response = model.generate_content(
        "Explain quantum computing in simple terms",
        request_options={
            "headers": {
                "X-Cloudidr-Key": "trk_...",
                "X-Department": "research",
                "X-Team": "ml",
                "X-Agent": "science-explainer"
            }
        }
    )

    print(response.text)
    ```

    ### Streaming Example

    ```python theme={null}
    import google.generativeai as genai

    genai.configure(
        api_key="AIzaSy...",
        transport="rest",
        client_options={
            "api_endpoint": "https://api.llm-ops.cloudidr.com"
        }
    )

    model = genai.GenerativeModel('gemini-2.0-flash-exp')

    response = model.generate_content(
        "Write a story about a robot learning to paint",
        stream=True,
        request_options={
            "headers": {
                "X-Cloudidr-Key": "trk_...",
                "X-Agent": "story-generator"
            }
        }
    )

    for chunk in response:
        print(chunk.text, end="", flush=True)
    ```

    ### Chat Example (Multi-turn)

    ```python theme={null}
    import google.generativeai as genai

    genai.configure(
        api_key="AIzaSy...",
        transport="rest",
        client_options={
            "api_endpoint": "https://api.llm-ops.cloudidr.com"
        }
    )

    model = genai.GenerativeModel('gemini-2.0-flash-exp')
    chat = model.start_chat(history=[])

    headers = {
        "X-Cloudidr-Key": "trk_...",
        "X-Agent": "chat-bot"
    }

    response1 = chat.send_message(
        "Hello! What's your name?",
        request_options={"headers": headers}
    )
    print(response1.text)

    response2 = chat.send_message(
        "Can you help me with Python?",
        request_options={"headers": headers}
    )
    print(response2.text)
    ```

    ### Multimodal Example (Image Analysis)

    ```python theme={null}
    import google.generativeai as genai
    from PIL import Image

    genai.configure(
        api_key="AIzaSy...",
        transport="rest",
        client_options={
            "api_endpoint": "https://api.llm-ops.cloudidr.com"
        }
    )

    model = genai.GenerativeModel('gemini-2.0-flash-exp')

    img = Image.open('photo.jpg')
    response = model.generate_content(
        ["What's in this image?", img],
        request_options={
            "headers": {
                "X-Cloudidr-Key": "trk_...",
                "X-Agent": "vision-analyzer"
            }
        }
    )

    print(response.text)
    ```
  </Tab>

  <Tab title="JavaScript">
    ### Install SDK

    ```bash theme={null}
    npm install @google/generative-ai
    ```

    ### Basic Example

    ```javascript theme={null}
    import { GoogleGenerativeAI } from '@google/generative-ai';

    const genAI = new GoogleGenerativeAI('AIzaSy...', {
      baseUrl: 'https://api.llm-ops.cloudidr.com'
    });

    const model = genAI.getGenerativeModel({ 
      model: 'gemini-2.0-flash-exp' 
    });

    const result = await model.generateContent(
      'What is the capital of France?',
      {
        headers: {
          'X-Cloudidr-Key': 'trk_...'
        }
      }
    );

    console.log(result.response.text());
    ```

    ### With Metadata (Department/Team/Agent Tracking)

    ```javascript theme={null}
    import { GoogleGenerativeAI } from '@google/generative-ai';

    const genAI = new GoogleGenerativeAI('AIzaSy...', {
      baseUrl: 'https://api.llm-ops.cloudidr.com'
    });

    const model = genAI.getGenerativeModel({ 
      model: 'gemini-2.0-flash-exp' 
    });

    const result = await model.generateContent(
      'Explain quantum computing in simple terms',
      {
        headers: {
          'X-Cloudidr-Key': 'trk_...',
          'X-Department': 'research',
          'X-Team': 'ml',
          'X-Agent': 'science-explainer'
        }
      }
    );

    console.log(result.response.text());
    ```

    ### Streaming (Node.js)

    The snippet below uses `process.stdout`, which is available in Node.js. In the browser, append chunk text to your UI or a string buffer instead.

    ```javascript theme={null}
    import { GoogleGenerativeAI } from '@google/generative-ai';

    const genAI = new GoogleGenerativeAI('AIzaSy...', {
      baseUrl: 'https://api.llm-ops.cloudidr.com'
    });

    const model = genAI.getGenerativeModel({ 
      model: 'gemini-2.0-flash-exp' 
    });

    const result = await model.generateContentStream(
      'Write a story about a robot learning to paint',
      {
        headers: {
          'X-Cloudidr-Key': 'trk_...',
          'X-Agent': 'story-generator'
        }
      }
    );

    for await (const chunk of result.stream) {
      process.stdout.write(chunk.text());
    }
    ```

    ### Chat Example (Multi-turn)

    ```javascript theme={null}
    import { GoogleGenerativeAI } from '@google/generative-ai';

    const genAI = new GoogleGenerativeAI('AIzaSy...', {
      baseUrl: 'https://api.llm-ops.cloudidr.com'
    });

    const model = genAI.getGenerativeModel({ 
      model: 'gemini-2.0-flash-exp' 
    });

    const chat = model.startChat({
      history: []
    });

    const headers = {
      'X-Cloudidr-Key': 'trk_...',
      'X-Agent': 'chat-bot'
    };

    const result1 = await chat.sendMessage(
      "Hello! What's your name?",
      { headers }
    );
    console.log(result1.response.text());

    const result2 = await chat.sendMessage(
      'Can you help me with Python?',
      { headers }
    );
    console.log(result2.response.text());
    ```
  </Tab>

  <Tab title="cURL">
    ### Basic Example

    ```bash theme={null}
    curl "https://api.llm-ops.cloudidr.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=AIzaSy..." \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Key: trk_..." \
      -d '{
        "contents": [
          {
            "parts": [
              {
                "text": "What is the capital of France?"
              }
            ]
          }
        ]
      }'
    ```

    ### With Metadata (Department/Team/Agent Tracking)

    ```bash theme={null}
    curl "https://api.llm-ops.cloudidr.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=AIzaSy..." \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Department: research" \
      -H "X-Team: ml" \
      -H "X-Agent: science-explainer" \
      -d '{
        "contents": [
          {
            "parts": [
              {
                "text": "Explain quantum computing in simple terms"
              }
            ]
          }
        ]
      }'
    ```

    ### Streaming Example

    ```bash theme={null}
    curl "https://api.llm-ops.cloudidr.com/v1beta/models/gemini-2.0-flash-exp:streamGenerateContent?key=AIzaSy..." \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Agent: story-generator" \
      -d '{
        "contents": [
          {
            "parts": [
              {
                "text": "Write a story about a robot learning to paint"
              }
            ]
          }
        ]
      }'
    ```

    ### With Generation Config

    ```bash theme={null}
    curl "https://api.llm-ops.cloudidr.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=AIzaSy..." \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Agent: creative-writer" \
      -d '{
        "contents": [
          {
            "parts": [
              {
                "text": "Write a creative product description"
              }
            ]
          }
        ],
        "generationConfig": {
          "temperature": 0.9,
          "topK": 1,
          "topP": 1,
          "maxOutputTokens": 2048
        }
      }'
    ```
  </Tab>
</Tabs>

***

## Cost Tracking Headers

| Header           | Description                                         | Example                                           |
| ---------------- | --------------------------------------------------- | ------------------------------------------------- |
| `X-Cloudidr-Key` | **Required** - Your Cloudidr tracking token         | `trk_abc123...`                                   |
| `X-Department`   | Track costs by department                           | `engineering`, `sales`, `marketing`, `support`    |
| `X-Project`      | Track costs by project/team (preferred)             | `backend`, `frontend`, `ml`, `data`, `qa`         |
| `X-Team`         | Legacy alias for project/team (same as `X-Project`) | `backend`, `frontend`                             |
| `X-Agent`        | Track costs by agent/application                    | `chatbot`, `summarizer`, `analyzer`, `translator` |

***

## Supported Models

All Google Gemini models supported by the proxy are available. See the [Supported Models](/guides/llm-ops/supported-models) page for the complete list of available models and pricing.

***

## What Gets Tracked

LLM Ops automatically captures:

✅ **Token usage** - Input and output tokens (including multimodal input counted toward input tokens)\
✅ **Cost** - Real-time cost calculation\
✅ **Latency** - Request duration\
✅ **Model** - Which Gemini model was used\
✅ **Metadata** - Department, team, agent\
✅ **Errors** - Failed requests and error types\
✅ **Multimodal inputs** - Media you send affects token usage; totals appear in input tokens from Google's usage metadata

Google may report **hidden** or **thoughts** tokens (context, safety, etc.) in usage; LLM Ops uses those counts for billing alignment where present.

<Warning>
  **What We DON'T Track:**

  * ❌ Customer API keys
  * ❌ Request content (prompts)
  * ❌ Response content (completions)
  * ❌ Raw image/video/audio bytes in our analytics database

  We only persist metadata needed for cost analytics in our application database. We do **not** store full Gemini safety rating objects as separate dashboard fields unless your product explicitly adds that—token-based usage is the primary signal in the proxy.
</Warning>

***

## View Your Data

After making requests, view your costs in the [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard):

* **Agent Explorer** - See costs by agent/application
* **Department Breakdown** - Compare department spending
* **Team Analysis** - Track team-level costs
* **Model Comparison** - Compare costs across Gemini models
* **Time Series** - Track spending over time

***

## Migration from Direct API

Switching from direct Gemini API to LLM Ops requires updating the endpoint and adding the tracking header on each request:

```python theme={null}
# Before
genai.configure(api_key="AIzaSy...")

# After - point REST transport at the proxy host
genai.configure(
    api_key="AIzaSy...",
    transport="rest",
    client_options={
        "api_endpoint": "https://api.llm-ops.cloudidr.com"
    }
)

response = model.generate_content(
    "Your prompt",
    request_options={
        "headers": {"X-Cloudidr-Key": "trk_..."}
    }
)
```

***

## Multimodal Support

Gemini supports images, video, and audio—all requests go through the same proxy and are billed from Google's usage metadata:

```python theme={null}
import google.generativeai as genai
from PIL import Image

genai.configure(
    api_key="AIzaSy...",
    transport="rest",
    client_options={
        "api_endpoint": "https://api.llm-ops.cloudidr.com"
    }
)

model = genai.GenerativeModel('gemini-2.0-flash-exp')

img = Image.open('photo.jpg')
response = model.generate_content(
    ["What's in this image?", img],
    request_options={
        "headers": {
            "X-Cloudidr-Key": "trk_...",
            "X-Agent": "vision-analyzer"
        }
    }
)

print(response.text)
```

<Note>
  **Multimodal token tracking:**

  Google converts images/video/audio to tokens and includes them in usage. LLM Ops records **total input/output tokens** and cost—typically **not** a separate line item per modality in the database.
</Note>

***

## Cost Optimization Tips

<AccordionGroup>
  <Accordion title="Monitor High-Token Multimodal Requests" icon="image">
    Images and videos can consume significant tokens:

    * Track total input token usage in dashboard
    * Identify agents with high token consumption
    * Optimize image resolution before sending to API

    LLM Ops tracks total input tokens (text + multimodal combined in usage).
  </Accordion>

  <Accordion title="Leverage Large Context Windows" icon="expand">
    Many Gemini models support large context windows (limits vary by model):

    * Process large documents in fewer calls when appropriate
    * Balance context size vs. token cost

    Fewer round trips can reduce overhead; very large contexts still incur proportional token cost.
  </Accordion>

  <Accordion title="Compare Model Variants" icon="chart-bar">
    Use the dashboard to find cost-saving opportunities:

    * Track performance vs. cost by model
    * Test different Gemini variants for your workload
    * Move high-volume, low-complexity tasks to cheaper models where quality allows
  </Accordion>
</AccordionGroup>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Requests not being tracked" icon="exclamation-triangle">
    **Check these common issues:**

    * ✅ Use API host `https://api.llm-ops.cloudidr.com` with Gemini paths (`/v1beta/models/...`)—same structure as `generativelanguage.googleapis.com`.
    * ✅ Confirm the header name is **`X-Cloudidr-Key`** (not `X-Cloudidr-Token`) on every request.
    * ✅ Pass your Google API key as Google expects (`?key=`, `x-goog-api-key`, or `Authorization`, depending on client).
    * ✅ Verify your Cloudidr tracking token is valid.
  </Accordion>

  <Accordion title="Authentication errors" icon="key">
    **Two separate keys are needed:**

    * Your Google API key (for Gemini access)
    * Your Cloudidr tracking token (for cost tracking)

    Make sure both are set correctly and not swapped.
  </Accordion>

  <Accordion title="Cost data not appearing" icon="chart-line">
    **Wait a few moments:**

    * Cost data may take 10-30 seconds to appear in dashboard
    * Check the correct time range in dashboard filters
    * Verify requests are returning 200 OK status
  </Accordion>
</AccordionGroup>

***

## Next Steps

<CardGroup>
  <Card title="View Dashboard" icon="chart-line" href="https://llmfinops.ai/dashboard">
    See your Gemini API costs in real-time
  </Card>

  <Card title="Supported Models" icon="list" href="/guides/llm-ops/supported-models">
    View all supported Gemini models
  </Card>

  <Card title="OpenAI Integration" icon="brain" href="/guides/llm-ops/integrations/openai">
    Add cost tracking for GPT models
  </Card>

  <Card title="Set Budgets" icon="wallet" href="guides/llm-ops/budget-guard">
    Configure spending alerts and limits
  </Card>
</CardGroup>


# LiteLLM Integration
Source: https://docs.cloudidr.com/guides/llm-ops/integrations/litellm

Track your LiteLLM proxy usage and costs in Cloudidr LLM Ops

## Overview

Integrate LiteLLM with Cloudidr LLM Ops to automatically track API usage and costs. LiteLLM sends webhook callbacks to our system after each API request, allowing you to monitor usage, costs, and organize by department, team, and agent.

<Note>
  **How it works:**

  LiteLLM → Webhook Callback → CloudIDR LLM Ops → Dashboard

  Configure LiteLLM to send webhooks to our endpoint, add tracking headers to your API requests, and we'll automatically track all usage and costs.
</Note>

***

## Your Tracking Token

<Warning>
  **Keep this token secure!**

  Your tracking token is required for all LiteLLM webhook callbacks. Don't share it publicly or commit it to version control.
</Warning>

```
trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr
```

<Tip>
  Use environment variables to store your tracking token securely.
</Tip>

***

## Configuration Steps

<Steps>
  <Step title="Update LiteLLM config.yaml">
    Add webhook callback configuration to your LiteLLM config file
  </Step>

  <Step title="Add Tracking Headers">
    Include required and optional headers in your API requests
  </Step>

  <Step title="Restart & Test">
    Restart LiteLLM and verify tracking in your dashboard
  </Step>
</Steps>

***

## Step 1: Update LiteLLM config.yaml

Choose the configuration that matches your setup:

<Tabs>
  <Tab title="With Database (Recommended)">
    ### Configuration with Database

    Use this if you want to keep LiteLLM's database features (user management, spend tracking, etc.)

    ```yaml theme={null}
    # LiteLLM Proxy Configuration - With Database
    model_list:
      - model_name: gpt-4
        litellm_params:
          model: gpt-4
          api_key: os.environ/OPENAI_API_KEY
      - model_name: claude-sonnet-4
        litellm_params:
          model: claude-sonnet-4
          api_key: os.environ/ANTHROPIC_API_KEY

    # Generic API Callback - Point to our webhook endpoint
    callback_settings:
      llm_ops:
        callback_type: generic_api
        endpoint: https://api.llm-ops.cloudidr.com/api/litellm/callback
        headers:
          Content-Type: application/json
        event_types:
          - llm_api_success
          - llm_api_failure

    litellm_settings:
      callbacks: ["llm_ops"]
      # Use your existing LiteLLM database (PostgreSQL, SQLite, etc.)
      database_url: os.environ/DATABASE_URL  # e.g., "postgresql://user:pass@host:5432/litellm"

    general_settings:
      master_key: os.environ/LITELLM_MASTER_KEY
    ```

    <Info>
      **Recommended for most users** - This preserves all LiteLLM features while adding LLM Ops tracking.
    </Info>
  </Tab>

  <Tab title="Without Database (Optional)">
    ### Configuration without Database

    Use this **only** if you don't need LiteLLM's database features and want to disable them.

    ```yaml theme={null}
    # LiteLLM Proxy Configuration - Without Database
    # ⚠️  Note: This disables LiteLLM's built-in features like user management and spend tracking
    # Only use this if you're managing everything through LLM Ops

    model_list:
      - model_name: gpt-4
        litellm_params:
          model: gpt-4
          api_key: os.environ/OPENAI_API_KEY

    # Generic API Callback - Point to our webhook endpoint
    callback_settings:
      llm_ops:
        callback_type: generic_api
        endpoint: https://api.llm-ops.cloudidr.com/api/litellm/callback
        headers:
          Content-Type: application/json
        event_types:
          - llm_api_success
          - llm_api_failure

    litellm_settings:
      callbacks: ["llm_ops"]
      database_url: ""  # Empty string disables database

    general_settings:
      master_key: os.environ/LITELLM_MASTER_KEY
      disable_database_checks: true  # Required when database_url is empty
    ```

    <Warning>
      **Important:** This disables LiteLLM's user management and spend tracking features. Only use if you're managing everything through LLM Ops.
    </Warning>
  </Tab>
</Tabs>

<Tip>
  **Recommendation:** If you're already using LiteLLM with a database, use the "With Database" configuration. Only use "Without Database" if you don't need LiteLLM's built-in features.
</Tip>

***

## Step 2: Add Tracking Headers to Your Requests

Include tracking headers in your API requests to LiteLLM.

### Required Header

| Header             | Description                        | Example                        |
| ------------------ | ---------------------------------- | ------------------------------ |
| `X-Cloudidr-Token` | **Required** - Your tracking token | `trk_fXOn-A1V8VrCxXyJ1WuMX...` |

### Optional Metadata Headers

| Header         | Description                         | Example                             |
| -------------- | ----------------------------------- | ----------------------------------- |
| `X-Department` | Organize costs by department        | `engineering`, `sales`, `marketing` |
| `X-Team`       | Organize costs by team              | `backend`, `frontend`, `ml`         |
| `X-Agent`      | Organize costs by agent/application | `chatbot`, `summarizer`, `analyzer` |

<Note>
  **Optional Metadata:** These headers are optional. If omitted, requests will still be tracked, but won't be organized by department/team/agent in your dashboard.
</Note>

***

### Code Examples

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import openai
    from openai import OpenAI

    client = OpenAI(
        api_key="your-openai-key",
        base_url="http://localhost:4000/v1",  # Your LiteLLM proxy URL local host or your production url
    )

    # Make request with tracking headers
    # X-Cloudidr-Token is required; X-Department, X-Team, X-Agent are optional
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello!"}],
        extra_headers={
            "X-Cloudidr-Token": "trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr",  # Required
            # Optional metadata headers:
            "X-Department": "engineering",  # Optional
            "X-Team": "backend",            # Optional
            "X-Agent": "chatbot"            # Optional
        }
    )

    print(response.choices[0].message.content)
    ```

    <Tip>
      Use `extra_headers` parameter to pass custom headers to LiteLLM.
    </Tip>
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    import OpenAI from 'openai';

    const client = new OpenAI({
      apiKey: 'your-openai-key',
      baseURL: 'http://localhost:4000/v1',  // Your LiteLLM proxy URL
      defaultHeaders: {
        'X-Cloudidr-Token': 'trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr',  // Required
        // Optional metadata headers:
        'X-Department': 'engineering',  // Optional
        'X-Team': 'backend',            // Optional
        'X-Agent': 'chatbot'            // Optional
      }
    });

    const response = await client.chat.completions.create({
      model: 'gpt-4',
      messages: [{ role: 'user', content: 'Hello!' }]
    });

    console.log(response.choices[0].message.content);
    ```

    <Tip>
      Use `defaultHeaders` to set headers once for all requests.
    </Tip>
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST http://localhost:4000/v1/chat/completions \
      -H "Authorization: Bearer your-master-key" \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Token: trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr" \
      -H "X-Department: engineering" \
      -H "X-Team: backend" \
      -H "X-Agent: chatbot" \
      -d '{
        "model": "gpt-4",
        "messages": [
          {
            "role": "user",
            "content": "Hello!"
          }
        ]
      }'
    ```

    <Note>
      **Note:** `X-Cloudidr-Token` is required. `X-Department`, `X-Team`, `X-Agent` are optional.
    </Note>
  </Tab>
</Tabs>

***

## Step 3: Restart LiteLLM and Test

Restart your LiteLLM proxy and make a test request. Check your dashboard to verify the request was tracked.

```bash theme={null}
# Restart LiteLLM with updated config
litellm --config config.yaml --port 4000

# Make a test request
curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer your-master-key" \
  -H "Content-Type: application/json" \
  -H "X-Cloudidr-Token: trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

<Info>
  **Success!** If everything is configured correctly, you should see the request appear in your [LLM Ops Dashboard](https://llmfinops.ai/dashboard) within a few seconds.
</Info>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Webhooks not being received?" icon="webhook">
    **Common issues:**

    * ✅ Verify `callback_settings` is correctly formatted in `config.yaml`
    * ✅ Check that `headers` field is included (even if just `Content-Type: application/json`)
    * ✅ Ensure the endpoint URL is accessible from your LiteLLM server
    * ✅ Check LiteLLM logs for callback errors (enable `set_verbose: true`)

    **Test webhook connectivity:**

    ```bash theme={null}
    curl -X POST https://api.llm-ops.cloudidr.com/api/litellm/callback \
      -H "Content-Type: application/json" \
      -d '{"test": true}'
    ```
  </Accordion>

  <Accordion title="Requests not showing in dashboard?" icon="chart-line">
    **Common issues:**

    * ✅ Verify `X-Cloudidr-Token` header is included in your requests
    * ✅ Check that the tracking token is active (not revoked)
    * ✅ Ensure the token belongs to your organization
    * ✅ Check API server logs for `[LITELLM]` messages

    **Debug tip:** Enable verbose logging in LiteLLM:

    ```yaml theme={null}
    litellm_settings:
      set_verbose: true
    ```
  </Accordion>

  <Accordion title="Metadata (Department/Team/Agent) not showing?" icon="tags">
    **Remember:**

    * These headers are **optional** - requests will still be tracked without them
    * Verify headers are passed correctly: `X-Department`, `X-Team`, `X-Agent`
    * Check that headers are passed via `extra_headers` (Python) or `defaultHeaders` (JavaScript)
    * For cURL, include headers directly: `-H "X-Department: engineering"`

    <Note>
      **Note:** We currently extract metadata from custom headers. LiteLLM's built-in `x-litellm-tags` header is not automatically mapped to our metadata fields.
    </Note>
  </Accordion>

  <Accordion title="Database configuration issues?" icon="database">
    **With Database:**

    * Keep your existing `database_url` configuration
    * Don't add `disable_database_checks`

    **Without Database:**

    * Set `database_url: ""` (empty string)
    * Must add `disable_database_checks: true`

    **Error: "Database checks failed"**

    * You forgot `disable_database_checks: true` when using empty `database_url`
  </Accordion>
</AccordionGroup>

***

## What Gets Tracked

LLM Ops automatically captures from LiteLLM webhooks:

✅ **Token usage** - Input, output, and total tokens\
✅ **Cost** - Real-time cost calculation\
✅ **Latency** - Request duration\
✅ **Model** - Which model was used\
✅ **Metadata** - Department, team, agent (from headers)\
✅ **Errors** - Failed requests and error types\
✅ **Source** - Marked as `litellm` in the database

<Warning>
  **What We DON'T Track:**

  * ❌ Customer API keys
  * ❌ Request content (prompts)
  * ❌ Response content (completions)

  We only track metadata needed for cost analytics.
</Warning>

***

## Important Notes

<AccordionGroup>
  <Accordion title="Required vs Optional Headers" icon="info-circle">
    * **Required:** `X-Cloudidr-Token` header must be included in all requests
    * **Optional:** `X-Department`, `X-Team`, `X-Agent` headers are optional metadata for organizing costs
  </Accordion>

  <Accordion title="LiteLLM Proxy Mode" icon="server">
    * LiteLLM proxy mode requires `callback_settings` with `generic_api` type
    * Direct URLs don't work - must use webhook callbacks
    * The `headers` field is required in `callback_settings`, even if just `Content-Type: application/json`
  </Accordion>

  <Accordion title="Database Configuration" icon="database">
    * If you're using LiteLLM's database, keep your existing `database_url` configuration
    * Only set `database_url: ""` and `disable_database_checks: true` if you don't need LiteLLM's database features
  </Accordion>

  <Accordion title="Security Best Practices" icon="shield">
    * For production, use environment variables for API keys and master keys
    * Never commit tokens or keys to version control
    * Rotate tracking tokens regularly
    * Use HTTPS for production deployments
  </Accordion>

  <Accordion title="LiteLLM Tags" icon="tag">
    We currently extract metadata from custom headers (`X-Department`, etc.).

    LiteLLM's built-in `x-litellm-tags` header is not automatically mapped to our metadata fields.

    Use the `X-Department`, `X-Team`, `X-Agent` headers for proper organization.
  </Accordion>
</AccordionGroup>

***

## View Your Data

After making requests, view your costs in the [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard):

* **Agent Explorer** - See costs by agent/application
* **Department Breakdown** - Compare department spending
* **Team Analysis** - Track team-level costs
* **Model Comparison** - Compare costs across models routed through LiteLLM
* **Time Series** - Track spending over time
* **Source Filter** - Filter by source (all LiteLLM requests marked as `litellm`)

***

## LiteLLM Features

LiteLLM provides powerful features that work seamlessly with LLM Ops tracking:

<CardGroup>
  <Card title="Multi-Provider Support" icon="layer-group">
    Route to OpenAI, Anthropic, Google, and 100+ providers
  </Card>

  <Card title="Load Balancing" icon="scale-balanced">
    Distribute requests across multiple API keys
  </Card>

  <Card title="Fallback Handling" icon="shield">
    Automatic failover when providers are down
  </Card>

  <Card title="Rate Limiting" icon="gauge">
    Control costs with built-in rate limits
  </Card>
</CardGroup>

All features work with LLM Ops cost tracking - no configuration needed!

***

## Need Help?

<CardGroup>
  <Card title="Email Support" icon="envelope" href="mailto:support@cloudidr.com">
    Contact us at [support@cloudidr.com](mailto:hello@cloudidr.com)
  </Card>

  <Card title="Discord Community" icon="discord" href="https://discord.gg/cloudidr">
    Join our Discord for quick help
  </Card>

  <Card title="LiteLLM Docs" icon="book" href="https://docs.litellm.ai">
    Official LiteLLM documentation
  </Card>

  <Card title="View Dashboard" icon="chart-line" href="https://llm-ops.cloudidr.com/dashboard">
    Check your tracked requests
  </Card>
</CardGroup>

***

## Next Steps

<Steps>
  <Step title="Configure LiteLLM">
    Add webhook callback to your config.yaml
  </Step>

  <Step title="Add Headers">
    Include X-Cloudidr-Token in your requests
  </Step>

  <Step title="Monitor Costs">
    View usage and costs in your dashboard
  </Step>

  <Step title="Optimize">
    Use insights to reduce API costs
  </Step>
</Steps>


# OpenAI Integration
Source: https://docs.cloudidr.com/guides/llm-ops/integrations/openai

Learn how to integrate LLM Ops with OpenAI's API for cost tracking and monitoring

## Overview

Track costs and monitor usage for OpenAI's API (GPT-4, GPT-3.5, o1) by routing your requests through LLM Ops. This guide shows you how to integrate using Python, JavaScript, or cURL.

<Note>
  **Security Guarantee**: LLM Ops does not store your API keys, request prompts, or response content in the **analytics database**—only metadata needed for cost analytics. The proxy must forward request bodies to OpenAI to complete the call; optional operational logging may exist in your deployment environment.
</Note>

## Quick Start

The OpenAI official SDKs expect `base_url` (or `baseURL`) to include the **`/v1` path segment** (e.g. `https://api.openai.com/v1`). Point that at the LLM Ops API host:

* **Original:** `https://api.openai.com/v1`
* **LLM Ops (SDK `base_url` / `baseURL`):** `https://api.llm-ops.cloudidr.com/v1`
* **cURL:** use the full URL `https://api.llm-ops.cloudidr.com/v1/chat/completions`

This differs from the Anthropic integration: Claude's SDK uses a host **without** `/v1`; OpenAI's SDK uses a base URL **with** `/v1`.

## API Keys

You'll need two credentials:

1. **OpenAI API Key** - Your API key from [platform.openai.com](https://platform.openai.com)
2. **Cloudidr Key** - Your tracking token from the [LLM Ops dashboard](https://llm-ops.cloudidr.com/dashboard) (tokens are typically prefixed with `trk_`)

The marketing site [llmfinops.ai](https://llmfinops.ai) points at the same product; the dashboard URL above is the canonical app host.

Set them as environment variables:

```bash theme={null}
export OPENAI_API_KEY="sk-proj-..."
export CLOUDIDR_KEY="trk_..."
```

***

## Integration Examples

<Tabs>
  <Tab title="Python">
    ### Install SDK

    ```bash theme={null}
    pip install openai
    ```

    ### Basic Example

    ```python theme={null}
    from openai import OpenAI

    # Initialize client with LLM Ops proxy (/v1 is required for the OpenAI SDK)
    client = OpenAI(
        api_key="sk-proj-...",  # Your OpenAI API key
        base_url="https://api.llm-ops.cloudidr.com/v1",
        default_headers={
            "X-Cloudidr-Key": "trk_..."  # Required for cost tracking
        }
    )

    # Make API call - costs are automatically tracked
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": "What is the capital of France?"}
        ]
    )

    print(response.choices[0].message.content)
    ```

    ### With Metadata (Department/Team/Agent Tracking)

    ```python theme={null}
    from openai import OpenAI

    client = OpenAI(
        api_key="sk-proj-...",
        base_url="https://api.llm-ops.cloudidr.com/v1"
    )

    # Track costs by department, team, and agent
    # Use X-Project for team/project (X-Team is supported as a legacy alias)
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful coding assistant."},
            {"role": "user", "content": "Explain async/await in Python"}
        ],
        extra_headers={
            "X-Cloudidr-Key": "trk_...",
            "X-Department": "engineering",
            "X-Team": "backend",
            "X-Agent": "code-tutor"
        }
    )

    print(response.choices[0].message.content)
    ```

    ### Streaming Example

    ```python theme={null}
    from openai import OpenAI

    client = OpenAI(
        api_key="sk-proj-...",
        base_url="https://api.llm-ops.cloudidr.com/v1",
        default_headers={
            "X-Cloudidr-Key": "trk_...",
            "X-Agent": "chat-bot"
        }
    )

    # Streaming is fully supported
    stream = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": "Write a haiku about programming"}
        ],
        stream=True
    )

    for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="", flush=True)
    ```

    ### Function Calling Example

    ```python theme={null}
    from openai import OpenAI

    client = OpenAI(
        api_key="sk-proj-...",
        base_url="https://api.llm-ops.cloudidr.com/v1",
        default_headers={
            "X-Cloudidr-Key": "trk_...",
            "X-Agent": "weather-assistant"
        }
    )

    # Function calling with cost tracking
    tools = [{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }]

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": "What's the weather in San Francisco?"}
        ],
        tools=tools
    )

    print(response.choices[0].message.tool_calls)
    ```
  </Tab>

  <Tab title="JavaScript">
    ### Install SDK

    ```bash theme={null}
    npm install openai
    ```

    ### Basic Example

    ```javascript theme={null}
    import OpenAI from 'openai';

    // Initialize client with LLM Ops proxy (/v1 is required for the OpenAI SDK)
    const client = new OpenAI({
      apiKey: 'sk-proj-...',  // Your OpenAI API key
      baseURL: 'https://api.llm-ops.cloudidr.com/v1',
      defaultHeaders: {
        'X-Cloudidr-Key': 'trk_...'  // Required for cost tracking
      }
    });

    // Make API call - costs are automatically tracked
    const response = await client.chat.completions.create({
      model: 'gpt-4o',
      messages: [
        { role: 'user', content: 'What is the capital of France?' }
      ]
    });

    console.log(response.choices[0].message.content);
    ```

    ### With Metadata (Department/Team/Agent Tracking)

    ```javascript theme={null}
    import OpenAI from 'openai';

    const client = new OpenAI({
      apiKey: 'sk-proj-...',
      baseURL: 'https://api.llm-ops.cloudidr.com/v1'
    });

    const response = await client.chat.completions.create({
      model: 'gpt-4o',
      messages: [
        { role: 'system', content: 'You are a helpful coding assistant.' },
        { role: 'user', content: 'Explain async/await in JavaScript' }
      ]
    }, {
      headers: {
        'X-Cloudidr-Key': 'trk_...',
        'X-Department': 'engineering',
        'X-Team': 'frontend',
        'X-Agent': 'code-tutor'
      }
    });

    console.log(response.choices[0].message.content);
    ```

    ### Streaming (Node.js)

    The snippet below uses `process.stdout`, which is available in Node.js. In the browser, append chunk content to your UI or a string buffer instead.

    ```javascript theme={null}
    import OpenAI from 'openai';

    const client = new OpenAI({
      apiKey: 'sk-proj-...',
      baseURL: 'https://api.llm-ops.cloudidr.com/v1',
      defaultHeaders: {
        'X-Cloudidr-Key': 'trk_...',
        'X-Agent': 'chat-bot'
      }
    });

    const stream = await client.chat.completions.create({
      model: 'gpt-4o',
      messages: [
        { role: 'user', content: 'Write a haiku about programming' }
      ],
      stream: true
    });

    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content;
      if (content) {
        process.stdout.write(content);
      }
    }
    ```

    ### Function Calling Example

    ```javascript theme={null}
    import OpenAI from 'openai';

    const client = new OpenAI({
      apiKey: 'sk-proj-...',
      baseURL: 'https://api.llm-ops.cloudidr.com/v1',
      defaultHeaders: {
        'X-Cloudidr-Key': 'trk_...',
        'X-Agent': 'weather-assistant'
      }
    });

    const tools = [{
      type: 'function',
      function: {
        name: 'get_weather',
        description: 'Get current weather for a location',
        parameters: {
          type: 'object',
          properties: {
            location: { type: 'string' }
          },
          required: ['location']
        }
      }
    }];

    const response = await client.chat.completions.create({
      model: 'gpt-4o',
      messages: [
        { role: 'user', content: "What's the weather in San Francisco?" }
      ],
      tools: tools
    });

    console.log(response.choices[0].message.tool_calls);
    ```
  </Tab>

  <Tab title="cURL">
    ### Basic Example

    ```bash theme={null}
    curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer sk-proj-..." \
      -H "X-Cloudidr-Key: trk_..." \
      -d '{
        "model": "gpt-4o",
        "messages": [
          {
            "role": "user",
            "content": "What is the capital of France?"
          }
        ]
      }'
    ```

    ### With Metadata (Department/Team/Agent Tracking)

    ```bash theme={null}
    curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer sk-proj-..." \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Department: engineering" \
      -H "X-Team: backend" \
      -H "X-Agent: code-tutor" \
      -d '{
        "model": "gpt-4o",
        "messages": [
          {
            "role": "system",
            "content": "You are a helpful coding assistant."
          },
          {
            "role": "user",
            "content": "Explain async/await in JavaScript"
          }
        ]
      }'
    ```

    ### Streaming Example

    ```bash theme={null}
    curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer sk-proj-..." \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Agent: chat-bot" \
      -d '{
        "model": "gpt-4o",
        "messages": [
          {
            "role": "user",
            "content": "Write a haiku about programming"
          }
        ],
        "stream": true
      }'
    ```

    ### Function Calling Example

    ```bash theme={null}
    curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer sk-proj-..." \
      -H "X-Cloudidr-Key: trk_..." \
      -H "X-Agent: weather-assistant" \
      -d '{
        "model": "gpt-4o",
        "messages": [
          {
            "role": "user",
            "content": "What'\''s the weather in San Francisco?"
          }
        ],
        "tools": [
          {
            "type": "function",
            "function": {
              "name": "get_weather",
              "description": "Get current weather for a location",
              "parameters": {
                "type": "object",
                "properties": {
                  "location": { "type": "string" }
                },
                "required": ["location"]
              }
            }
          }
        ]
      }'
    ```
  </Tab>
</Tabs>

***

## Cost Tracking Headers

Add these headers to organize your costs by department, team, or agent:

| Header           | Description                                         | Example                                           |
| ---------------- | --------------------------------------------------- | ------------------------------------------------- |
| `X-Cloudidr-Key` | **Required** - Your Cloudidr tracking token         | `trk_abc123...`                                   |
| `X-Department`   | Track costs by department                           | `engineering`, `sales`, `marketing`, `support`    |
| `X-Project`      | Track costs by project/team (preferred)             | `backend`, `frontend`, `ml`, `data`, `qa`         |
| `X-Team`         | Legacy alias for project/team (same as `X-Project`) | `backend`, `frontend`                             |
| `X-Agent`        | Track costs by agent/application                    | `chatbot`, `summarizer`, `analyzer`, `translator` |

***

## Supported Models

All OpenAI models are supported. See the [Supported Models](/guides/llm-ops/supported-models) page for the complete list of available models and pricing.

***

## What Gets Tracked

LLM Ops automatically captures:

✅ **Token usage** - Prompt, completion, and total tokens\
✅ **Cost** - Real-time cost calculation\
✅ **Latency** - Request duration (TTFT, total time)\
✅ **Model** - Which OpenAI model was used\
✅ **Metadata** - Department, team, agent\
✅ **Errors** - Failed requests and error types\
✅ **Function calls** - Tool/function usage tracking

<Warning>
  **What We DON'T Track:**

  * ❌ Customer API keys
  * ❌ Request content (prompts)
  * ❌ Response content (completions)

  We only persist metadata needed for cost analytics in our application database.
</Warning>

***

## View Your Data

After making requests, view your costs in the [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard):

* **Agent Explorer** - See costs by agent/application
* **Department Breakdown** - Compare department spending
* **Team Analysis** - Track team-level costs
* **Model Comparison** - Compare costs across models
* **Time Series** - Track spending over time
* **Cost Optimization** - Get recommendations for cheaper models

***

## Migration from Direct API

Switching from direct OpenAI API to LLM Ops is a two-line change:

```python theme={null}
# Before
client = OpenAI(api_key="sk-proj-...")

# After - add base_url (include /v1) and X-Cloudidr-Key header
client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1",      # ← Add this
    default_headers={"X-Cloudidr-Key": "trk_..."}  # ← Add this
)
```

Everything else stays the same - no code changes needed!

***

## Cost Optimization Tips

<AccordionGroup>
  <Accordion title="Compare Model Costs in Dashboard" icon="chart-bar">
    Use the LLM Ops dashboard to identify which agents can switch to cheaper models:

    * Track cost per request by model
    * Compare quality vs. cost trade-offs
    * Identify high-volume, low-complexity tasks

    Perfect candidates for model switching appear in the Agent Explorer.
  </Accordion>

  <Accordion title="Monitor Function Call Costs" icon="function">
    Function calling adds token overhead:

    * Track function call frequency per agent
    * Identify redundant or unnecessary calls
    * Optimize function descriptions to reduce tokens

    LLM Ops tracks tool usage separately from chat completion.
  </Accordion>

  <Accordion title="Enable Prompt Caching" icon="database">
    OpenAI's prompt caching can reduce costs for repeated system prompts:

    * Track cache hit rates in dashboard
    * Identify agents with repeated prompts
    * Structure prompts for maximum cache benefit

    LLM Ops shows cache savings in cost breakdowns when applicable.
  </Accordion>
</AccordionGroup>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Requests not being tracked" icon="exclamation-triangle">
    **Check these common issues:**

    * ✅ For **OpenAI SDKs**, set `base_url` / `baseURL` to `https://api.llm-ops.cloudidr.com/v1` (OpenAI expects the `/v1` prefix).
    * ✅ For **cURL**, call `https://api.llm-ops.cloudidr.com/v1/chat/completions`.
    * ✅ Confirm the header name is **`X-Cloudidr-Key`** (not `X-Cloudidr-Token`) on every request.
    * ✅ Check that your OpenAI API key is valid.
  </Accordion>

  <Accordion title="Authentication errors" icon="key">
    **Two separate keys are needed:**

    * Your OpenAI API key (for GPT access)
    * Your Cloudidr tracking token (for cost tracking)

    Make sure both are set correctly and not swapped.
  </Accordion>

  <Accordion title="Cost data not appearing" icon="chart-line">
    **Wait a few moments:**

    * Cost data may take 10-30 seconds to appear in dashboard
    * Check the correct time range in dashboard filters
    * Verify requests are returning 200 OK status
  </Accordion>
</AccordionGroup>

***

## Next Steps

<CardGroup>
  <Card title="View Dashboard" icon="chart-line" href="https://llm-ops.cloudidr.com/dashboard">
    See your OpenAI API costs in real-time
  </Card>

  <Card title="Supported Models" icon="list" href="/guides/llm-ops/supported-models">
    View all supported OpenAI models
  </Card>

  <Card title="Anthropic Integration" icon="sparkles" href="/guides/llm-ops/integrations/anthropic">
    Add cost tracking for Claude models
  </Card>

  <Card title="Set Budgets" icon="wallet" href="guides/llm-ops/budget-guard">
    Configure spending alerts and limits
  </Card>
</CardGroup>


# LLM Cost Savings
Source: https://docs.cloudidr.com/guides/llm-ops/llm-optimization

Organization-wide optimizer settings, savings reporting, per-agent overrides, and how non-tagged traffic appears in metrics

## Overview

| Area                        | Where in the app                                                                  | What it does                                                                                                                                                        |
| --------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **LLM Optimizer Settings**  | **Settings** → **LLM Optimizer Settings**                                         | **Defaults**: turn optimization on/off, choose provider and routing strategy, safety fallback behavior, and whether to optimize requests **without** agent tagging. |
| **LLM Cost Optimizer**      | **Actions** → **LLM Cost Optimizer**                                              | **Reporting**: summary cards, filters, and a breakdown table (by department / project / agent) including a **Non-Tagged** row when applicable.                      |
| **Per-agent configuration** | Same page (**LLM Cost Optimizer**), section **Advanced: Per-Agent Configuration** | **Overrides**: when global optimization is on, each agent can be included or excluded without changing org defaults.                                                |
|                             |                                                                                   |                                                                                                                                                                     |

***

## 1. LLM Optimizer Settings (organization defaults)

<Frame>
  <img alt="Image" />
</Frame>

### LLM Model Optimization (master toggle)

When **on**, Cloudidr may route eligible API traffic to cheaper models according to the strategies below. When **off**, requests use the model the client asked for (no automatic substitution).

### Provider strategy

Controls **how far** routing may move from the originally requested provider:

* **Intra Provider** — Stay within the same upstream provider (for example, a more expensive OpenAI model → a cheaper OpenAI model). Typical savings are lower than cross-provider options but preserve provider-specific behavior.
* **Flexible - Maximum Savings** — May route to **Cloudidr-hosted** open models for higher potential savings. This path can require **prepaid credits**; if the balance is zero, the UI may disable or warn until credits are added.
* **Optimize Specific Providers Only** — Optimization runs only when the request targets one of the **selected** providers (OpenAI, Anthropic, Google, AWS Bedrock). Use the checkboxes that appear when this option is selected.

### Domain Plugins

These plugins enhance model routing ability based on semantics associated with the domain. Available plugins are for banking/financials, healthcare, legal, and engineering.

### Routing strategy

* **Smart (Intelligent pattern matching)** — Uses complexity-style scoring so simple prompts can be sent to very cheap models while harder tasks keep stronger models.
* **Adaptive (AI-powered learning)** — Shown as **contact us** / not selectable in the current UI; reserved for future or custom rollout.

### Safety controls (if optimization fails)

These apply to **all** optimization attempts (tagged and non-tagged):

* **Fail request (strict mode)** — Return an error if a substitute model cannot be used as planned.
* **Use original model (safe fallback)** — Fall back to the **original** model the client requested so the request still completes.
* **Try cheapest alternative** — Shown as **contact us** / not selectable in the current UI.

### Non-tagged requests

* **Yes - Optimize all requests** — Optimization may run even when the client does **not** send tagging headers. Those requests use these **global** defaults (unless a per-agent rule applies—tagged traffic can still use agent-specific settings when present).
* **No - Only optimize tagged requests** — Requests **without** an agent identifier skip optimization and pass through unchanged.

> Enabling optimization typically requires a **payment method on file** or **positive org prepaid credits** (the product bills a percentage of verified savings—see the in-app banner and subscription screens). If optimization is off and the org has no funding source, the UI explains that a card or credits are needed before turning optimization on.

### Recency protection

**Recency protection** is an optional (default **on**) layer in Cloudidr’s routing pipeline. If the user’s prompt looks like it needs **current world knowledge** (news, live markets, who holds a role today, “as of” dates, etc.), Cloudidr **does not substitute a cheaper model** and keeps traffic on the **baseline model** the customer selected.

**Why it exists:** Cheaper routed models often have **older or different training cutoffs**. For “who won the last election?” or “what is Apple’s stock price?” routing to a smaller model can increase **factual wrongness** even when the prompt is *simple* in complexity terms. Recency protection trades possible cost savings for **lower risk of stale answers** on those prompts.

**What it is not:** It does not call an external search, web browse, or “grounding” API. It is **phrase-based detection** on the prompt text only, then a **skip-routing** decision.

***

## 2. LLM Cost Optimizer (savings and breakdown)

<Frame>
  <img alt="Image" />
</Frame>

### Top summary cards

Typical cards include:

* **Requests Optimized This Month** — Count and share of traffic that used an optimized route in the **current calendar month** (definitions are shown on the page).
* **Savings This Month** — Dollar savings and savings rate for the current month, often with a comparison to the prior month.
* **Savings Last Month** — Prior month totals for quick comparison.
* **All Time Savings** — Cumulative verified savings since tracking began for the org.

These roll up **all** included traffic in scope for the optimizer, including rows that have no agent tag (see below).

### Savings Details (filters and aggregates)

Use **Department**, **Project**, and **Agent** filters and the **time range** (Today, 7 / 30 / 90 days, year, custom) to focus the view.

The aggregate line (**Total Requests**, **Optimized**, **Savings**, **Savings %**) reflects the **filtered** period and dimensions. Percentages are computed from **optimization-enabled** traffic as labeled on the page.

### Agent breakdown table

Each row is one **agent dimension** (department / project / agent). Metrics include total requests, how many were optimized, original vs actual cost, savings, and savings rate.

**Non-tagged requests in savings**

When **Yes - Optimize all requests** is enabled and the proxy **does** apply optimization to traffic without `X-Agent` (and related) tags, those requests are stored **without** an agent name in usage data and reported "Non-tagged".

***

## 3. Advanced: Per-Agent Configuration

<Frame>
  <img alt="Image" />
</Frame>

Optional table: **Department**, **Project**, **Agent**, and **Enable Optimization** per row.

* When **global** LLM Model Optimization is **off**, **no** agent traffic is optimized (all requests use the requested model).
* When global optimization is **on**, agents are **included by default**; turn **off** for specific agents to **exclude** them from optimization while leaving others unchanged.

> if the agent is **disabled** in **Advanced: Per-Agent Configuration**, those requests are **not** optimized, **regardless** of **Yes – Optimize all requests**.

***

## Who can change settings

**Saving organization defaults** (toggle, strategies, safety, non-tagged behavior) requires an **organization owner** (`super_user`), same as other org-wide billing-related settings. Team members can open the pages; only owners can persist changes where the API enforces `get_super_user`.

***

## Quick reference: non-tagged traffic

| Setting                                | Behavior                                                                                                                                                              |
| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Yes - Optimize all requests**        | Non-tagged requests can be optimized using org defaults; successful optimizations contribute savings and appear under a **Non-Tagged** row (and in top-level totals). |
| **No - Only optimize tagged requests** | Send **`X-Agent`** (and optional department/project headers) if you want a row per agent; non-tagged traffic is not optimized by default.                             |


# Using Promo Codes
Source: https://docs.cloudidr.com/guides/llm-ops/promo-codes



Please follow these steps to apply promo codes

1. In the left side panel, go to **Settings** and select **Subscription**

<Frame>
  <img alt="Screenshot2026 02 19at13 52 44 1" />
</Frame>

2. Select the desired plan like "Growth" and click **Upgrade**

   <Frame>
     <img alt="Screenshot2026 02 19at13 56 28" />
   </Frame>
3. Select between **Monthly** or **Yearly** payment tab. It will update the pricing amount due and then click \*\*Continue to Checkout. \*\*

   NOTE: If you hava promotion code, you will select YEARLY and then enter the promotion code in the next step before final checkout,

   <Frame>
     <img alt="Screenshot2026 02 19at13 58 22" />
   </Frame>
4. \[OPTIONAL] Enter**Promotion**  code if you have one

   <Frame>
     <img alt="Screenshot2026 02 19at14 02 27" />
   </Frame>

   5. After entering promotion code, the **Total due** will be shown.

      <Frame>
        <img alt="Screenshot2026 02 19at14 03 31" />
      </Frame>
5. Enter credit card. You can uncheck the \*\*Save my information \*\*if you so desire. You will need to enter the credit card information (as this is required by our payment processor Stripe). After this it will take you to the subscription management page, where you can see your updated plan.

<Frame>
  <img alt="Screenshot2026 02 19at14 08 35" />
</Frame>


# Roles & Permissions
Source: https://docs.cloudidr.com/guides/llm-ops/roles

What org owners can do compared to team members.

This document summarizes what **org owners** (`super_user`) can do that **non-owners** (typically `member`) cannot, and what everyone can do.

***

## Owner-only (not available to non-owners)

| Capability                                                                   | Notes                                                                                                                                                                                                    | Related guide                                         |
| ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| **Invite and manage [team members](/guides/llm-ops/team-mgmt)**              | Add/remove users, roles, invitations                                                                                                                                                                     | [Team members](guides/llm-ops/team-mgmt)              |
| **Billing and [subscription](/guides/llm-ops/subscription)**                 | Plan changes, payment method, invoices                                                                                                                                                                   | [Subscription](guides/llm-ops/subscription)           |
| **Revoke any org [tracking key](/guides/llm-ops/api-keys)**                  | Owners can revoke **any** member’s API tracking key from the Company keys view; members can only revoke their own                                                                                        | [Team members](guides/llm-ops/team-mgmt) (org access) |
| **Overall org monthly [budget](/guides/llm-ops/budget-guard)**               | Top-level cap in Alert / budget settings (owner configuration)                                                                                                                                           | [Budget Guard](guides/llm-ops/budget-guard)           |
| **Org-wide [LLM Cost Optimizer](/guides/llm-ops/llm-optimization) defaults** | Global optimizer defaults (e.g. enable/disable at org level) via Settings; separate from per-agent toggles                                                                                               | Product settings (LLM Optimizer Settings)             |
| **Edit all agents in Company view**                                          | On Budget Guard and **LLM Cost Optimizer**, **Company** view is **read-only** for non-owners; they must use **Personal** view to change budgets or per-agent optimization for agents tied to their usage |                                                       |

***

## Non-owners (and owners) — shared capabilities

| Capability                     | Notes                                                                                                                                                                                                    |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Company / Personal view**    | Toggle on LLM Usage, Agent Explorer, Budget Guard, LLM Cost Optimizer, Managed Inference, and main Dashboard (where applicable). **Company** = org-wide analytics and lists where the backend allows it. |
| **Personal view — own agents** | Set monthly budgets, enforcement, agent alerts, and enable/disable optimization for agents that appear in **their** tracked usage (same scope as Budget Guard lists for that user).                      |
| **View org-wide usage**        | In **Company** view, non-owners can see org-wide metrics and agent lists where the product exposes read-only company data.                                                                               |
| **Own API keys**               | Create and revoke **their own** tracking keys.                                                                                                                                                           |

***

## Quick reference

* **Owner:** full org administration — team, billing, all keys, org-wide budgets/defaults, and can edit every agent in Company view on Budget Guard and Cost Optimizer.
* **Non-owner:** full visibility in Company view where offered; edits for budgets and optimizer are limited to **Personal** view and **agents in their tracked usage**; cannot manage team billing or revoke others’ keys.

***


# Enterprise SSO
Source: https://docs.cloudidr.com/guides/llm-ops/sso



Single Sign-On (SSO) lets your entire team sign in to LLM Ops using your company's existing identity provider (Okta, Microsoft Entra ID, Google Workspace, or any SAML 2.0 / OIDC provider) — no separate passwords required.

SSO is available on the **Enterprise plan**.

***

## How SSO works

When SSO is enabled for your organization:

1. A team member goes to [Sign in](https://docs.cloudidr.com/llm-ops.cloudidr.com/signin) page and enters their work email in the **Enterprise** **SSO** section of the sign-in box.
2. They are redirected to your company's login page (Okta, Azure, Google, etc.).
3. They authenticate with their existing company credentials.
4. They are automatically signed in to LLM Ops and land on the dashboard.
5. If it is their first time signing in, their account is created automatically — no invitation needed.

Your team never needs to remember a separate LLM Ops password. Access is controlled entirely through your identity provider.

***

## First time setup for the organizational  owner

Follow these steps once to enable SSO for your organization.

### Step 1 — Sign in

Signup with your company email on the[Sign in](https://docs.cloudidr.com/llm-ops.cloudidr.com/signin)

Go to **Settings → Subscription  → Billing** inside the app and send us email by clicking through **Upgrade - Contact Sales** button. SSO requires the Enterprise plan.

### Step 2 — Provide upgrade information for nterprise Subscription

Email [sales@cloudidr.com](mailto:sales@cloudidr.com) with:

* Your email, phone number and name
* Your organization name inside LLM Ops
* Your company email domain (e.g. `acme.com`)
* Your identity provider (Okta, Microsoft Entra ID, Google Workspace, or other)

`Our sales executive will contact you to determine the enteprise pricing based on your usage, users and deployment needs.`

After that, we will provision your SSO connection in WorkOS (our SSO provider) and send you an **Admin Portal link** .

### Step 3 — Configure your identity provider

Open the Admin Portal link we send you. It will guide you through connecting your identity provider:

* **Okta** — paste the metadata URL or upload the metadata XML
* **Microsoft Entra ID (Azure AD)** — enter the tenant ID and client credentials
* **Google Workspace** — follow the SAML app setup instructions

The Admin Portal is self-service. You do not need to share any credentials with us.

### Step 4 — Test with one user

Before rolling out to the whole team:

1. Go to [Sign in](llm-ops.cloudidr.com/login)
2. In the **Enterprise SSO** section, enter your work email and click **Continue with SSO**
3. Complete the login flow through your identity provider
4. Confirm you land on the LLM Ops dashboard

### Step 5 — Roll out to your team

Once the test succeeds, share the login URL with your team. There is nothing to install or configure on their end — they just enter their work email and click **Continue with SSO**.

> **Note:** Existing team members who previously used email/password login can continue using that method. SSO and password login work side by side.

***

## What your team members see

On the login page, the **Enterprise SSO** section sits below the main sign-in form:

```text theme={null}
─────────────── Enterprise SSO ───────────────
        For Enterprise plan customers only

  [ work@yourcompany.com ]  [ Continue with SSO ]
```

They enter their work email and are redirected to your identity provider. After authenticating, they are brought straight to the dashboard.

***

## Frequently asked questions

**Do my team members need to be invited first?** No. Any user who authenticates through your identity provider is automatically provisioned in LLM Ops as a member of your organization on their first login.

**What role do new SSO users get?** New users are assigned the **Member** role by default. Admins can adjust roles inside the app under **Settings → Team**.

**Can I use both SSO and email/password?** Yes. SSO and email/password logins work side by side. You can migrate your team gradually.

**What happens if our identity provider is down?** Team members can still sign in using their email and password if they have one set. We recommend that at least one admin retains a password as a break-glass option.

**Can I restrict login to SSO only?** Contact [sales@cloudidr.com](mailto:sales@cloudidr.com) to discuss enforced SSO (password login disabled for your org).

***

## Troubleshooting

### "SSO is not set up for your organization yet"

SSO has not been activated for your domain. Complete Steps 2–3 in the setup sequence above, or contact [sales@cloudidr.com](mailto:sales@cloudidr.com).

### "Your organization is not on the Enterprise plan"

Your account is on a lower plan. Upgrade to Enterprise under **Settings → Subscription**, then contact us to activate SSO.

### "Please use your organization email address, not a personal one"

You entered a personal email address (e.g. gmail.com, yahoo.com). Use your company work email instead.

### Redirected back to login with an error after authenticating

This usually means the identity provider configuration is incomplete. Check the following:

| Check                  | What to verify                                                               |
| ---------------------- | ---------------------------------------------------------------------------- |
| Metadata / certificate | Ensure the IdP metadata is uploaded and not expired in the Admin Portal      |
| Attribute mapping      | Email, first name, and last name must be mapped in your IdP SAML/OIDC config |
| Assigned users         | The user must be assigned to the LLM Ops app in your IdP                     |
| Browser cookies        | Try an incognito window to rule out stale session cookies                    |

If the issue persists, copy the error message from the URL bar (`?sso_error=...`) and send it to [support@cloudidr.com](mailto:support@cloudidr.com).

### Login loop (redirected back to IdP repeatedly)

Clear your browser cookies for both `llm-ops.cloudidr.com` and your IdP domain, then try again in an incognito window.

### New user not appearing in the team list after SSO login

The user's account is created on their first successful login. Ask them to complete the login flow and then refresh **Settings → Team**.

***

## Security notes

* LLM Ops never sees or stores your identity provider credentials.
* SSO tokens are short-lived and single-use (handled by WorkOS).
* Your company remains in full control of access: removing a user from your IdP immediately prevents them from signing in via SSO.
* All LLM Ops sessions use the same encrypted JWT tokens regardless of login method.

***

## Contact

| Purpose                         | Contact                                             |
| ------------------------------- | --------------------------------------------------- |
| Activating SSO / plan upgrade   | [sales@cloudidr.com](mailto:sales@cloudidr.com)     |
| Technical issues during setup   | [support@cloudidr.com](mailto:support@cloudidr.com) |
| Identity provider configuration | Use the Admin Portal link we provide                |


# Subscriptions
Source: https://docs.cloudidr.com/guides/llm-ops/subscription

We offer Subscription plans for LLM visibility, budget control and routing. Free tier to advanced paid tiers in future. For managed inference usage we offer purchasable credits starting at $5.

### Starter - Free Forever

**Perfect for:** Small teams getting started with AI cost tracking

* Track up to \$5K/mo AI spend
* 3 team members (1 owner + 2)
* All core features
* Budget Control: Up to 3 agents
* LLM Cost Optimizer: Inferred routing
* 90-day data retention
* CSV export
* Community support

**\$0/month** - No credit card required

***

### Growth - Most Popular

**Perfect for:** Growing teams scaling AI usage

* Track up to \$30K/mo AI spend
* 10 team members
* Everything in Starter, PLUS:
* Budget Control: Up to 20 agents
* LLM Cost Optimizer: All provider strategies (Flexible, Fixed)
* 1-year data retention
* Slack integration
* Email support (48hr)

**\$49/month**

***

### Scale - Best Value

**Perfect for:** Teams optimizing serious AI spend

* Track up to \$100K/mo AI spend
* Unlimited team members
* Everything in Growth, PLUS:
* Budget Control: Unlimited agents
* Adaptive AI Routing (ML-powered)
* Forecasting
* 3-year data retention
* API access
* SSO (SAML)
* Priority support (24hr)
* 99.9% SLA

**\$299/month**

**Typical savings: 40K/month**

***

### Enterprise

**Perfect for:** Mission-critical AI infrastructure

* Unlimited tracked spend
* Everything in Scale, PLUS:
* Custom ML routing logic
* Dedicated Customer Success Manager
* 99.99% SLA (99.999% available)
* Custom integrations
* Unlimited data retention
* White-glove onboarding

**Custom Pricing** - Contact Sales


# Supported AI Models
Source: https://docs.cloudidr.com/guides/llm-ops/supported-models

Complete list of AI models supported by Cloudidr LLM Ops with current pricing. 

## Overview

These are the models supported by Cloudidr LLM Ops to track API costs. The model pricing is from the providers which we use to calculate your spend.

> Up to date supported models, providers and pricing can be found in the LLM Ops left side panel **Starting Guide** → **Model Pricing** tab

<Note>
  **Last Updated:** January 10, 2026

  Model pricing is subject to change by the providers. We update our pricing regularly to ensure accurate cost tracking.
</Note>

<Warning>
  **Model Not Listed?**

  If your model is not in this list, please contact us at [support@cloudidr.com](mailto:support@cloudidr.com) and we'll add support for it.
</Warning>

***

## Pricing Tables

<Tabs>
  <Tab title="Anthropic">
    ## Anthropic Claude Models

    All pricing is per 1 million tokens.

    | Model                      | Input Cost | Output Cost |
    | -------------------------- | ---------- | ----------- |
    | **Claude Opus 4.5**        |            |             |
    | claude-opus-4-5-20251101   | \$5.00     | \$25.00     |
    | **Claude Opus 4.1**        |            |             |
    | claude-opus-4-1-20250805   | \$15.00    | \$75.00     |
    | **Claude Opus 4**          |            |             |
    | claude-opus-4-20250514     | \$15.00    | \$75.00     |
    | **Claude Sonnet 4.5**      |            |             |
    | claude-sonnet-4-5-20250929 | \$3.00     | \$15.00     |
    | **Claude Sonnet 4**        |            |             |
    | claude-sonnet-4-20250514   | \$3.00     | \$15.00     |
    | **Claude Haiku 4.5**       |            |             |
    | claude-haiku-4-5-20251001  | \$1.00     | \$5.00      |
    | **Claude 3.5 Haiku**       |            |             |
    | claude-3-5-haiku-20241022  | \$0.80     | \$4.00      |
    | **Claude 3 Haiku**         |            |             |
    | claude-3-haiku-20240307    | \$0.25     | \$1.25      |

    <Info>
      **Model Recommendations:**

      * **Opus** - Most capable, best for complex reasoning
      * **Sonnet** - Balanced performance and cost
      * **Haiku** - Fastest and most affordable
    </Info>

    ### Integration Guide

    See the [Anthropic Integration](/guides/llm-ops/integrations/anthropic) page to start tracking costs.
  </Tab>

  <Tab title="OpenAI">
    ## OpenAI Models

    All pricing is per 1 million tokens unless otherwise noted.

    ### GPT-5 Family

    | Model               | Input Cost | Output Cost |
    | ------------------- | ---------- | ----------- |
    | gpt-5.2             | \$1.75     | \$14.00     |
    | gpt-5.2-chat-latest | \$1.75     | \$14.00     |
    | gpt-5.2-pro         | \$21.00    | \$168.00    |
    | gpt-5.1             | \$1.25     | \$10.00     |
    | gpt-5.1-chat-latest | \$1.25     | \$10.00     |
    | gpt-5.1-codex-max   | \$1.25     | \$10.00     |
    | gpt-5.1-codex       | \$1.25     | \$10.00     |
    | gpt-5.1-codex-mini  | \$0.25     | \$2.00      |
    | gpt-5               | \$1.25     | \$10.00     |
    | gpt-5-chat-latest   | \$1.25     | \$10.00     |
    | gpt-5-codex         | \$1.25     | \$10.00     |
    | gpt-5-pro           | \$15.00    | \$120.00    |
    | gpt-5-mini          | \$0.25     | \$2.00      |
    | gpt-5-nano          | \$0.05     | \$0.40      |
    | gpt-5-search-api    | \$1.25     | \$10.00     |

    ### GPT-4.1 Family

    | Model        | Input Cost | Output Cost |
    | ------------ | ---------- | ----------- |
    | gpt-4.1      | \$2.00     | \$8.00      |
    | gpt-4.1-mini | \$0.40     | \$1.60      |
    | gpt-4.1-nano | \$0.10     | \$0.40      |

    ### GPT-4o Family

    | Model                      | Input Cost | Output Cost |
    | -------------------------- | ---------- | ----------- |
    | gpt-4o                     | \$2.50     | \$10.00     |
    | gpt-4o-2024-05-13          | \$5.00     | \$15.00     |
    | gpt-4o-mini                | \$0.15     | \$0.60      |
    | gpt-4o-mini-2024-07-18     | \$0.15     | \$0.60      |
    | gpt-4o-search-preview      | \$2.50     | \$10.00     |
    | gpt-4o-mini-search-preview | \$0.15     | \$0.60      |

    ### Realtime & Audio Models

    | Model                        | Input Cost | Output Cost |
    | ---------------------------- | ---------- | ----------- |
    | gpt-realtime                 | \$4.00     | \$16.00     |
    | gpt-realtime-mini            | \$0.60     | \$2.40      |
    | gpt-4o-realtime-preview      | \$5.00     | \$20.00     |
    | gpt-4o-mini-realtime-preview | \$0.60     | \$2.40      |
    | gpt-audio                    | \$2.50     | \$10.00     |
    | gpt-audio-mini               | \$0.60     | \$2.40      |
    | gpt-4o-audio-preview         | \$2.50     | \$10.00     |
    | gpt-4o-mini-audio-preview    | \$0.15     | \$0.60      |

    ### Image Generation Models (Per-Image Pricing)

    | Model    | Resolution | Quality  | Price Per Image |
    | -------- | ---------- | -------- | --------------- |
    | dall-e-3 | 1024×1024  | Standard | \$0.040         |
    | dall-e-3 | 1024×1792  | Standard | \$0.080         |
    | dall-e-3 | 1792×1024  | Standard | \$0.080         |
    | dall-e-3 | 1024×1024  | HD       | \$0.080         |
    | dall-e-3 | 1024×1792  | HD       | \$0.120         |
    | dall-e-3 | 1792×1024  | HD       | \$0.120         |

    ### Audio Transcription Models (Per-Second Pricing)

    | Model     | Price Per Second | Notes                     |
    | --------- | ---------------- | ------------------------- |
    | whisper-1 | \$0.0001         | \$0.006/min transcription |

    ### Text-to-Speech Models (Per-Character Pricing)

    | Model    | Price Per 1M Characters | Notes            |
    | -------- | ----------------------- | ---------------- |
    | tts-1    | \$15.00                 | Standard quality |
    | tts-1-hd | \$30.00                 | High definition  |

    ### o-Series (Reasoning Models)

    | Model                 | Input Cost | Output Cost |
    | --------------------- | ---------- | ----------- |
    | o1                    | \$15.00    | \$60.00     |
    | o1-pro                | \$150.00   | \$600.00    |
    | o1-mini               | \$1.10     | \$4.40      |
    | o3                    | \$2.00     | \$8.00      |
    | o3-pro                | \$20.00    | \$80.00     |
    | o3-mini               | \$1.10     | \$4.40      |
    | o3-deep-research      | \$10.00    | \$40.00     |
    | o4-mini               | \$1.10     | \$4.40      |
    | o4-mini-deep-research | \$2.00     | \$8.00      |

    ### GPT-4 Legacy

    | Model               | Input Cost | Output Cost |
    | ------------------- | ---------- | ----------- |
    | gpt-4               | \$30.00    | \$60.00     |
    | gpt-4-turbo         | \$10.00    | \$30.00     |
    | gpt-4-turbo-preview | \$10.00    | \$30.00     |

    ### GPT-3.5 Family

    | Model             | Input Cost | Output Cost |
    | ----------------- | ---------- | ----------- |
    | gpt-3.5-turbo     | \$0.50     | \$1.50      |
    | gpt-3.5-turbo-16k | \$3.00     | \$4.00      |

    <Info>
      **Model Recommendations:**

      * **gpt-4o** - Best balance of capability and speed
      * **gpt-4o-mini** - Most cost-effective for simple tasks
      * **o1/o3** - Advanced reasoning for complex problems
      * **dall-e-3** - High-quality image generation
      * **whisper-1** - Audio transcription at \$0.006/minute
      * **tts-1** - Natural text-to-speech
    </Info>

    ### Integration Guide

    See the [OpenAI Integration](/guides/llm-ops/integrations/openai) page to start tracking costs.
  </Tab>

  <Tab title="Google">
    ## Google Gemini Models

    All pricing is per 1 million tokens unless otherwise noted.

    ### Gemini 3 Series

    | Model                      | Input Cost | Output Cost | Special Rates          |
    | -------------------------- | ---------- | ----------- | ---------------------- |
    | gemini-3-pro-preview       | \$2.00     | \$12.00     |                        |
    | gemini-3-pro-image-preview | \$2.00     | \$12.00     | Image Output: \$120.00 |
    | gemini-3-flash-preview     | \$0.50     | \$3.00      | Audio Input: \$1.00    |

    ### Gemini 2.5 Series

    | Model                                         | Input Cost | Output Cost | Special Rates                            |
    | --------------------------------------------- | ---------- | ----------- | ---------------------------------------- |
    | **Pro Models**                                |            |             |                                          |
    | gemini-2.5-pro                                | \$1.25     | \$10.00     |                                          |
    | gemini-2.5-pro-preview-tts                    | \$1.00     | \$20.00     | TTS: Audio output                        |
    | **Flash Models**                              |            |             |                                          |
    | gemini-2.5-flash                              | \$0.30     | \$2.50      | Audio Input: \$1.00                      |
    | gemini-2.5-flash-preview-09-2025              | \$0.30     | \$2.50      | Audio Input: \$1.00                      |
    | gemini-2.5-flash-preview-tts                  | \$0.50     | \$10.00     | TTS: Audio output                        |
    | gemini-2.5-flash-image                        | \$0.30     | \$2.50      | Image Output: \$30.00                    |
    | **Flash-Lite Models**                         |            |             |                                          |
    | gemini-2.5-flash-lite                         | \$0.10     | \$0.40      | Audio Input: \$0.30                      |
    | gemini-2.5-flash-lite-preview-09-2025         | \$0.10     | \$0.40      | Audio Input: \$0.30                      |
    | **Specialized Models**                        |            |             |                                          |
    | gemini-2.5-computer-use-preview-10-2025       | \$1.25     | \$10.00     |                                          |
    | gemini-2.5-flash-native-audio-preview-12-2025 | \$0.50     | \$2.00      | Audio In: \$3.00<br />Audio Out: \$12.00 |

    ### Gemini 2.0 Series

    | Model                 | Input Cost | Output Cost | Special Rates       |
    | --------------------- | ---------- | ----------- | ------------------- |
    | gemini-2.0-flash      | \$0.10     | \$0.40      | Audio Input: \$0.70 |
    | gemini-2.0-flash-lite | \$0.075    | \$0.30      |                     |

    ### Latest Aliases (Dynamic)

    | Model                    | Input Cost | Output Cost | Maps To               |
    | ------------------------ | ---------- | ----------- | --------------------- |
    | gemini-flash-latest      | \$0.30     | \$2.50      | gemini-2.5-flash      |
    | gemini-pro-latest        | \$1.25     | \$10.00     | gemini-2.5-pro        |
    | gemini-flash-lite-latest | \$0.10     | \$0.40      | gemini-2.5-flash-lite |

    ### Gemini 1.5 Series (Legacy)

    | Model            | Input Cost | Output Cost |
    | ---------------- | ---------- | ----------- |
    | gemini-1.5-pro   | \$1.25     | \$5.00      |
    | gemini-1.5-flash | \$0.075    | \$0.30      |

    ### Other Legacy Models

    | Model        | Input Cost | Output Cost |
    | ------------ | ---------- | ----------- |
    | gemini-pro   | \$0.50     | \$1.50      |
    | gemini-flash | \$0.075    | \$0.30      |
    | palm-2       | \$0.50     | \$1.50      |

    ### Image Generation Models (Per-Image Pricing)

    | Model                         | Price Per Image | Notes               |
    | ----------------------------- | --------------- | ------------------- |
    | imagen-4.0-fast-generate-001  | \$0.02          | Fast generation     |
    | imagen-4.0-generate-001       | \$0.04          | Standard generation |
    | imagen-4.0-ultra-generate-001 | \$0.06          | Ultra quality       |

    ### Video Generation Models (Per-Second Pricing)

    | Model                         | Price Per Second | Notes                     |
    | ----------------------------- | ---------------- | ------------------------- |
    | veo-3.1-fast-generate-preview | \$0.15           | Fast generation (Preview) |
    | veo-3.1-generate-preview      | \$0.40           | Standard (Preview)        |
    | veo-3.0-fast-generate-001     | \$0.15           | Fast generation (Stable)  |
    | veo-3.0-generate-001          | \$0.40           | Standard (Stable)         |

    <Info>
      **Model Recommendations:**

      * **gemini-2.5-pro** - Most capable for complex tasks
      * **gemini-2.5-flash** - Best cost/performance balance
      * **gemini-2.5-flash-lite** - Most affordable option
      * **gemini-2.5-flash-image** - Native image generation
      * **imagen-4.0-fast** - Fast image generation at \$0.02/image
      * **veo-3.0-fast** - Fast video generation at \$0.15/second
    </Info>

    <Note>
      **Special Pricing Notes:**

      * **TTS Models:** Output is audio tokens, not text
      * **Audio Input:** Premium pricing for audio/video multimodal input
      * **Image Generation (Imagen):** Charged per image generated, not per token
      * **Video Generation (Veo):** Charged per second of video generated
      * **gemini-2.5-flash-image:**
        * Text input: \$0.30 per 1M tokens
        * Text output: \$2.50 per 1M tokens
        * Image output: \$30.00 per 1M tokens (equivalent to \$0.039 per image)
    </Note>

    ### Integration Guide

    See the [Gemini Integration](/guides/llm-ops/integrations/gemini) page to start tracking costs.
  </Tab>
</Tabs>

***

## Cost Comparison

<AccordionGroup>
  <Accordion title="Most Affordable Models (Under $1/M tokens)" icon="dollar-sign">
    **Best for high-volume, simple tasks:**

    * **GPT-5 Nano:** $0.05 input / $0.40 output
    * **Gemini 2.0 Flash Lite:** $0.075 input / $0.30 output
    * **Gemini 1.5 Flash:** $0.075 input / $0.30 output
    * **GPT-4.1 Nano:** $0.10 input / $0.40 output
    * **Gemini 2.5 Flash Lite:** $0.10 input / $0.40 output
    * **GPT-4o Mini:** $0.15 input / $0.60 output
    * **Claude 3 Haiku:** $0.25 input / $1.25 output
    * **Claude 3.5 Haiku:** $0.80 input / $4.00 output

    Perfect for: Classification, extraction, simple Q\&A, high-throughput tasks
  </Accordion>

  <Accordion title="Mid-Range Models ($1-$5/M tokens)" icon="balance-scale">
    **Balanced performance and cost:**

    * **Claude Haiku 4.5:** $1.00 input / $5.00 output
    * **GPT-4.1:** $2.00 input / $8.00 output
    * **GPT-4o:** $2.50 input / $10.00 output
    * **Claude Sonnet 4.5:** $3.00 input / $15.00 output
    * **Claude Opus 4.5:** $5.00 input / $25.00 output

    Perfect for: Customer support, content generation, code assistance
  </Accordion>

  <Accordion title="Premium Models ($10+/M tokens)" icon="crown">
    **Advanced reasoning and complex tasks:**

    * **Claude Opus 4:** $15.00 input / $75.00 output
    * **o1:** $15.00 input / $60.00 output
    * **o3:** $2.00 input / $8.00 output
    * **o1-pro:** $150.00 input / $600.00 output
    * **GPT-5 Pro:** $15.00 input / $120.00 output
    * **GPT-4 (legacy):** $30.00 input / $60.00 output

    Perfect for: Complex reasoning, research, code generation, expert analysis
  </Accordion>

  <Accordion title="Image Generation (Per-Image Pricing)" icon="image">
    **AI image generation models:**

    * **Imagen 4 Fast:** \$0.02 per image
    * **DALL-E 3 Standard (1024×1024):** \$0.040 per image
    * **Imagen 4 Standard:** \$0.04 per image
    * **gemini-2.5-flash-image:** \$0.039 per image (+ token costs)
    * **Imagen 4 Ultra:** \$0.06 per image
    * **DALL-E 3 HD (1024×1024):** \$0.080 per image
    * **DALL-E 3 Large (1792×1024):** $0.080-$0.120 per image

    Perfect for: Marketing materials, product images, illustrations
  </Accordion>

  <Accordion title="Video Generation (Per-Second Pricing)" icon="video">
    **AI video generation models:**

    * **Veo 3.0/3.1 Fast:** \$0.15 per second
    * **Veo 3.0/3.1 Standard:** \$0.40 per second

    Perfect for: Marketing videos, product demos, content creation
  </Accordion>

  <Accordion title="Audio Models (Transcription & TTS)" icon="microphone">
    **Audio transcription models:**

    * **Whisper-1:** $0.006 per minute ($0.0001/second)

    **Text-to-speech models:**

    * **TTS-1 (Standard):** \$15.00 per 1M characters
    * **TTS-1-HD (High Definition):** \$30.00 per 1M characters

    Perfect for: Transcription services, voice assistants, audiobooks, accessibility
  </Accordion>
</AccordionGroup>

***

## How Pricing Works

### Token Calculation

LLM Ops tracks both input and output tokens separately:

* **Input tokens** = Your prompt + any system messages + conversation history
* **Output tokens** = The model's response

**Example:**

```text theme={null}
Input: "Write a haiku about AI" (6 tokens)
Output: "Silicon dreams flow / Algorithms learn and grow / Future unfolds now" (15 tokens)

Cost with GPT-4o:
Input: 6 tokens × $2.50/1M = $0.000015
Output: 15 tokens × $10.00/1M = $0.00015
Total: $0.000165
```

### Cost Calculation

Your total cost is calculated as:

```text theme={null}
Total Cost = (Input Tokens × Input Price) + (Output Tokens × Output Price)
```

All costs are tracked in real-time and displayed in your [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard).

***

## Image & Video Generation Pricing

### Image Generation Models

Models like **DALL-E 3**, **Imagen 4**, and **gemini-2.5-flash-image** generate images and are priced differently:

**DALL-E 3 (Per-Image):**

```text theme={null}
Cost = Number of Images × Price Per Image

Example (Standard 1024×1024):
Generate 5 images with dall-e-3
Cost = 5 images × $0.040 = $0.20

Example (HD 1024×1792):
Generate 3 images with dall-e-3 HD quality
Cost = 3 images × $0.120 = $0.36
```

**Imagen 4 Models (Per-Image):**

```text theme={null}
Cost = Number of Images × Price Per Image

Example:
Generate 5 images with imagen-4.0-fast-generate-001
Cost = 5 images × $0.02 = $0.10
```

**gemini-2.5-flash-image (Token-Based):**

```text theme={null}
Text Input Cost = Input Tokens × $0.30/1M
Text Output Cost = Text Output Tokens × $2.50/1M
Image Output Cost = Image Output Tokens × $30.00/1M

Example:
Input: "Generate a sunset image" (100 tokens)
Output: Text description (50 tokens) + Image (1,290 tokens)

Text Input: 100 × $0.30/1M = $0.00003
Text Output: 50 × $2.50/1M = $0.000125
Image Output: 1,290 × $30.00/1M = $0.0387
Total: $0.038855

Note: Each image consumes approximately 1,290 tokens
This equals ~$0.039 per image
```

### Video Generation Models

**Veo Models (Per-Second):**

```text theme={null}
Cost = Video Duration (seconds) × Price Per Second

Example:
Generate 10-second video with veo-3.0-fast-generate-001
Cost = 10 seconds × $0.15 = $1.50
```

### Audio Transcription Models

**Whisper (Per-Second):**

```text theme={null}
Cost = Audio Duration (seconds) × Price Per Second

Example:
Transcribe 125.5-second audio file with whisper-1
Cost = 125.5 seconds × $0.0001 = $0.01255

Note: $0.0001/second = $0.006/minute
```

### Text-to-Speech Models

**TTS Models (Per-Character):**

```text theme={null}
Cost = (Characters / 1,000,000) × Price Per Million Characters

Example (Standard):
Generate speech from 1,250 characters with tts-1
Cost = (1,250 / 1,000,000) × $15.00 = $0.01875

Example (HD):
Generate speech from 1,250 characters with tts-1-hd
Cost = (1,250 / 1,000,000) × $30.00 = $0.0375
```

<Note>
  **Provider Counting:**

  Image, video, and audio generation costs are calculated based on what the provider reports:

  * **Images (DALL-E 3, Imagen 4):** Provider returns number of images generated
  * **Videos (Veo):** Provider returns video duration in seconds
  * **Audio (Whisper):** Provider returns audio duration in seconds
  * **TTS (TTS-1, TTS-1-HD):** Provider returns character count of input text

  We trust the provider's counts and multiply by our pricing table.
</Note>

***

## Multimodal Pricing

### Two Types of Multimodal Models

**1. Multimodal Understanding Models (Token-Based)**

* These models **analyze** images, videos, and audio
* Input media is converted to tokens by the provider
* Charged per token (text + media tokens combined)
* Examples: GPT-4o, Claude Opus, Gemini 2.5 Flash

**2. Media Generation Models (Per-Unit)**

* These models **create** images, videos, or audio
* DALL-E 3, Imagen 4: Charged per image generated
* Veo 3: Charged per second of video generated
* Whisper: Charged per second of audio transcribed
* TTS: Charged per character of text input
* gemini-2.5-flash-image: Hybrid (token-based, but image output uses premium rate)

<Note>
  **How Multimodal Tokens Are Tracked:**

  For **understanding models**, providers (OpenAI, Anthropic, Google) automatically convert images, video, and audio into tokens and include them in the response. LLM Ops tracks the total token count returned by the provider.

  **Image/video/audio input tokens are included in `input_tokens`** - they are not tracked separately.

  For **generation models**, we track based on what the provider charges:

  * **Text tokens:** Standard input/output pricing
  * **Images generated:** Per-image or per-token (depending on model)
  * **Video generated:** Per-second of video
  * **Audio transcribed:** Per-second of audio
  * **TTS generated:** Per-character of input text
</Note>

### Multimodal Understanding Models (Token-Based)

**These models analyze images, video, and audio sent as input:**

<Tabs>
  <Tab title="Images">
    **Models with image understanding:**

    * Gemini 2.0/2.5 Flash
    * GPT-4o
    * GPT-4o Mini
    * Claude Opus 4/4.5
    * Claude Sonnet 4/4.5

    **How it works:**

    1. You send an image with your prompt
    2. Provider converts image to tokens based on resolution
    3. Provider returns total `input_tokens` (text + image)
    4. LLM Ops tracks the total as input tokens
    5. Cost = `input_tokens × input_price`

    **Note:** Higher resolution images = more input tokens = higher cost
  </Tab>

  <Tab title="Audio">
    **Models with audio support:**

    * GPT-4o Audio Preview
    * GPT-4o Realtime Preview
    * Gemini 2.0 Flash
    * Gemini 2.5 Flash (with audio input premium)

    **How it works:**

    1. You send audio with your prompt
    2. Provider converts audio to tokens based on duration
    3. Provider returns total `input_tokens` (text + audio)
    4. LLM Ops tracks the total as input tokens
    5. Cost = `input_tokens × input_price`
  </Tab>

  <Tab title="Video">
    **Models with video support:**

    * Gemini 2.0/2.5 Flash
    * Gemini 1.5 Pro

    **How it works:**

    1. You send video with your prompt
    2. Provider converts video to tokens (duration × resolution × frames)
    3. Provider returns total `input_tokens` (text + video)
    4. LLM Ops tracks the total as input tokens
    5. Cost = `input_tokens × input_price`

    **Note:** Longer videos at higher resolution = significantly more input tokens
  </Tab>
</Tabs>

### Pricing Breakdown Summary

Here's how different types of content are charged:

| Content Type             | Understanding (Input)                                                                                | Generation (Output)                                                                                                                                              | Examples                                                                    |
| ------------------------ | ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| **Text**                 | Per token                                                                                            | Per token                                                                                                                                                        | All models                                                                  |
| **Images (Input)**       | Converted to tokens by provider<br />Included in `input_tokens`                                      | N/A                                                                                                                                                              | GPT-4o, Claude Opus, Gemini Flash                                           |
| **Audio (Input)**        | Converted to tokens by provider<br />Included in `input_tokens`<br />Some models charge premium rate | N/A                                                                                                                                                              | GPT-4o Audio, Gemini Flash <br /> (Audio input: $1.00/1M vs $ 0.30/1M text) |
| **Audio Transcription**  | Per second ( $0.0001/sec = $ 0.006/min)                                                              | Text output (per token)                                                                                                                                          | whisper-1                                                                   |
| **Video (Input)**        | Converted to tokens by provider<br />Included in `input_tokens`                                      | N/A                                                                                                                                                              | Gemini 2.5 Flash                                                            |
| **Images (Output)**      | N/A                                                                                                  | **DALL-E 3:** Per image ( $0.040-$ 0.120) <br /> **Imagen 4:** Per image ( $0.02-$ 0.06) <br /> **gemini-2.5-flash-image:** Per token ( $30/1M = ~$ 0.039/image) | dall-e-3, imagen-4.0-fast<br />gemini-2.5-flash-image                       |
| **Video (Output)**       | N/A                                                                                                  | Per second ( $0.15-$ 0.40/sec)                                                                                                                                   | veo-3.0-fast, veo-3.1                                                       |
| **Audio (Output - TTS)** | N/A                                                                                                  | **OpenAI TTS:** Per character ( $15-$ 30/1M) <br /> **Gemini TTS:** Per token ( $10-$ 20/1M)                                                                     | tts-1, tts-1-hd<br />gemini-2.5-flash-tts                                   |

<Note>
  **Key Takeaways:**

  * **Understanding (Input):** Media → Tokens → Cost per token
  * **Generation (Output):**
    * Images: Per image (DALL-E 3, Imagen 4) OR per token (gemini-2.5-flash-image)
    * Video: Per second (Veo)
    * Audio Transcription: Per second (Whisper)
    * Text-to-Speech: Per character (OpenAI TTS) OR per token (Gemini TTS)
  * **Provider Controls Conversion:** We trust provider counts
  * **Not Tracked Separately:** Input media tokens are combined with text tokens in `input_tokens`
</Note>

### Example: Image Token Calculation

```text theme={null}
Request:
- Text prompt: "Describe this image" (4 tokens)
- Image: 1024x1024 JPG (converted to 765 tokens by provider)

Provider Response:
{
  "usage": {
    "input_tokens": 769,     // 4 text + 765 image
    "output_tokens": 50      // Response tokens
  }
}

Cost Calculation (using GPT-4o pricing):
Input: 769 tokens × $2.50/1M = $0.0019225
Output: 50 tokens × $10.00/1M = $0.0005
Total: $0.0024225
```

**LLM Ops Dashboard shows:**

* Input Tokens: 769 (includes both text and image)
* Output Tokens: 50
* Total Cost: \$0.0024225

<Warning>
  **Image/Audio/Video Breakdown Not Available:**

  LLM Ops does **not** currently separate multimodal tokens from text tokens. All input tokens (text + image + video + audio) are tracked together as `input_tokens`.

  If you need separate multimodal token tracking, please contact us at [support@cloudidr.com](mailto:support@cloudidr.com).
</Warning>

***

## Pricing Updates

Model pricing is set by the providers (Anthropic, OpenAI, Google) and can change at any time.

**How we handle updates:**

* ✅ We monitor provider pricing pages daily
* ✅ Updates are applied within 24 hours of provider changes
* ✅ Historical data uses pricing from the time of request
* ✅ You're notified of major pricing changes

<Note>
  **Last pricing update:** January 10, 2026

  Check this page regularly for pricing updates.
</Note>

***

## Need a Model Added?

If you're using a model that's not listed here:

<Steps>
  <Step title="Check Provider Documentation">
    Verify the model exists in your provider's official API docs
  </Step>

  <Step title="Contact Us">
    Email [support@cloudidr.com](mailto:support@cloudidr.com) with:

    * Model name
    * Provider (Anthropic/OpenAI/Google)
    * Link to provider pricing
  </Step>

  <Step title="We'll Add It">
    We typically add new models within 2-3 business days
  </Step>
</Steps>

***

## Next Steps

<CardGroup>
  <Card title="Anthropic Integration" icon="sparkles" href="/guides/llm-ops/integrations/anthropic">
    Start tracking Claude costs
  </Card>

  <Card title="OpenAI Integration" icon="brain" href="/guides/llm-ops/integrations/openai">
    Start tracking GPT costs
  </Card>

  <Card title="Google Integration" icon="google" href="/guides/llm-ops/integrations/gemini">
    Start tracking Gemini costs
  </Card>
</CardGroup>


# Cost Attribution (Tagging) 
Source: https://docs.cloudidr.com/guides/llm-ops/tagging



Tagging is how Cloudidr LLM Ops cost attributes every API request to the right part of your organization. Three optional HTTP headers added to your request unlock full cost visibility by department, project, and agent — across every model and provider.

***

## How Tagging Works

Tags are passed as standard HTTP headers on each request through the Cloudidr proxy, alongside your `X-Cloudidr-Key` authentication header. Cloudidr logs each tag value alongside the request's token count, cost, model, and timestamp. The result is a fully attributed cost record — queryable by any tag dimension in your LLM Ops dashboard.

```http theme={null}
X-Cloudidr-Key: trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx
X-Department:   Engineering
X-Project:      Customer-Search
X-Agent:        Query-Optimizer
```

***

## The Three Tag Dimensions

| Header         | Purpose                                        | Example values                                      |
| -------------- | ---------------------------------------------- | --------------------------------------------------- |
| `X-Department` | Business unit, org division, or team           | `Engineering`, `Marketing`, `Sales`, `Data-Science` |
| `X-Project`    | Specific initiative, workload, or product area | `Customer-Search`, `Seo-Pipeline`, `Lead-Gen-v2`    |
| `X-Agent`      | Individual agent, service, or use case name    | `Monitoring`, `SEO`, `Lead Gen`, `API Service`      |

> **`X-Department` can represent a department or a team — whichever fits your org structure.** Some organizations tag by formal department (`Engineering`, `Marketing`). Others use it as a team label (`Backend`, `DevOps`, `Content`). Both are valid. If you only have one level of group hierarchy, use `X-Department` for it.

***

## Flexibility — Use What You Need

All three tags are completely optional. Use none, one, two, or all three — any combination works.

| Tags used                  | What you get in the dashboard                       |
| -------------------------- | --------------------------------------------------- |
| None                       | Organization-wide token and cost totals only        |
| `X-Agent` only             | Per-agent cost and savings breakdown                |
| `X-Department` only        | Cost rollup by business unit or team                |
| `X-Department` + `X-Agent` | Department-level rollup with per-agent drill-down   |
| `X-Project` + `X-Agent`    | Project-level cost with per-agent attribution       |
| All three                  | Full cost attribution: department → project → agent |

Start with just `X-Agent` and add more dimensions as your observability needs grow.

***

## Code Examples

### Python — Anthropic

```python theme={null}
from anthropic import Anthropic

client = Anthropic(
    api_key="your-anthropic-key",
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Key": "trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
        "X-Department": "Engineering",    # Optional: Organize by department/team
        "X-Project": "Customer-Search",   # Optional: Organize by project
        "X-Agent": "Query-Optimizer"      # Optional: Organize by agent/use case
    }
)

message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarize this support ticket."}]
)
```

### Python — OpenAI

```python theme={null}
from openai import OpenAI

client = OpenAI(
    api_key="your-openai-key",
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Key": "trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
        "X-Department": "Marketing",       # Optional: Organize by department/team
        "X-Project": "SEO-Pipeline",       # Optional: Organize by project
        "X-Agent": "SEO"                   # Optional: Organize by agent/use case
    }
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write SEO meta descriptions."}]
)
```

### JavaScript / Node.js — Anthropic

```javascript theme={null}
import Anthropic from '@anthropic-ai/sdk'

const client = new Anthropic({
  apiKey: 'your-anthropic-key',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Key': 'trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'X-Department': 'Sales',              // Optional: Organize by department/team
    'X-Project': 'Lead-Qualification',    // Optional: Organize by project
    'X-Agent': 'Lead Gen'                 // Optional: Organize by agent/use case
  }
})

const message = await client.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Qualify this inbound lead.' }]
})
```

### JavaScript / Node.js — OpenAI

```javascript theme={null}
import OpenAI from 'openai'

const client = new OpenAI({
  apiKey: 'your-openai-key',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Key': 'trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    'X-Department': 'Engineering',        // Optional: Organize by department/team
    'X-Project': 'Infra-Monitoring',      // Optional: Organize by project
    'X-Agent': 'Monitoring'               // Optional: Organize by agent/use case
  }
})

const response = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Check system health status.' }]
})
```

### cURL — Anthropic

```bash theme={null}
curl https://api.llm-ops.cloudidr.com/v1/messages \
  -H "Authorization: Bearer your-anthropic-key" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -H "X-Cloudidr-Key: trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "X-Department: Engineering" \
  -H "X-Project: Infra-Monitoring" \
  -H "X-Agent: Monitoring" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Check system health status."}]
  }'
```

### cURL — OpenAI

```bash theme={null}
curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Authorization: Bearer your-openai-key" \
  -H "Content-Type: application/json" \
  -H "X-Cloudidr-Key: trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "X-Department: Marketing" \
  -H "X-Project: PPC-Campaigns" \
  -H "X-Agent: PPC Campaigns" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Generate ad copy."}]
  }'
```

***

## Real-World Tagging Patterns

### Pattern 1 — Agent Only

Suitable for single-team setups or early-stage cost tracking. Immediately surfaces per-agent cost breakdown in Agent Explorer.

```http theme={null}
X-Cloudidr-Key: trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx
X-Agent: Chatbot
```

**Dashboard result:** Total cost and tokens broken down per agent. Budget Guard limits can be set immediately.

***

### Pattern 2 — Department as Team Label

Use `X-Department` to represent a team when your org doesn't distinguish department from team hierarchy.

```http theme={null}
X-Cloudidr-Key: trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx
X-Department: Backend
X-Agent: API Service
```

**Dashboard result:** Team-level cost rollup (`Backend`) with per-agent detail (`API Service`).

***

### Pattern 3 — Full Attribution (Recommended for Scale)

Full three-tag setup gives the complete org cost picture — department → project → agent — in a single dashboard view.

```http theme={null}
X-Cloudidr-Key: trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx
X-Department: Marketing
X-Project: SEO-Pipeline
X-Agent: SEO
```

**Dashboard result:**

* Department: `Marketing` → \$900/month
* Project: `SEO-Pipeline` → \$900/month
* Agent: `SEO` → \$900/month, 91.2% savings rate

***

### Pattern 4 — Multi-Agent Project

Tag each agent call with the same project so the dashboard rolls them up into a single project cost view.

```python theme={null}
# Agent 1 — research step
headers_research = {
    "X-Cloudidr-Key": "trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "X-Department": "Sales",
    "X-Project": "Outbound-Pipeline",
    "X-Agent": "Lead Research",
}

# Agent 2 — drafting step
headers_draft = {
    "X-Cloudidr-Key": "trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "X-Department": "Sales",
    "X-Project": "Outbound-Pipeline",
    "X-Agent": "Email Drafter",
}

# Agent 3 — qualification step
headers_qualify = {
    "X-Cloudidr-Key": "trk_xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "X-Department": "Sales",
    "X-Project": "Outbound-Pipeline",
    "X-Agent": "Lead Qualifier",
}
```

**Dashboard result:** `Outbound-Pipeline` total cost shown in one view, split across Lead Research, Email Drafter, and Lead Qualifier.

***

## What You See in the Dashboard

Once tags are flowing, the LLM Ops dashboard lets you:

* **Slice by any tag dimension** — filter spend by `X-Department`, `X-Project`, or `X-Agent`
* **Compare across departments/teams** — see which group drives the most spend and at what savings rate
* **Set Budget Guard limits per agent** — Budget Guard uses the `X-Agent` value to enforce per-agent monthly budgets
* **Track project-level ROI** — tie token spend directly to a business initiative using `X-Project`
* **Identify runaway agents** — an agent with unexpectedly high token consumption shows up immediately in Agent Explorer

***

## Tag Naming Conventions

> ⚠️ **Tag values are case-sensitive and matched by exact string.** `Engineering` and `engineering` are stored as two separate groups in the dashboard. A single inconsistency across your codebase will split what should be one group into two. Establish your convention once and enforce it everywhere.

**Recommended: use Title Case for all tag values.**

* `X-Department` → `Engineering`, `Marketing`, `Inside Sales`, `Data Science`
* `X-Agent` → `Lead Gen`, `Monitoring`, `SEO`, `API Service`, `Content Writer`
* `X-Project` → `Customer-Search`, `SEO-Pipeline`, `Lead-Gen-V2` (hyphenated Title Case works well for programmatic names)

Title Case reads cleanly as display labels in the dashboard, matches how agents appear in Agent Explorer, and reduces the most common mistake — one service using `marketing` and another using `Marketing` creating two separate cost groups.

**Other conventions to follow:**

* Use **hyphens for multi-word project names** — `Customer-Search` not `Customer Search` (spaces can cause issues in some header parsers)
* Keep values **short and stable** — these become filter labels and budget group identifiers; changing them mid-deployment splits your historical data
* **Never mix conventions** — if you start with `Engineering`, don't switch to `engineering` or `ENGINEERING` mid-project

***

## Summary

| Header           | Required | Description                | Example           |
| ---------------- | -------- | -------------------------- | ----------------- |
| `X-Cloudidr-Key` | **Yes**  | Your Cloudidr tracking key | `trk_xxxx...`     |
| `X-Department`   | No       | Department or team         | `Engineering`     |
| `X-Project`      | No       | Project or workload        | `Customer-Search` |
| `X-Agent`        | No       | Agent or service name      | `Lead Gen`        |

Use none, one, or all three optional tags — tagging is additive. Every tag unlocks a new dimension of cost visibility in your LLM Ops dashboard without any other configuration required.

***

*For questions or integration help → [support@cloudidr.com](mailto:hello@cloudidr.com) · [llm-ops.cloudidr.com](https://llm-ops.cloudidr.com)*


# Team Management
Source: https://docs.cloudidr.com/guides/llm-ops/team-mgmt



Owners can invite team members to join Cloudidr. The first user who signed up with the company name is the Owner. Owner can invite other users and pay for the subscription. Here are the steps to invite team members

1. Go to **Settings** in left side panel and click **Team Managment**

<Frame>
  <img alt="Screenshot2026 02 24at13 01 16 1" />
</Frame>

2 Click on **+ Invite Team Member** to send invitation. The team members an then sign up as Members

<Frame>
  <img alt="Screenshot2026 02 24at13 01 38" />
</Frame>

3. Owner can **Change role** for team members from Team management page, enable/disable member.
4. Owner can  make other members as owners as well.
5. Owner can use **Team Analytics** side panel to view usage across each team member.
6. Owner can use **Settings** > **API Keys** to **revoke** API Keys for team members.


# What is LLM Ops?
Source: https://docs.cloudidr.com/guides/llm-ops/what-is-llm-ops

Stop overpaying for AI. Start with complete visibility — optimize from day one.

LLM Ops is Cloudidr's **AI FinOps** platform that gives your team complete **visibility, control, and intelligent optimization** of LLM API costs across AWS Bedrock, OpenAI, Anthropic Claude, and Google Gemini. Teams using LLM Ops reduce AI API spend by 30-90% through intelligent **model routing, hard budget enforcement, and real-time cost intelligence** — without changing their provider relationships or AWS billing.

Set up in 60 seconds. Free forever for small teams.

**Your provider relationship stays yours.** LLM Ops sits between your application and your AI providers as a transparent proxy — tracking metadata only. Two things never change:

* **Your API keys are never stored.** They pass through in memory for mili-seconds during each request and are immediately discarded. We never write them to disk, never log them, and never store them in our database. Your prompts and responses remain completely private.
* **You pay your provider directly.** LLM Ops does not sit in your billing path. Your AWS Bedrock charges go to AWS. Your OpenAI charges go to OpenAI. Your existing contracts, enterprise discounts, and committed spend agreements are completely unaffected. We only charge for LLM Ops itself.

<Frame>
  <img alt="Dashboard Main" />
</Frame>

***

## The Problem — AI Costs Are Invisible Until It's Too Late

Your AI bill went from \$200 to \$8,000 in one month. Your CFO asks: "Why?"

Without LLM Ops, you cannot answer:

* Which project or agent caused the spike?
* Are we using Claude Opus for tasks Haiku could handle at 20x lower cost?
* Which Bedrock models are being called and at what frequency?
* Where is budget being wasted on over-provisioned model capacity?
* What would we save if we routed simple requests to cheaper models?

Most companies discover these problems when the invoice arrives — too late to fix, too late to explain, and too late to prevent next month's repeat.

LLM Ops gives you the answer to every one of these questions in real time — and acts on them automatically through intelligent routing and budget enforcement.

***

## How LLM Ops Works

LLM Ops sits as a transparent proxy between your application and your AI providers. Every API call passes through the Cloudidr proxy endpoint, which logs metadata in real time and optionally applies routing and budget enforcement — then forwards the request to your chosen provider unchanged.

> Your Application
>
> ↓
>
> Cloudidr LLM Ops Proxy ([api.llm-ops.cloudidr.com](http://api.llm-ops.cloudidr.com))
>
> ↓ tracks metadata, enforces budgets, applies routing
>
> AI Provider (AWS Bedrock / OpenAI / Anthropic / Gemini)
>
> ↓
>
> Response returned to your application

**Your API keys are never stored.** They pass through securely via HTTPS and are immediately discarded after the request completes — typically within 1-2 seconds. We only log metadata: token counts, model names, timestamps, and calculated costs. Your prompts and responses remain completely private and are never stored.

Latency overhead: under 40ms on average. Your users will not notice the difference as the providers 2 way latencies are 800-2000ms.

***

## Simple Integration  — 2 Lines of Code

Add 2 lines of code to start tracking. See example below for OpenAI.

```python highlight={6,9} theme={null}
from openai import OpenAI

client = OpenAI(
    api_key="your-openai-key",
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={

        "X-Cloudidr-Key": "trk_your_token",       # Required
        "X-Department": "engineering",           # Optional
        "X-Project": "ml",                      # Optional
        "X-Agent": "chatbot"                    # Optional
    }
)

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

See more details at : [Anthropic Code Integration](/guides/llm-ops/integrations/anthropic), [OpenAI Integration](/guides/llm-ops/integrations/openai), [Google Integration](/guides/llm-ops/integrations/google) and AWS Bedrock.

***

## Supported Providers

| Provider           | Integration                                 |
| :----------------- | :------------------------------------------ |
| AWS Bedrock        | ✅ Full catalog — 15 providers, 62 models    |
| OpenAI             | ✅ All GPT models                            |
| Anthropic          | ✅ Direct API — all Claude models            |
| Google Gemini      | ✅ All Gemini models                         |
| Self-hosted models | ✅ Via Cloudidr hosted model access (Scale+) |

***

## Key Capabilities

### 1. Real-Time Cost Visibility

Track LLM spending across all providers in a single unified dashboard. See costs broken down by:

* **Department** — Engineering vs Marketing vs Product
* **Project** — Individual projects
* **Agent or Feature** — Customer support bot, code assistant, content generator, RAG pipeline
* **Model** — Which specific models are consuming budget
* **Provider** — AWS Bedrock vs OpenAI vs Anthropic vs Gemini
* **Time** — Hourly, daily, weekly, and monthly trends

### 2. Hard Budget Controls — Budget Guard

Set hard spending limits per agent, team, project, or API key. When the limit is reached, requests automatically block — no overages, no surprises.

* Alerts fire at **80% and 90%** of budget
* Requests **auto-block at 100%** — not just alert
* Budget Guard supports up to 5 agents on the free tier, 30 on Growth, and unlimited on Scale and Enterprise
* Works across all providers simultaneously from a single control layer

### 3. Intelligent Model Routing

Smart routing automatically selects the cheapest model capable of handling each request based on complexity scoring. You define the quality threshold — LLM Ops handles the routing decision.

**Three routing strategies:**

**Intra Provider (available on all plans)** Routes within a single provider's model family based on task complexity. Examples:

* Claude Opus → routed to Claude Haiku for simple prompts
* Nova Premier → routed to Nova Micro for lightweight tasks
* GPT-4o → routed to GPT-4o Mini where complexity allows

**Flexible**  Routes across providers to find the best price-performance match for each request. Compares AWS Bedrock, OpenAI, Anthropic, and Gemini models in real time.

**Fixed**  Routes all requests to a specific model you define. Useful for testing, compliance requirements, or locking a cost ceiling.

**Typical savings from routing: 30-90% reduction in LLM costs.**

### 4. AI-Powered Optimization Insights

LLM Ops continuously analyzes your usage patterns and surfaces actionable recommendations:

* Which requests could use cheaper models without quality loss
* Where you are over-provisioning capacity
* Opportunities to batch requests for cost reduction
* Potential monthly savings from model switching
* Cost anomalies and unusual spending patterns flagged automatically

Adaptive AI learning (Growth and above) improves routing decisions over time based on your specific workload patterns.

### 5. GPU Usage and Compute Metrics

For teams running self-hosted or GPU-accelerated inference workloads, LLM Ops extends visibility beyond API costs to include:

* GPU utilization tracking
* Compute cost per inference
* GPU instance efficiency metrics
* Cost comparison between API and self-hosted inference

### 6. Forecasting

Project future LLM spending based on current usage trends:

* Monthly spend forecasts by team and agent
* Budget runway estimates
* Growth trend analysis
* Capacity planning recommendations

***

## Why Teams Choose LLM Ops

**60-second setup** — No complex installation, no infrastructure changes. Add 2 lines of code and you are tracking costs across every provider.

**Under 40ms overhead** — Your API calls go directly to providers via HTTPS. Users do not notice the difference.

**Free forever for small teams** — Core features including cost tracking, Budget Guard, smart routing, and multi-provider support are completely free. No credit card required.

**Privacy first** — API keys never stored. Prompts and responses never logged. Only metadata is tracked: token counts, model names, timestamps, and costs. Full encryption in transit and at rest.

**No vendor lock-in** — Remove 2 lines of code and your application connects directly to your AI provider exactly as before. Nothing changes on your provider side.

**Works with AWS Bedrock natively** — No changes to your AWS account, IAM roles, or billing. Your Bedrock relationship stays between you and AWS.

***

## Who Uses LLM Ops

**AI Startups** Track costs as you scale from \$100 to tens of thousands per month. Catch runaway agent costs before they become a crisis. Start free, upgrade as you grow.

**Engineering Teams** Show leadership exactly where AI budget goes. Justify optimization investments with real data. Enforce per-team budgets without manual monitoring.

**FinOps Professionals** Apply cloud cost management discipline to AI infrastructure. LLM Ops brings the same visibility and control you have for AWS EC2 and S3 to your LLM spend.

**Product Teams** Understand which features drive AI costs and make data-driven architecture decisions. Know the true cost of each product capability before committing to scale.

**Platform Teams Building on AWS Bedrock** Get complete visibility across all 62 Bedrock models without changing your AWS setup, IAM configuration, or billing relationship.

***

## Why "AI FinOps" Not Just "Cost Tracking"

Most tools tell you what you spent. LLM Ops tells you what you wasted — and automatically fixes it.

FinOps transformed how engineering teams manage cloud infrastructure costs on AWS, Azure, and GCP. LLM Ops brings the same discipline to AI infrastructure — giving you the visibility, governance, and optimization levers that FinOps teams apply to EC2 and S3, now applied to every LLM API call your applications make.

> What is AI FinOps ? Read this [blog](https://www.cloudidr.com/blog/ai-finops) to master key concepts behind AI FinOps

| Cloud FinOps                   | AI FinOps with LLM Ops                      |
| ------------------------------ | ------------------------------------------- |
| Right-size EC2 instances       | Right-size model selection per request      |
| Budget alerts on AWS spend     | Budget Guard per agent, project, department |
| Reserved Instance optimization | Intelligent routing to cheaper models       |
| Cost allocation by team        | Cost breakdown by team, agent, model        |
| Savings Plans for commitment   | Routing strategies for cost reduction       |
| CloudWatch cost anomalies      | Real-time AI spend anomaly detection        |

***

## Support

**Need Help?**

* 📧 Email: [support@cloudidr.com](mailto:support@cloudidr.com)
* 💬 Discord: [Join our community](https://discord.gg/V3VXFnex)

***

**Try LLM Ops:** [llm-ops.cloudidr.com/signup](http://llm-ops.cloudidr.com/signup)


# Intro
Source: https://docs.cloudidr.com/intro



The **Playground** is a dashboard page where you can send prompts through the Cloudidr proxy without writing code or using a terminal. It has two tabs:

| Tab                             | Purpose                                                                                                                             |
| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| [**Try a Model**](/try-model)   | Run a single prompt against one provider and model; inspect tokens, cost, and latency.                                              |
| [**Evaluations**](/evaluations) | Compare your **baseline** model with Cloudidr’s **routed** substitute, see savings, and optionally score quality with an LLM judge. |

## Video tutorial

[Watch on YouTube →](https://youtu.be/Po4xf1ZmevY?si=L0XpBa_35ufb0QP3)

![Playground walkthrough](https://img.youtube.com/vi/Po4xf1ZmevY/0.jpg)


# Try Model
Source: https://docs.cloudidr.com/try-model



Use this tab to **experiment with one model at a time**—same flow as a minimal chat client, but requests go through the proxy and are logged like production traffic.

## What this tab is for

* Validate that your tracking token and tags work.
* Compare latency and cost across providers or models.
* Try sample prompts or paste your own text.

## Configuration

### Provider and API key

Choose **OpenAI**, **Anthropic**, **Google Gemini**, or **Cloudidr**.

* **Third-party providers** (OpenAI, Anthropic, Google): paste your own provider API key. The key is sent with the request and is **not** stored by Cloudidr.
* **Cloudidr**: select a managed model (Gemma or Qwen). Uses your Cloudidr tracking token only; **Managed Inference credits** apply. If your org has no credits, Cloudidr models are blocked with a clear billing message.

### Model

Pick a **model** from the list for the selected provider. Lists are ordered with newer / flagship entries first where applicable.

### Tracking token (Cloudidr API key)

Pick the default tracking token or choose another active token from the dropdown. This ties the run to your org for usage and billing.

### Tags (optional)

Optional **X-Department**, **X-Project**, and **X-Agent** values (with sample chips) flow into **LLM Usage** as dimensions for filtering—same headers as the proxy.

<Frame>
  <img alt="Image" />
</Frame>

## Prompt and sample prompts

Enter any prompt in the text field. **Sample prompts** (e.g. summarize in bullets, REST vs GraphQL) fill the editor in one click.

### Max tokens

Caps the model’s completion length for this run.

### Thinking budget (Gemini and Qwen only)

For **Google Gemini** and **Cloudidr Qwen**, a **thinking budget** slider controls how many tokens may be used for internal reasoning before the visible answer. If thinking consumes the whole **max tokens** budget, the UI explains that you should raise max tokens. **Gemma** does not use this control.

### Run Prompt

Submits the request. Keyboard shortcut: **⌘ Enter** (Mac) or **Ctrl+Enter** (Windows).

## Response panel

Shows the assistant text plus **model**, **latency**, **input / thinking / output tokens**, and **estimated cost**. A note confirms the request was logged to **LLM Usage**.