> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cloudidr.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM Integration

> Track your LiteLLM proxy usage and costs in Cloudidr LLM Ops

## Overview

Integrate LiteLLM with Cloudidr LLM Ops to automatically track API usage and costs. LiteLLM sends webhook callbacks to our system after each API request, allowing you to monitor usage, costs, and organize by department, team, and agent.

<Note>
  **How it works:**

  LiteLLM → Webhook Callback → CloudIDR LLM Ops → Dashboard

  Configure LiteLLM to send webhooks to our endpoint, add tracking headers to your API requests, and we'll automatically track all usage and costs.
</Note>

***

## Your Tracking Token

<Warning>
  **Keep this token secure!**

  Your tracking token is required for all LiteLLM webhook callbacks. Don't share it publicly or commit it to version control.
</Warning>

```
trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr
```

<Tip>
  Use environment variables to store your tracking token securely.
</Tip>

***

## Configuration Steps

<Steps>
  <Step title="Update LiteLLM config.yaml">
    Add webhook callback configuration to your LiteLLM config file
  </Step>

  <Step title="Add Tracking Headers">
    Include required and optional headers in your API requests
  </Step>

  <Step title="Restart & Test">
    Restart LiteLLM and verify tracking in your dashboard
  </Step>
</Steps>

***

## Step 1: Update LiteLLM config.yaml

Choose the configuration that matches your setup:

<Tabs>
  <Tab title="With Database (Recommended)">
    ### Configuration with Database

    Use this if you want to keep LiteLLM's database features (user management, spend tracking, etc.)

    ```yaml theme={null}
    # LiteLLM Proxy Configuration - With Database
    model_list:
      - model_name: gpt-4
        litellm_params:
          model: gpt-4
          api_key: os.environ/OPENAI_API_KEY
      - model_name: claude-sonnet-4
        litellm_params:
          model: claude-sonnet-4
          api_key: os.environ/ANTHROPIC_API_KEY

    # Generic API Callback - Point to our webhook endpoint
    callback_settings:
      llm_ops:
        callback_type: generic_api
        endpoint: https://api.llm-ops.cloudidr.com/api/litellm/callback
        headers:
          Content-Type: application/json
        event_types:
          - llm_api_success
          - llm_api_failure

    litellm_settings:
      callbacks: ["llm_ops"]
      # Use your existing LiteLLM database (PostgreSQL, SQLite, etc.)
      database_url: os.environ/DATABASE_URL  # e.g., "postgresql://user:pass@host:5432/litellm"

    general_settings:
      master_key: os.environ/LITELLM_MASTER_KEY
    ```

    <Info>
      **Recommended for most users** - This preserves all LiteLLM features while adding LLM Ops tracking.
    </Info>
  </Tab>

  <Tab title="Without Database (Optional)">
    ### Configuration without Database

    Use this **only** if you don't need LiteLLM's database features and want to disable them.

    ```yaml theme={null}
    # LiteLLM Proxy Configuration - Without Database
    # ⚠️  Note: This disables LiteLLM's built-in features like user management and spend tracking
    # Only use this if you're managing everything through LLM Ops

    model_list:
      - model_name: gpt-4
        litellm_params:
          model: gpt-4
          api_key: os.environ/OPENAI_API_KEY

    # Generic API Callback - Point to our webhook endpoint
    callback_settings:
      llm_ops:
        callback_type: generic_api
        endpoint: https://api.llm-ops.cloudidr.com/api/litellm/callback
        headers:
          Content-Type: application/json
        event_types:
          - llm_api_success
          - llm_api_failure

    litellm_settings:
      callbacks: ["llm_ops"]
      database_url: ""  # Empty string disables database

    general_settings:
      master_key: os.environ/LITELLM_MASTER_KEY
      disable_database_checks: true  # Required when database_url is empty
    ```

    <Warning>
      **Important:** This disables LiteLLM's user management and spend tracking features. Only use if you're managing everything through LLM Ops.
    </Warning>
  </Tab>
</Tabs>

<Tip>
  **Recommendation:** If you're already using LiteLLM with a database, use the "With Database" configuration. Only use "Without Database" if you don't need LiteLLM's built-in features.
</Tip>

***

## Step 2: Add Tracking Headers to Your Requests

Include tracking headers in your API requests to LiteLLM.

### Required Header

| Header             | Description                        | Example                        |
| ------------------ | ---------------------------------- | ------------------------------ |
| `X-Cloudidr-Token` | **Required** - Your tracking token | `trk_fXOn-A1V8VrCxXyJ1WuMX...` |

### Optional Metadata Headers

| Header         | Description                         | Example                             |
| -------------- | ----------------------------------- | ----------------------------------- |
| `X-Department` | Organize costs by department        | `engineering`, `sales`, `marketing` |
| `X-Team`       | Organize costs by team              | `backend`, `frontend`, `ml`         |
| `X-Agent`      | Organize costs by agent/application | `chatbot`, `summarizer`, `analyzer` |

<Note>
  **Optional Metadata:** These headers are optional. If omitted, requests will still be tracked, but won't be organized by department/team/agent in your dashboard.
</Note>

***

### Code Examples

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import openai
    from openai import OpenAI

    client = OpenAI(
        api_key="your-openai-key",
        base_url="http://localhost:4000/v1",  # Your LiteLLM proxy URL local host or your production url
    )

    # Make request with tracking headers
    # X-Cloudidr-Token is required; X-Department, X-Team, X-Agent are optional
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello!"}],
        extra_headers={
            "X-Cloudidr-Token": "trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr",  # Required
            # Optional metadata headers:
            "X-Department": "engineering",  # Optional
            "X-Team": "backend",            # Optional
            "X-Agent": "chatbot"            # Optional
        }
    )

    print(response.choices[0].message.content)
    ```

    <Tip>
      Use `extra_headers` parameter to pass custom headers to LiteLLM.
    </Tip>
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    import OpenAI from 'openai';

    const client = new OpenAI({
      apiKey: 'your-openai-key',
      baseURL: 'http://localhost:4000/v1',  // Your LiteLLM proxy URL
      defaultHeaders: {
        'X-Cloudidr-Token': 'trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr',  // Required
        // Optional metadata headers:
        'X-Department': 'engineering',  // Optional
        'X-Team': 'backend',            // Optional
        'X-Agent': 'chatbot'            // Optional
      }
    });

    const response = await client.chat.completions.create({
      model: 'gpt-4',
      messages: [{ role: 'user', content: 'Hello!' }]
    });

    console.log(response.choices[0].message.content);
    ```

    <Tip>
      Use `defaultHeaders` to set headers once for all requests.
    </Tip>
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST http://localhost:4000/v1/chat/completions \
      -H "Authorization: Bearer your-master-key" \
      -H "Content-Type: application/json" \
      -H "X-Cloudidr-Token: trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr" \
      -H "X-Department: engineering" \
      -H "X-Team: backend" \
      -H "X-Agent: chatbot" \
      -d '{
        "model": "gpt-4",
        "messages": [
          {
            "role": "user",
            "content": "Hello!"
          }
        ]
      }'
    ```

    <Note>
      **Note:** `X-Cloudidr-Token` is required. `X-Department`, `X-Team`, `X-Agent` are optional.
    </Note>
  </Tab>
</Tabs>

***

## Step 3: Restart LiteLLM and Test

Restart your LiteLLM proxy and make a test request. Check your dashboard to verify the request was tracked.

```bash theme={null}
# Restart LiteLLM with updated config
litellm --config config.yaml --port 4000

# Make a test request
curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer your-master-key" \
  -H "Content-Type: application/json" \
  -H "X-Cloudidr-Token: trk_fXOn-A1V8VrCxXyJ1WuMX-KlXn-e84mr" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

<Info>
  **Success!** If everything is configured correctly, you should see the request appear in your [LLM Ops Dashboard](https://llmfinops.ai/dashboard) within a few seconds.
</Info>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="Webhooks not being received?" icon="webhook">
    **Common issues:**

    * ✅ Verify `callback_settings` is correctly formatted in `config.yaml`
    * ✅ Check that `headers` field is included (even if just `Content-Type: application/json`)
    * ✅ Ensure the endpoint URL is accessible from your LiteLLM server
    * ✅ Check LiteLLM logs for callback errors (enable `set_verbose: true`)

    **Test webhook connectivity:**

    ```bash theme={null}
    curl -X POST https://api.llm-ops.cloudidr.com/api/litellm/callback \
      -H "Content-Type: application/json" \
      -d '{"test": true}'
    ```
  </Accordion>

  <Accordion title="Requests not showing in dashboard?" icon="chart-line">
    **Common issues:**

    * ✅ Verify `X-Cloudidr-Token` header is included in your requests
    * ✅ Check that the tracking token is active (not revoked)
    * ✅ Ensure the token belongs to your organization
    * ✅ Check API server logs for `[LITELLM]` messages

    **Debug tip:** Enable verbose logging in LiteLLM:

    ```yaml theme={null}
    litellm_settings:
      set_verbose: true
    ```
  </Accordion>

  <Accordion title="Metadata (Department/Team/Agent) not showing?" icon="tags">
    **Remember:**

    * These headers are **optional** - requests will still be tracked without them
    * Verify headers are passed correctly: `X-Department`, `X-Team`, `X-Agent`
    * Check that headers are passed via `extra_headers` (Python) or `defaultHeaders` (JavaScript)
    * For cURL, include headers directly: `-H "X-Department: engineering"`

    <Note>
      **Note:** We currently extract metadata from custom headers. LiteLLM's built-in `x-litellm-tags` header is not automatically mapped to our metadata fields.
    </Note>
  </Accordion>

  <Accordion title="Database configuration issues?" icon="database">
    **With Database:**

    * Keep your existing `database_url` configuration
    * Don't add `disable_database_checks`

    **Without Database:**

    * Set `database_url: ""` (empty string)
    * Must add `disable_database_checks: true`

    **Error: "Database checks failed"**

    * You forgot `disable_database_checks: true` when using empty `database_url`
  </Accordion>
</AccordionGroup>

***

## What Gets Tracked

LLM Ops automatically captures from LiteLLM webhooks:

✅ **Token usage** - Input, output, and total tokens\
✅ **Cost** - Real-time cost calculation\
✅ **Latency** - Request duration\
✅ **Model** - Which model was used\
✅ **Metadata** - Department, team, agent (from headers)\
✅ **Errors** - Failed requests and error types\
✅ **Source** - Marked as `litellm` in the database

<Warning>
  **What We DON'T Track:**

  * ❌ Customer API keys
  * ❌ Request content (prompts)
  * ❌ Response content (completions)

  We only track metadata needed for cost analytics.
</Warning>

***

## Important Notes

<AccordionGroup>
  <Accordion title="Required vs Optional Headers" icon="info-circle">
    * **Required:** `X-Cloudidr-Token` header must be included in all requests
    * **Optional:** `X-Department`, `X-Team`, `X-Agent` headers are optional metadata for organizing costs
  </Accordion>

  <Accordion title="LiteLLM Proxy Mode" icon="server">
    * LiteLLM proxy mode requires `callback_settings` with `generic_api` type
    * Direct URLs don't work - must use webhook callbacks
    * The `headers` field is required in `callback_settings`, even if just `Content-Type: application/json`
  </Accordion>

  <Accordion title="Database Configuration" icon="database">
    * If you're using LiteLLM's database, keep your existing `database_url` configuration
    * Only set `database_url: ""` and `disable_database_checks: true` if you don't need LiteLLM's database features
  </Accordion>

  <Accordion title="Security Best Practices" icon="shield">
    * For production, use environment variables for API keys and master keys
    * Never commit tokens or keys to version control
    * Rotate tracking tokens regularly
    * Use HTTPS for production deployments
  </Accordion>

  <Accordion title="LiteLLM Tags" icon="tag">
    We currently extract metadata from custom headers (`X-Department`, etc.).

    LiteLLM's built-in `x-litellm-tags` header is not automatically mapped to our metadata fields.

    Use the `X-Department`, `X-Team`, `X-Agent` headers for proper organization.
  </Accordion>
</AccordionGroup>

***

## View Your Data

After making requests, view your costs in the [LLM Ops Dashboard](https://llm-ops.cloudidr.com/dashboard):

* **Agent Explorer** - See costs by agent/application
* **Department Breakdown** - Compare department spending
* **Team Analysis** - Track team-level costs
* **Model Comparison** - Compare costs across models routed through LiteLLM
* **Time Series** - Track spending over time
* **Source Filter** - Filter by source (all LiteLLM requests marked as `litellm`)

***

## LiteLLM Features

LiteLLM provides powerful features that work seamlessly with LLM Ops tracking:

<CardGroup cols={2}>
  <Card title="Multi-Provider Support" icon="layer-group">
    Route to OpenAI, Anthropic, Google, and 100+ providers
  </Card>

  <Card title="Load Balancing" icon="scale-balanced">
    Distribute requests across multiple API keys
  </Card>

  <Card title="Fallback Handling" icon="shield">
    Automatic failover when providers are down
  </Card>

  <Card title="Rate Limiting" icon="gauge">
    Control costs with built-in rate limits
  </Card>
</CardGroup>

All features work with LLM Ops cost tracking - no configuration needed!

***

## Need Help?

<CardGroup cols={2}>
  <Card title="Email Support" icon="envelope" href="mailto:support@cloudidr.com">
    Contact us at [support@cloudidr.com](mailto:hello@cloudidr.com)
  </Card>

  <Card title="Discord Community" icon="discord" href="https://discord.gg/cloudidr">
    Join our Discord for quick help
  </Card>

  <Card title="LiteLLM Docs" icon="book" href="https://docs.litellm.ai">
    Official LiteLLM documentation
  </Card>

  <Card title="View Dashboard" icon="chart-line" href="https://llm-ops.cloudidr.com/dashboard">
    Check your tracked requests
  </Card>
</CardGroup>

***

## Next Steps

<Steps>
  <Step title="Configure LiteLLM">
    Add webhook callback to your config.yaml
  </Step>

  <Step title="Add Headers">
    Include X-Cloudidr-Token in your requests
  </Step>

  <Step title="Monitor Costs">
    View usage and costs in your dashboard
  </Step>

  <Step title="Optimize">
    Use insights to reduce API costs
  </Step>
</Steps>