OpenAI Integration

Overview

Track costs and monitor usage for OpenAI’s API (GPT-4, GPT-3.5, o1) by routing your requests through LLM Ops. This guide shows you how to integrate using Python, JavaScript, or cURL.

Security Guarantee: LLM Ops does not store your API keys, request prompts, or response content in the analytics database—only metadata needed for cost analytics. The proxy must forward request bodies to OpenAI to complete the call; optional operational logging may exist in your deployment environment.

Quick Start

The OpenAI official SDKs expect base_url (or baseURL) to include the /v1 path segment (e.g. https://api.openai.com/v1). Point that at the LLM Ops API host:

Original: https://api.openai.com/v1
LLM Ops (SDK base_url / baseURL): https://api.llm-ops.cloudidr.com/v1
cURL: use the full URL https://api.llm-ops.cloudidr.com/v1/chat/completions

This differs from the Anthropic integration: Claude’s SDK uses a host without /v1; OpenAI’s SDK uses a base URL with /v1.

API Keys

You’ll need two credentials:

OpenAI API Key - Your API key from platform.openai.com
Cloudidr Key - Your tracking token from the LLM Ops dashboard (tokens are typically prefixed with trk_)

The marketing site llmfinops.ai points at the same product; the dashboard URL above is the canonical app host. Set them as environment variables:

export OPENAI_API_KEY="sk-proj-..."
export CLOUDIDR_KEY="trk_..."

Integration Examples

Python
JavaScript
cURL

Install SDK

pip install openai

Basic Example

from openai import OpenAI

# Initialize client with LLM Ops proxy (/v1 is required for the OpenAI SDK)
client = OpenAI(
    api_key="sk-proj-...",  # Your OpenAI API key
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Key": "trk_..."  # Required for cost tracking
    }
)

# Make API call - costs are automatically tracked
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

With Metadata (Department/Team/Agent Tracking)

from openai import OpenAI

client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1"
)

# Track costs by department, team, and agent
# Use X-Project for team/project (X-Team is supported as a legacy alias)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": "Explain async/await in Python"}
    ],
    extra_headers={
        "X-Cloudidr-Key": "trk_...",
        "X-Department": "engineering",
        "X-Team": "backend",
        "X-Agent": "code-tutor"
    }
)

print(response.choices[0].message.content)

Streaming Example

from openai import OpenAI

client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Key": "trk_...",
        "X-Agent": "chat-bot"
    }
)

# Streaming is fully supported
stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Write a haiku about programming"}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Function Calling Example

from openai import OpenAI

client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Key": "trk_...",
        "X-Agent": "weather-assistant"
    }
)

# Function calling with cost tracking
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            },
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "What's the weather in San Francisco?"}
    ],
    tools=tools
)

print(response.choices[0].message.tool_calls)

Install SDK

npm install openai

Basic Example

import OpenAI from 'openai';

// Initialize client with LLM Ops proxy (/v1 is required for the OpenAI SDK)
const client = new OpenAI({
  apiKey: 'sk-proj-...',  // Your OpenAI API key
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Key': 'trk_...'  // Required for cost tracking
  }
});

// Make API call - costs are automatically tracked
const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'What is the capital of France?' }
  ]
});

console.log(response.choices[0].message.content);

With Metadata (Department/Team/Agent Tracking)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-proj-...',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1'
});

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are a helpful coding assistant.' },
    { role: 'user', content: 'Explain async/await in JavaScript' }
  ]
}, {
  headers: {
    'X-Cloudidr-Key': 'trk_...',
    'X-Department': 'engineering',
    'X-Team': 'frontend',
    'X-Agent': 'code-tutor'
  }
});

console.log(response.choices[0].message.content);

Streaming (Node.js)

The snippet below uses process.stdout, which is available in Node.js. In the browser, append chunk content to your UI or a string buffer instead.

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-proj-...',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Key': 'trk_...',
    'X-Agent': 'chat-bot'
  }
});

const stream = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'Write a haiku about programming' }
  ],
  stream: true
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}

Function Calling Example

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-proj-...',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Key': 'trk_...',
    'X-Agent': 'weather-assistant'
  }
});

const tools = [{
  type: 'function',
  function: {
    name: 'get_weather',
    description: 'Get current weather for a location',
    parameters: {
      type: 'object',
      properties: {
        location: { type: 'string' }
      },
      required: ['location']
    }
  }
}];

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: "What's the weather in San Francisco?" }
  ],
  tools: tools
});

console.log(response.choices[0].message.tool_calls);

Basic Example

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Key: trk_..." \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What is the capital of France?"
      }
    ]
  }'

With Metadata (Department/Team/Agent Tracking)

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Key: trk_..." \
  -H "X-Department: engineering" \
  -H "X-Team: backend" \
  -H "X-Agent: code-tutor" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful coding assistant."
      },
      {
        "role": "user",
        "content": "Explain async/await in JavaScript"
      }
    ]
  }'

Streaming Example

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Key: trk_..." \
  -H "X-Agent: chat-bot" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Write a haiku about programming"
      }
    ],
    "stream": true
  }'

Function Calling Example

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Key: trk_..." \
  -H "X-Agent: weather-assistant" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What'\''s the weather in San Francisco?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": { "type": "string" }
            },
            "required": ["location"]
          }
        }
      }
    ]
  }'

Cost Tracking Headers

Add these headers to organize your costs by department, team, or agent:

Header	Description	Example
`X-Cloudidr-Key`	Required - Your Cloudidr tracking token	`trk_abc123...`
`X-Department`	Track costs by department	`engineering`, `sales`, `marketing`, `support`
`X-Project`	Track costs by project/team (preferred)	`backend`, `frontend`, `ml`, `data`, `qa`
`X-Team`	Legacy alias for project/team (same as `X-Project`)	`backend`, `frontend`
`X-Agent`	Track costs by agent/application	`chatbot`, `summarizer`, `analyzer`, `translator`

Supported Models

All OpenAI models are supported. See the Supported Models page for the complete list of available models and pricing.

What Gets Tracked

LLM Ops automatically captures: ✅ Token usage - Prompt, completion, and total tokens
✅ Cost - Real-time cost calculation
✅ Latency - Request duration (TTFT, total time)
✅ Model - Which OpenAI model was used
✅ Metadata - Department, team, agent
✅ Errors - Failed requests and error types
✅ Function calls - Tool/function usage tracking

What We DON’T Track:

❌ Customer API keys
❌ Request content (prompts)
❌ Response content (completions)

We only persist metadata needed for cost analytics in our application database.

View Your Data

After making requests, view your costs in the LLM Ops Dashboard:

Agent Explorer - See costs by agent/application
Department Breakdown - Compare department spending
Team Analysis - Track team-level costs
Model Comparison - Compare costs across models
Time Series - Track spending over time
Cost Optimization - Get recommendations for cheaper models

Migration from Direct API

Switching from direct OpenAI API to LLM Ops is a two-line change:

# Before
client = OpenAI(api_key="sk-proj-...")

# After - add base_url (include /v1) and X-Cloudidr-Key header
client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1",      # ← Add this
    default_headers={"X-Cloudidr-Key": "trk_..."}  # ← Add this
)

Everything else stays the same - no code changes needed!

Cost Optimization Tips

Compare Model Costs in Dashboard

Use the LLM Ops dashboard to identify which agents can switch to cheaper models:

Track cost per request by model
Compare quality vs. cost trade-offs
Identify high-volume, low-complexity tasks

Perfect candidates for model switching appear in the Agent Explorer.

Monitor Function Call Costs

Function calling adds token overhead:

Track function call frequency per agent
Identify redundant or unnecessary calls
Optimize function descriptions to reduce tokens

LLM Ops tracks tool usage separately from chat completion.

Enable Prompt Caching

OpenAI’s prompt caching can reduce costs for repeated system prompts:

Track cache hit rates in dashboard
Identify agents with repeated prompts
Structure prompts for maximum cache benefit

LLM Ops shows cache savings in cost breakdowns when applicable.

Troubleshooting

Requests not being tracked

Check these common issues:

✅ For OpenAI SDKs, set base_url / baseURL to https://api.llm-ops.cloudidr.com/v1 (OpenAI expects the /v1 prefix).
✅ For cURL, call https://api.llm-ops.cloudidr.com/v1/chat/completions.
✅ Confirm the header name is X-Cloudidr-Key (not X-Cloudidr-Token) on every request.
✅ Check that your OpenAI API key is valid.

Authentication errors

Two separate keys are needed:

Your OpenAI API key (for GPT access)
Your Cloudidr tracking token (for cost tracking)

Make sure both are set correctly and not swapped.

Cost data not appearing

Wait a few moments:

Cost data may take 10-30 seconds to appear in dashboard
Check the correct time range in dashboard filters
Verify requests are returning 200 OK status

Next Steps

View Dashboard

See your OpenAI API costs in real-time

Supported Models

View all supported OpenAI models

Anthropic Integration

Add cost tracking for Claude models

Set Budgets

Configure spending alerts and limits

Get Started

LLM Ops

Flex Compute

Overview

Quick Start

API Keys

Integration Examples

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming (Node.js)

Function Calling Example

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Cost Tracking Headers

Supported Models

What Gets Tracked

View Your Data

Migration from Direct API

Cost Optimization Tips

Troubleshooting

Next Steps

View Dashboard

Supported Models

Anthropic Integration

Set Budgets

​Overview

​Quick Start

​API Keys

​Integration Examples

​Install SDK

​Basic Example

​With Metadata (Department/Team/Agent Tracking)

​Streaming Example

​Function Calling Example

​Install SDK

​Basic Example

​With Metadata (Department/Team/Agent Tracking)

​Streaming (Node.js)

​Function Calling Example

​Basic Example

​With Metadata (Department/Team/Agent Tracking)

​Streaming Example

​Function Calling Example

​Cost Tracking Headers

​Supported Models

​What Gets Tracked

​View Your Data

​Migration from Direct API

​Cost Optimization Tips

​Troubleshooting

​Next Steps

View Dashboard

Supported Models

Anthropic Integration

Set Budgets

Overview

Quick Start

API Keys

Integration Examples

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming (Node.js)

Function Calling Example

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Cost Tracking Headers

Supported Models

What Gets Tracked

View Your Data

Migration from Direct API

Cost Optimization Tips

Troubleshooting

Next Steps