OpenAI (GPT) Integration

Overview

Track costs and monitor usage for OpenAI’s API (GPT-4, GPT-3.5, o1) by routing your requests through LLM Ops. This guide shows you how to integrate using Python, JavaScript, or cURL.

Security Guarantee: LLM Ops does NOT store your API keys, request prompts, or response content. We only track metadata needed for cost analytics.

Quick Start

Replace OpenAI’s base URL with LLM Ops proxy URL to automatically track costs:

Original: https://api.openai.com
LLM Ops: https://api.llm-ops.cloudidr.com

API Keys

You’ll need two credentials:

OpenAI API Key - Your API key from platform.openai.com
Cloudidr Token - Your tracking token from llmfinops.ai

Set them as environment variables:

export OPENAI_API_KEY="sk-proj-..."
export CLOUDIDR_TOKEN="cloudidr_..."

Integration Examples

Python
JavaScript
cURL

Install SDK

pip install openai

Basic Example

from openai import OpenAI

# Initialize client with LLM Ops proxy
client = OpenAI(
    api_key="sk-proj-...",  # Your OpenAI API key
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Token": "cloudidr_..."  # Required for cost tracking
    }
)

# Make API call - costs are automatically tracked
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

With Metadata (Department/Team/Agent Tracking)

from openai import OpenAI

client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1"
)

# Track costs by department, team, and agent
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": "Explain async/await in Python"}
    ],
    extra_headers={
        "X-Cloudidr-Token": "cloudidr_...",
        "X-Department": "engineering",
        "X-Team": "backend",
        "X-Agent": "code-tutor"
    }
)

print(response.choices[0].message.content)

Streaming Example

from openai import OpenAI

client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Token": "cloudidr_...",
        "X-Agent": "chat-bot"
    }
)

# Streaming is fully supported
stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Write a haiku about programming"}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Function Calling Example

from openai import OpenAI

client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1",
    default_headers={
        "X-Cloudidr-Token": "cloudidr_...",
        "X-Agent": "weather-assistant"
    }
)

# Function calling with cost tracking
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            },
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "What's the weather in San Francisco?"}
    ],
    tools=tools
)

print(response.choices[0].message.tool_calls)

Install SDK

npm install openai

Basic Example

import OpenAI from 'openai';

// Initialize client with LLM Ops proxy
const client = new OpenAI({
  apiKey: 'sk-proj-...',  // Your OpenAI API key
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Token': 'cloudidr_...'  // Required for cost tracking
  }
});

// Make API call - costs are automatically tracked
const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'What is the capital of France?' }
  ]
});

console.log(response.choices[0].message.content);

With Metadata (Department/Team/Agent Tracking)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-proj-...',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1'
});

// Track costs by department, team, and agent
const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are a helpful coding assistant.' },
    { role: 'user', content: 'Explain async/await in JavaScript' }
  ]
}, {
  headers: {
    'X-Cloudidr-Token': 'cloudidr_...',
    'X-Department': 'engineering',
    'X-Team': 'frontend',
    'X-Agent': 'code-tutor'
  }
});

console.log(response.choices[0].message.content);

Streaming Example

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-proj-...',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Token': 'cloudidr_...',
    'X-Agent': 'chat-bot'
  }
});

// Streaming is fully supported
const stream = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'Write a haiku about programming' }
  ],
  stream: true
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}

Function Calling Example

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-proj-...',
  baseURL: 'https://api.llm-ops.cloudidr.com/v1',
  defaultHeaders: {
    'X-Cloudidr-Token': 'cloudidr_...',
    'X-Agent': 'weather-assistant'
  }
});

// Function calling with cost tracking
const tools = [{
  type: 'function',
  function: {
    name: 'get_weather',
    description: 'Get current weather for a location',
    parameters: {
      type: 'object',
      properties: {
        location: { type: 'string' }
      },
      required: ['location']
    }
  }
}];

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: "What's the weather in San Francisco?" }
  ],
  tools: tools
});

console.log(response.choices[0].message.tool_calls);

Basic Example

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Token: cloudidr_..." \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What is the capital of France?"
      }
    ]
  }'

With Metadata (Department/Team/Agent Tracking)

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Token: cloudidr_..." \
  -H "X-Department: engineering" \
  -H "X-Team: backend" \
  -H "X-Agent: code-tutor" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful coding assistant."
      },
      {
        "role": "user",
        "content": "Explain async/await in JavaScript"
      }
    ]
  }'

Streaming Example

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Token: cloudidr_..." \
  -H "X-Agent: chat-bot" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Write a haiku about programming"
      }
    ],
    "stream": true
  }'

Function Calling Example

curl https://api.llm-ops.cloudidr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-proj-..." \
  -H "X-Cloudidr-Token: cloudidr_..." \
  -H "X-Agent: weather-assistant" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What'\''s the weather in San Francisco?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": { "type": "string" }
            },
            "required": ["location"]
          }
        }
      }
    ]
  }'

Cost Tracking Headers

Add these headers to organize your costs by department, team, or agent:

Header	Description	Example
`X-Cloudidr-Token`	Required - Your CloudIDR tracking token	`cloudidr_abc123...`
`X-Department`	Track costs by department	`engineering`, `sales`, `marketing`, `support`
`X-Team`	Track costs by team	`backend`, `frontend`, `ml`, `data`, `qa`
`X-Agent`	Track costs by agent/application	`chatbot`, `summarizer`, `analyzer`, `translator`

Supported Models

All OpenAI models are supported. See the Supported Models page for the complete list of available models and pricing.

What Gets Tracked

LLM Ops automatically captures: ✅ Token usage - Prompt, completion, and total tokens
✅ Cost - Real-time cost calculation
✅ Latency - Request duration (TTFT, total time)
✅ Model - Which OpenAI model was used
✅ Metadata - Department, team, agent
✅ Errors - Failed requests and error types
✅ Function calls - Tool/function usage tracking

What We DON’T Track:

❌ Customer API keys
❌ Request content (prompts)
❌ Response content (completions)

We only track metadata needed for cost analytics.

View Your Data

After making requests, view your costs in the LLM Ops Dashboard:

Agent Explorer - See costs by agent/application
Department Breakdown - Compare department spending
Team Analysis - Track team-level costs
Model Comparison - Compare costs across models
Time Series - Track spending over time
Cost Optimization - Get recommendations for cheaper models

Migration from Direct API

Switching from direct OpenAI API to LLM Ops is a two-line change:

# Before
client = OpenAI(api_key="sk-proj-...")

# After - add base_url and X-Cloudidr-Token header
client = OpenAI(
    api_key="sk-proj-...",
    base_url="https://api.llm-ops.cloudidr.com/v1",      # ← Add this
    default_headers={"X-Cloudidr-Token": "cloudidr_..."}  # ← Add this
)

Everything else stays the same - no code changes needed!

Cost Optimization Tips

Compare Model Costs in Dashboard

Use the LLM Ops dashboard to identify which agents can switch to cheaper models:

Track cost per request by model
Compare quality vs. cost trade-offs
Identify high-volume, low-complexity tasks

Perfect candidates for model switching appear in the Agent Explorer.

Monitor Function Call Costs

Function calling adds token overhead:

Track function call frequency per agent
Identify redundant or unnecessary calls
Optimize function descriptions to reduce tokens

LLM Ops tracks tool usage separately from chat completion.

Enable Prompt Caching

OpenAI’s prompt caching can reduce costs by 50% for repeated system prompts:

Track cache hit rates in dashboard
Identify agents with repeated prompts
Structure prompts for maximum cache benefit

LLM Ops shows cache savings in cost breakdowns.

Troubleshooting

Requests not being tracked

Check these common issues:

✅ Verify base URL is https://api.llm-ops.cloudidr.com/v1
✅ Confirm X-Cloudidr-Token header is included in all requests
✅ Check that your OpenAI API key is valid
✅ Ensure you’re using /v1 in the endpoint path

Authentication errors

Two separate keys are needed:

Your OpenAI API key (for GPT access)
Your CloudIDR token (for cost tracking)

Make sure both are set correctly and not swapped.

Cost data not appearing

Wait a few moments:

Cost data may take 10-30 seconds to appear in dashboard
Check the correct time range in dashboard filters
Verify requests are returning 200 OK status

Next Steps

View Dashboard

See your OpenAI API costs in real-time

Supported Models

View all supported OpenAI models

Anthropic Integration

Add cost tracking for Claude models

Set Budgets

Configure spending alerts and limits

Get Started

Flex Compute

LLM Ops

Overview

Quick Start

API Keys

Integration Examples

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Cost Tracking Headers

Supported Models

What Gets Tracked

View Your Data

Migration from Direct API

Cost Optimization Tips

Troubleshooting

Next Steps

View Dashboard

Supported Models

Anthropic Integration

Set Budgets

Get Started

Flex Compute

LLM Ops

​Overview

​Quick Start

​API Keys

​Integration Examples

​Install SDK

​Basic Example

​With Metadata (Department/Team/Agent Tracking)

​Streaming Example

​Function Calling Example

​Install SDK

​Basic Example

​With Metadata (Department/Team/Agent Tracking)

​Streaming Example

​Function Calling Example

​Basic Example

​With Metadata (Department/Team/Agent Tracking)

​Streaming Example

​Function Calling Example

​Cost Tracking Headers

​Supported Models

​What Gets Tracked

​View Your Data

​Migration from Direct API

​Cost Optimization Tips

​Troubleshooting

​Next Steps

View Dashboard

Supported Models

Anthropic Integration

Set Budgets

Overview

Quick Start

API Keys

Integration Examples

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Install SDK

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Basic Example

With Metadata (Department/Team/Agent Tracking)

Streaming Example

Function Calling Example

Cost Tracking Headers

Supported Models

What Gets Tracked

View Your Data

Migration from Direct API

Cost Optimization Tips

Troubleshooting

Next Steps