### November 26, 2025

[**Claude Sonnet 4 and Sonnet 4.5 now support 1M context window**](/content/changelog/20251126-sonnet-4-1m-context/index.html)

Claude Sonnet 4 and Claude Sonnet 4.5 models on the AI Gateway now support 1M context window by default.

## What’s Changed

- **1M Context Window**: Sonnet 4 and Sonnet 4.5 models now use the extended 1M token context window by default
- **All Providers**: This update applies to Anthropic API, AWS Bedrock, and Google Vertex AI

No configuration changes are needed - requests to Sonnet 4 models will automatically use the extended context window.

### August 13, 2025

[**Control Reasoning Effort in Playground and better feedback on thinking models**](/content/changelog/20250813-gpt-5-playground/index.html)

You can now fine-tune how models think and respond with the reasoning effort parameter.

Your browser does not support the video tag.

## What’s New

- Reasoning Effort Control
  - **Minimal Reasoning**: Added “minimal” option for faster, more direct responses
  - **Low/Medium/High Reasoning**: Existing reasoning levels for different complexity needs
- **Visual Reasoning Display**: See the model’s thinking process when available

Try the new reasoning controls in the [Playground](/content/playground/index.html) to see how different settings affect model responses.

## Coming Soon

- Ability to set verbosity (low, medium, high) in the playground for GPT-5 models

### August 7, 2025

[**OpenAI GPT-5 Models Pricing and Playground Support**](/content/changelog/20250807-gpt-5-pricing/index.html)

You can now use GPT-5 models in the Playground with full cost tracking.

## What’s New (Models)

- **GPT-5**: Added pricing for GPT-5 models across multiple providers.
- **GPT-5-Mini**: Added pricing for GPT-5-Mini models across multiple providers.
- **GPT-5-Nano**: Added pricing for GPT-5-Nano models across multiple providers.
- **GPT-5-Chat-Latest**: Added pricing for GPT-5-Chat-Latest models across multiple providers.

Teams can now track GPT-5 costs and also use them in the Playground.

### August 5, 2025

[**OpenAI GPT OSS Models Pricing**](/content/changelog/20250805-gpt-oss-pricing/index.html)

Cost tracking is now available for GPT-OSS models across multiple providers.

## What’s New

- **Fireworks Models**: `accounts/fireworks/models/gpt-oss-20b` and `accounts/fireworks/models/gpt-oss-120b`
- **Groq Models**: `openai/gpt-oss-20b` and `openai/gpt-oss-120b`
- **OpenRouter Models**: `openai/gpt-oss-20b` and `openai/gpt-oss-120b`
- **Automatic Detection**: Costs calculated automatically when using GPT-OSS models
- **Dashboard Integration**: View GPT-OSS usage alongside other models in your cost analytics

With this update, teams using GPT-OSS models can track their spending across Fireworks, Groq, and OpenRouter providers and compare costs with other AI models.

### August 5, 2025

[**Claude Opus 4.1 Pricing**](/content/changelog/20250805-claude-opus-4-1-pricing/index.html)

Cost tracking is now available for Anthropic’s Claude Opus 4.1 model.

## What’s New

- **Models**: `claude-opus-4-1-20250805` and `anthropic/claude-opus-4.1` (OpenRouter)
- **Automatic Detection**: Costs calculated automatically when using Claude Opus 4.1 models
- **Dashboard Integration**: View Claude Opus 4.1 usage alongside other models in your cost analytics

With this update, teams using Anthropic’s Claude Opus 4.1 model can track their spending and compare costs across different AI providers.

### July 22, 2025

[**Prompt Management V2**](/content/changelog/20250722-prompts-v2/index.html)

Iterate on prompts without code deployments. Our new Prompt Management system brings powerful composability, version control, and instant deployment to your LLM workflows.

## Features

- **Powerful Composability**: Use typed variables anywhere - system prompts, messages, even tool schemas
- **Version Control**: Track, compare, and rollback prompt versions without code changes
- **Instant Deployment**: Reference prompts by ID through our AI Gateway - no rebuilds required
- **Real-time Testing**: Experiment with different models and parameters in the Playground
- **Dynamic Schemas**: Variables work within JSON schemas for tools and response formatting
- **TypeScript Support**: Full type safety with our helper types

## How It Works

Save prompts in our Playground with variables like `{{hc:customer_name:string}}`. Test them with different inputs and models. When ready, reference the prompt ID in your API calls through the AI Gateway:

```typescript
const response = await openai.chat.completions.create({
  model: "openai/gpt-4o-mini",
  prompt_id: "abc123",
  inputs: {
    customer_name: "John Doe",
    product: "AI Gateway"
  }
});
```

The AI Gateway compiles your saved prompt with runtime inputs and sends it to your chosen model. Update prompts in the dashboard and changes take effect immediately - no code changes or deployments needed.

## Get Started

Visit the [Prompts page](/content/prompts/index.html) in your dashboard to create your first prompt, or check out our [documentation](https://docs.helicone.ai/features/advanced-usage/prompts) for detailed examples and best practices.

### July 16, 2025

[**Improved Timezone Display for Requests**](/content/changelog/20250716-timezone-display/index.html)

We’ve improved how timestamps are displayed throughout the Helicone dashboard to provide a better user experience for teams working across different timezones.

## What’s New

- **Automatic Timezone Detection**: Requests now display in your local timezone automatically
- **Clear Timezone Indicators**: Timestamps now clearly show the timezone being used
- **Consistent Formatting**: Improved date/time formatting across all views

This update makes it easier for distributed teams to collaborate and analyze their LLM usage patterns without timezone confusion.

### July 15, 2025

[**Country-Based Request Filtering**](/content/changelog/20250715-country-filtering/index.html)

Analyze your LLM usage patterns by geographic location with our new country-based filtering feature.

## Features

- **Country Code Filtering**: Filter requests by country of origin
- **Geographic Analytics**: Understand where your API calls are coming from
- **Global Usage Patterns**: See how your application is being used around the world

## How It Works

The country code is determined by the Cloudflare edge server that receives your request. Since Helicone runs on Cloudflare’s global network, we can identify which region processed each request, giving you insights into where your users are located.

## How to Use

In the requests dashboard, you can now:

1. Use the new country filter in the advanced filters section
2. Add `country_code:US` (or any ISO country code) to your search queries
3. View geographic distribution in the analytics dashboard

This helps you understand your global usage patterns and user distribution. Country data is automatically collected for all new requests going forward.

### July 14, 2025

[**GPT-4o Search Model Pricing**](/content/changelog/20250714-gpt4o-search-pricing/index.html)

We’ve added pricing support for OpenAI’s new GPT-4o Search model, which includes built-in web search capabilities.

## What’s New

- **Model**: `gpt-4o-search`
- **Automatic Detection**: Helicone automatically detects and applies correct pricing when you use this model
- **Cost Tracking**: View search-enabled model costs separately in your dashboard

The GPT-4o Search model combines GPT-4o’s capabilities with real-time web search. With this update, you can accurately track costs for search-enhanced conversations and compare them with standard GPT-4o usage.

### July 12, 2025

[**Mistral Model Pricing Updates**](/content/changelog/20250712-mistral-pricing-update/index.html)

We’ve updated pricing for all Mistral models to reflect their current rates and added support for new model variants.

## What’s Updated

- **Chat Models**: `mistral-tiny`, `mistral-small`, `mistral-medium`, `mistral-large` (including `-latest` variants)
- **Instruction Models**: `mistral-7b-instruct`, `mixtral-8x7b-instruct`, `mixtral-8x22b-instruct`
- **Embedding Model**: `mistral-embed`
- **Historical Data**: Previous costs remain unchanged for accurate reporting

This ensures your cost tracking stays accurate as Mistral’s model offerings and pricing evolve.

### July 11, 2025

[**Grok 4 Model Pricing**](/content/changelog/20250711-grok4-pricing/index.html)

Cost tracking is now available for xAI’s Grok 4 model.

## What’s New

- **Models**: `grok-4` and `x-ai/grok-4` (OpenRouter)
- **Automatic Detection**: Costs calculated automatically when using Grok models
- **Dashboard Integration**: View Grok 4 usage alongside other models in your cost analytics

With this update, teams using xAI’s Grok models can track their spending and compare costs across different AI providers.

### June 19, 2025

[**Helicone AI Gateway - Now Available!**](/content/changelog/20250619-ai-gateway-launch/index.html)

We’re thrilled to announce the launch of **Helicone AI Gateway** \- a powerful open-source solution for routing, caching, and managing your LLM traffic at scale.

## 🚀 What is Helicone AI Gateway?

The AI Gateway is a high-performance proxy that sits between your application and LLM providers, offering enterprise-grade features:

- **Smart Load Balancing**: Distribute requests across multiple providers
- **Intelligent Caching**: Reduce costs with semantic caching
- **Automatic Failover**: Seamlessly switch providers during outages
- **Rate Limiting**: Protect against abuse and control costs
- **Built-in Observability**: Full integration with Helicone’s analytics

## 💻 Get Started

The AI Gateway is available as a separate open-source project:

**GitHub Repository**: [github.com/helicone/ai-gateway](https://github.com/helicone/ai-gateway)

Quick start with Docker:

```bash
docker run -p 8080:8080 helicone/ai-gateway
```

## 📚 Learn More

- [Read the documentation](https://docs.helicone.ai/ai-gateway/introduction)
- [View the quickstart guide](https://docs.helicone.ai/ai-gateway/quickstart)
- [Explore configuration options](https://docs.helicone.ai/ai-gateway/config)

## 🔧 Key Features

- **Multi-Provider Support**: OpenAI, Anthropic, Azure, and more
- **Request Routing**: Route by model, cost, or custom rules
- **Security**: API key management and request validation
- **Performance**: Built in Rust for minimal latency overhead

Start using the AI Gateway today to take control of your LLM infrastructure!

### May 28, 2025

[**Introducing Go Helicone Helpers Package**](/content/changelog/20250528-go-helicone-helpers/index.html)

We’re thrilled to announce that we now have a [Go SDK for Helicone’s Helpers Package](https://github.com/helicone/go-helicone-helpers). You can now integrate any custom LLM with Helicone using the Go Manual Logger. Install it now with:

```
go get github.com/helicone/go-helicone-helpers
```

Refer to our [docs](https://docs.helicone.ai/getting-started/integration-method/manual-logger-go) on how to use it.

### May 6, 2025

[**Introducing Smarter Sessions Design**

We’ve introduced a new design for Sessions, now with session-level metrics, smarter filters, and a cleaner UI.

The new Sessions include **average latency**, **total cost**, and **improved time filtering**, making it easier to debug multi-step LLM workflows. The UI has also been refined for a cleaner and more intuitive experience.

Try it out in [Sessions](https://us.helicone.ai/sessions), we’d love to hear your feedback!

Refer to our [docs](https://docs.helicone.ai/docs/sessions) on how to set up Sessions to start tracing your LLM workflows.

### May 5, 2025

[**Introducing Helicone Self-Hosting**

We’re excited to re-introduce **Helicone self-hosting**! Now you can deploy our powerful observability platform directly within your own infrastructure with a single Docker command.

This update addresses the needs of organizations with strict security and compliance requirements, allowing you to leverage Helicone’s insights into your LLM usage without sending sensitive data to third-party servers.

### Why self-host?

- **Complete Data Sovereignty**: Keep all LLM interactions and analytics within your security perimeter.
- **Protection of Intellectual Property**: Ensure proprietary prompts and model data never leave your infrastructure.
- **Customized Integration**: Seamlessly integrate with your internal systems and security protocols.
- **Cost Efficiency at Scale**: Favorable economics for high-volume LLM request processing.

Get started with a single Docker command:

```bash
git clone https://github.com/Helicone/helicone.git

# See your Helicone dashboard in localhost:3000!
cd docker
docker compose up -d
```

Read more about our self-hosting offering in our [blog](/content/blog/self-hosting-launch/index.html).

### April 23, 2025

[**Added support for OpenAI's new Responses API**

Helicone now supports OpenAI’s new Responses API, allowing you to monitor and analyze your model’s responses seamlessly.

### What is the OpenAI Responses API?

The OpenAI Responses API enables you to provide text or image inputs to generate text or JSON outputs by calling your own custom code or using built-in tools like web search or file search.

### Integrating with Helicone

1. Create a Helicone account and API key.
2. Set environment variables
3. Install the OpenAI SDK:

```bash
npm install openai
```

4. Configure your OpenAI client to use the Helicone proxy:

```typescript
import OpenAI from "openai";

const openai = new OpenAI({
     apiKey: process.env.OPENAI_API_KEY,
     baseURL: "https://oai.helicone.ai/v1", // Route requests through Helicone
     defaultHeaders: {
       "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`, // Authenticate with Helicone
     },
});
```

With this setup, any calls to the OpenAI Responses API will be automatically logged and monitored by Helicone.

For more examples, including file search, streaming, function calling, and reasoning, please refer to our [documentation](https://docs.helicone.ai/integrations/openai/responses).

### April 22, 2025

[**Introducing Open WebUI Integration**

Today, you can start monitoring your local LLM interactions with Open WebUI in Helicone!

### With the integration, you can:

- Monitor interactions across Ollama, OpenAI-compatible APIs, and custom LLM setups.
- Get a consolidated view across model types.
- Visualize and replay requests to see prompts and outputs for evaluation.
- Track local LLM performance, including response times and throughput.
- Analyze usage patterns by model in your Open WebUI setup.

We’ve just published a [comprehensive guide](/content/blog/monitoring-local-llms/index.html) on how to integrate Helicone with Open WebUI (formerly Ollama WebUI). This will show you how to gain comprehensive observability across all your LLM interactions, whether you’re using local Ollama models or cloud LLM APIs.

Get started: [Open WebUI x Helicone docs](https://docs.helicone.ai/other-integrations/open-webui).

### April 18, 2025

[**April 2025 Changelog**](/content/changelog/20250418-april-changelog/index.html)

Helicone is the first observability platform to support OpenAI’s Realtime API, the 4.1 model family, and more.

### What’s New?

- Added cost support for gpt-4.1, gpt-4.1-mini, and gpt-4.1-nano, including detailed token pricing (prompt, completion, and cache read).
- Redesigned the [Request Drawer](/content/requests/index.html) for better viewing of requests, responses, and key metrics.
- Added support for gpt-4.1 model family in Prompts.
- Added support for streaming tool use with Anthropic models.
- Added stop sequences parameter support in the Parameters panel.
- Added Tools Editor with streaming support when creating new tools.
- Enabled evaluation runs for free tier users!
- Added ability to delete datasets with confirmation modals and user notifications.
- Added support for streaming tool use with Anthropic models.

### Improvements

- Improved overall page load and view performance.
- Improved dataset page loading and reliability for request/response fetching.
- Improved datetime filtering with a calendar and time picker input.
- Updated backend logging for smarter and more traceable logs.
- Updated cache strategy to reduce database load during property filtering.
- Updated the API cost calculator and model comparison pages with better design and more consistent pricing information.
- Added a “Response Format Editor” component to allow users to specify JSON schema formats for responses.

### Bug Fixes

- Fixed streamed requests for Claude to see the entire response.
- Fixed loading prompts from requests with tool calls.
- Fixed table responsiveness issues in blogs.
- Fixed an issue with member invites for certain organization tiers.
- Fixed an issue where member invites didn’t properly check organization tiers.

### April 7, 2025

[**Helicone Now Supports OpenAI's Realtime API**

We’re thrilled to announce that Helicone now supports logging of OpenAI’s Realtime API, enabling low-latency, multi-modal conversational experiences.

### Seamless Integration with Helicone

Integrating OpenAI’s Realtime API with Helicone is as simple as ever. Following our standard one-line integration approach, you can immediately start monitoring performance, analyzing interactions, and gaining valuable insights into your real-time conversations.

### How it Works

Connect to the Realtime API through Helicone using your preferred provider (OpenAI or Azure). Helicone acts as a proxy, allowing you to leverage our observability features without changing your core application logic.

### Example: Connecting via WebSocket (OpenAI Provider)

```typescript
// Simply swap with the following url:
const url =
  "wss://api.helicone.ai/v1/gateway/oai/realtime?model=gpt-4o-realtime-preview-2024-12-17";

const ws = new WebSocket(url, {
  headers: {
    // Your OpenAI Key
    Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    // Your Helicone Key
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
    // Optional Helicone properties for enhanced analytics
    "Helicone-Session-Id": `session_123`,
    "Helicone-User-Id": "user_123",
  },
});
```

### Key Benefits

- **Effortless Setup**: Integrate with the standard Helicone proxy URL and API key header.
- **Real-time Monitoring**: Track latency, token usage, and other critical metrics for your real-time sessions.
- **Session Analysis**: Utilize Helicone headers like `Helicone-Session-Id` and `Helicone-User-Id` to group and analyze conversations.
- **Multi-modal Support**: Monitor both text and audio interactions.
- **Provider Flexibility**: Works seamlessly with both OpenAI and Azure endpoints.

Get started today by updating your WebSocket connection URL and adding your Helicone API key header. For more details, check out the full [OpenAI Realtime Integration documentation](https://docs.helicone.ai/integrations/openai/realtime).

### March 29, 2025

[**Filter UI Update: A New Look with Enhanced Features**

A fresh new look for our filtering system brings enhanced performance and usability to your data exploration experience. The redesigned filter interface is now more intuitive and powerful than ever, making it easier to analyze your LLM usage data.

## What’s new

- **Unified filter component**: All filter elements are now integrated into a single, cohesive component for a more streamlined experience
- **Persistent filters**: Your active filters now persist across different pages - from dashboard to requests, users, and sessions
- **URL sharing support**: Share your filtered views with team members by simply copying the URL
- **Enhanced performance**: The new implementation is more efficient, providing faster filter operations

## Try it out

Experience the new filtering system across all main pages:

- Dashboard analytics
- Request logs
- User metrics
- Session tracking

We’re excited to hear your feedback on these improvements! Let us know what you think about the new filtering experience.

### March 28, 2025

[**Online Evaluators New UI**

We’ve completely redesigned the online evaluators creation and editing experience, making it more intuitive and powerful than ever. The new interface guides developers through the process with real-time feedback and advanced configuration options.

## What’s new

- **Streamlined form layout**: A cleaner, more organized and intuitive interface
- **Advanced configuration**: More granular control over your evaluator settings
- **Real-time feedback**: Helpful validation messages to guide you through the setup
- **Enhanced user experience**: Every step of creating evaluators is now more straightforward

## Getting started

1. Navigate to the [Evaluators](https://us.helicone.ai/evaluators) tab in Helicone
2. Click “Create New Evaluator” or edit an existing one
3. Create/edit an evaluator with the new interface with real-time feedback

We’re continuously working to make Helicone’s features more accessible and powerful. Share your thoughts on the new evaluators interface - we’d love to hear your feedback!

### March 27, 2025

[**Vertex AI Gemini Support**](/content/changelog/20250327-vertex-ai-gemini-support/index.html)

We just added support for Vertex AI Gemini integration with Helicone logging framework, including stream handling and logging capabilities for both Python and TypeScript SDKs.

There are two ways to integrate Helicone with Vertex AI’s Python SDK, Proxy and Manual Logger. The manual logger provides more granular control over logging the LLM’s responses.

## Manual Logger Key Capabilities

1. Log both request parameters and responses with full context.
2. Handle streaming responses with proper chunk logging.
3. Built-in error tracking and logging.
4. Add custom metadata to your logs
5. Explicitly specify which model is being used for better analytics.
6. Automatic tracking of streaming response timing.

For integration guide and advanced configuration options, please refer to our [Vertex AI documentation](https://docs.helicone.ai/integrations/gemini/vertex/python).

### March 26, 2025

[**Groq Streaming Support: Enhanced Token Tracking**](/content/changelog/20250326-groq-streaming/index.html)

We’re excited to announce support for Groq streaming tokens in Helicone! This update enhances our integration with Groq’s API, providing better token tracking and usage monitoring for streaming responses.

## What’s new

- Full support for tracking and monitoring Groq’s streaming token responses
- Accurate tracking of prompt, completion, and total tokens
- Support for both standard and Groq-specific usage reporting formats
- Fallback mechanisms ensure consistent token tracking even with varying response formats

## Getting started

To use Groq streaming with Helicone:

1. Configure your [Helicone proxy settings for Groq](https://docs.helicone.ai/integrations/groq/)
2. Make streaming requests as usual through the Helicone proxy
3. Monitor your token usage in the Helicone dashboard

### March 23, 2025

[**Introducing LangGraph Integration: Monitor Your AI Agent Workflows**](/content/changelog/20250323-langgraph-vs-helicone-integration/index.html)

Helicone now integrates seamlessly with LangGraph, providing comprehensive observability for your graph-based AI agents and tools. With minimal configuration, you can get full visibility into your LangGraph workflows, from basic chains to complex multi-agent systems.

## Capabilties

LangGraph has emerged as a powerful framework for building stateful, multi-agent LLM applications. Our integration gives you:

- Complete visibility into every LLM call within your graph
- Session tracking for understanding full conversation flows
- Performance metrics to identify bottlenecks in complex agent interactions
- Cost monitoring across your entire agent ecosystem

## Supported models

The integration works with all major LLM providers supported by LangChain, including:

- OpenAI
- Anthropic
- And more to come!

For implementation details, check out our [LangGraph integration guide](https://docs.helicone.ai/other-integrations/langgraph).

### March 5, 2025

[**New HeliconeLogBuilder for Improved Stream Handling**](/content/changelog/20250305-helicone-log-builder/index.html)

We’re excited to introduce the `HeliconeLogBuilder`, a new approach to handling streaming responses with better error handling and a simplified workflow.

### Introducing HeliconeLogBuilder

The new `HeliconeLogBuilder` class provides a simplified way to handle streaming LLM responses with improved error handling and async support. This new approach makes it easier to:

- Handle errors gracefully with the `setError` method
- Simplify stream handling with the `toReadableStream` method
- Use more flexible async/await patterns with `sendLog`
- Track proper error status codes

### Example Usage with Next.js App Router

```typescript
import { HeliconeManualLogger } from "@helicone/helpers";
import { after } from "next/server";
import Together from "together-ai";

const together = new Together();
const helicone = new HeliconeManualLogger({
  apiKey: process.env.HELICONE_API_KEY!,
});

export async function POST(request: Request) {
  const { question } = await request.json();
  const body = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: question }],
    stream: true,
  };

const heliconeLogBuilder = helicone.logBuilder(body, {
    "Helicone-Property-Environment": "dev",
  });

try {
    const response = await together.chat.completions.create(body);
    return new Response(heliconeLogBuilder.toReadableStream(response));
  } catch (error) {
    heliconeLogBuilder.setError(error);
    throw error;
  } finally {
    after(async () => {
      // This will be executed after the response is sent to the client
      await heliconeLogBuilder.sendLog();
    });
  }
}
```

### Benefits Over Previous Methods

The `logBuilder` approach offers several advantages over the previous streaming methods:

- **Better Error Handling**: Automatically captures and logs errors with the `setError` method
- **Simplified Stream Handling**: Combines stream processing and logging in one step
- **More Flexible Async Patterns**: Better support for async workflows with the `sendLog` method
- **Proper Status Code Tracking**: Accurately records HTTP status codes, including errors
- **Improved Time-to-First-Token Metrics**: More reliable tracking of streaming performance

This new approach is now the recommended way to handle streaming responses with Helicone. For more information, check out our [Manual Logger with Streaming](https://docs.helicone.ai/guides/cookbooks/manual-logger-streaming) guide.

### March 4, 2025

[**Property Page Enhancements: Average Prompt Tokens & Sorting**

We’ve enhanced the Properties pages with new metrics and improved user experience:

- **Average Prompt Tokens**: Added average prompt tokens metrics to property tables, providing deeper insights into token usage patterns.
- **Backend Sorting**: Implemented server-side sorting for property data, enabling efficient sorting of large datasets by any column.
- **Improved Loading Experience**: Replaced generic loading animations with skeleton UI components that match the structure of the loaded content.
- **Responsive Skeletons**: Added custom skeleton loaders for property tables, metrics cards, and pie charts that adapt to both light and dark modes.

These enhancements make it easier to analyze property data by providing more metrics and a smoother, more responsive user interface. The skeleton loading states significantly reduce the perception of loading time and prevent jarring layout shifts when data loads.

The backend sorting implementation ensures that sorting large datasets is efficient and performant, as the sorting happens at the database level rather than in the browser.

### March 2, 2025

[**Webhook Enhancement: Combined Request/Response URL**](/content/changelog/20250302-webhook-enhancements/index.html)

We’ve enhanced our webhook functionality to improve integration with external systems:

- **Combined Request/Response URL**: Replaced separate URLs with a single `request_response_url` field that contains both request and response data in one object.
- **Performance Optimization**: Reduced S3 URL generation calls from two to one per webhook.
- **Configurable Data Inclusion**: Added `includeData` parameter to the webhook API to control whether enhanced data is included in payloads.
- **Updated Documentation**: Added examples showing how to work with the combined data structure.

These changes make webhook integration more efficient while maintaining backward compatibility with existing implementations.

For details, see our [updated documentation](https://docs.helicone.ai/features/webhooks#working-with-the-combined-requestresponse-url).

### February 27, 2025

[**Improved Streaming Support and Async Stream Parser**](/content/changelog/20250227-stream-fixes/index.html)

## Improved Streaming Support and Async Stream Parser

We’ve made significant improvements to our streaming functionality with two key updates:

### Stream Fixes

We’ve resolved several issues with stream handling across different LLM providers, ensuring more reliable and consistent streaming experiences. These fixes address edge cases and improve compatibility with various streaming implementations, including:

- Better handling of stream interruptions and reconnections
- Improved error handling for streaming responses
- Enhanced compatibility with different LLM provider streaming formats
- Fixed timing calculations for streamed responses

### New Streaming Methods

The `HeliconeManualLogger` class now includes enhanced methods for working with streams:

- `logStream`: Logs a streaming operation with full control over stream handling
- `logSingleStream`: Simplified method for logging a single ReadableStream
- `logSingleRequest`: Logs a single request with a response body

### Example Usage with Together AI

```typescript
import Together from "together-ai";
import { HeliconeManualLogger } from "@helicone/helpers";

// Initialize with properties
const helicone = new HeliconeManualLogger({
  apiKey: process.env.HELICONE_API_KEY!,
  loggingEndpoint: "https://api.worker.helicone.ai/oai/v1/log",
  headers: {
    "Helicone-Property-Environment": "production",
  },
});

export async function POST(request: Request) {
  const { question } = await request.json();

const body = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: question }],
    stream: true,
  } as Together.Chat.CompletionCreateParamsStreaming & { stream: true };

const response = await together.chat.completions.create(body);
  const [stream1, stream2] = response.tee();
  helicone.logStream(
    body,
    async (resultRecorder) => {
      resultRecorder.attachStream(stream2.toReadableStream());
    },
    {
      "Helicone-User-Id": "123",
    }
  );

return new Response(stream1.toReadableStream());
}
```

These improvements make working with streaming LLMs more reliable and efficient, especially for applications that require real-time responses.

### February 26, 2025

[**Complete Logging Control for Async Integration**](/content/changelog/20250226-disabled-logging-async/index.html)

We’ve enhanced our Python Asynchronous Logging integration with new methods to completely disable all logging to Helicone. This feature allows you to temporarily or permanently stop sending any data to our backend while maintaining your integration setup.

### New Methods

- `disable_logging()`: Completely disables all logging by shutting down the Traceloop SDK
- `enable_logging()`: Re-enables logging if it was previously disabled

This is different from the existing `disable_content_tracing()` which only omits request and response content but still sends other metrics. The new functionality gives you complete control over your data privacy and is only available when using Helicone’s async integration mode.

```python
# Example usage
from helicone_async import HeliconeAsyncLogger
from openai import OpenAI

logger = HeliconeAsyncLogger(api_key=HELICONE_API_KEY)
logger.init()

# Completely disable all logging
logger.disable_logging()

# Your OpenAI calls here - no data sent to Helicone

# Later, re-enable logging if needed
logger.enable_logging()
```

### February 25, 2025

[**Enhanced Cost Sorting and Data Organization**](/content/changelog/20250225-update-sort/index.html)

Improved sorting capabilities across the platform, particularly for cost request pages. This update ( [#3326](https://github.com/Helicone/helicone/pull/3326)) makes it easier to organize and analyze your data with more intuitive sorting controls.

### February 24, 2025

[**New Claude 3.7 Sonnet Support: Full Cost Tracking and Integration**

## Immediate Support for Claude 3.7 Sonnet

We’re excited to announce full support for Anthropic’s latest Claude 3.7 Sonnet model (claude-3-7-sonnet-20250219), including comprehensive cost tracking and monitoring capabilities.

### What’s New in Claude 3.7?

Claude 3.7 Sonnet represents Anthropic’s latest advancement in language AI, introducing groundbreaking hybrid reasoning capabilities. As [announced by Anthropic](https://www.anthropic.com/news/claude-3-7-sonnet), this model offers:

#### Core Capabilities

- First hybrid reasoning model combining quick responses and extended thinking modes
- Up to 128K token output limit for handling extensive conversations
- Same competitive pricing: 3permillioninputtokensand3 per million input tokens and 3permillioninputtokensand15 per million output tokens (including thinking tokens)

#### Enhanced Performance

- State-of-the-art results on SWE-bench Verified and TAU-bench
- Best-in-class coding and software development capabilities
- Advanced reasoning and analytical processing
- Improved instruction-following and task completion reliability

#### Real-world Applications

The model has demonstrated exceptional performance across:

- Complex codebase analysis and management
- Advanced tool usage and automation
- Full-stack development and web applications
- Production-ready code generation with reduced errors
- Large-scale refactoring and debugging

### Cost Tracking and Integration

Our platform provides comprehensive monitoring with:

#### Usage Analytics

- Full cost tracking for both standard and extended thinking modes
- Precise token counting for inputs and outputs
- Real-time usage monitoring and analytics
- Updated pricing calculator with latest model rates

#### Integration Features

- Seamless integration with existing Anthropic workflows
- Support for both API and direct integration paths
- Comprehensive request-level analytics
- Advanced monitoring dashboards

### How to Use

To start using Claude 3.7 Sonnet with Helicone:

1. Update your model parameter to `claude-3-7-sonnet-20250219`
2. Refer to our [Anthropic Integration Guide](https://docs.helicone.ai/integrations/anthropic/javascript) for implementation details
3. Monitor costs and usage through your Helicone dashboard

### Supported Features

- Real-time cost and usage tracking
- Extended thinking mode support and monitoring
- Comprehensive token usage analytics
- Full integration with existing workflows
- Advanced request-level insights

[Learn More About Claude 3.7 Sonnet](https://www.anthropic.com/news/claude-3-7)

### February 14, 2025

[**Introducing Auto-Improve for Prompts**](/content/changelog/20250214-auto-improve-beta/index.html)

We’re excited to launch **Auto-Improve**, an intelligent prompt optimization tool that helps you write more effective LLM prompts. While traditional prompt engineering requires extensive trial and error, Auto-Improve analyzes your prompts and suggests improvements instantly.

## How it Works

1. Click the Auto-Improve button in the Helicone Prompt Editor
2. Our AI analyzes each sentence of your prompt to understand:

- The semantic interpretation
- Your instructional intent
- Potential areas for enhancement

3. Get a new suggested optimized version of your prompt

## Key Benefits

- **Semantic Analysis**: Goes beyond simple text improvements by understanding the purpose behind each instruction
- **Maintains Intent**: Preserves your original goals while enhancing how they’re communicated
- **Time Saving**: Skip hours of prompt iteration and testing
- **Learning Tool**: Understand what makes an effective prompt by comparing your original with the improved version

Auto-Improve is now available in beta for all Helicone users. Try it today with [one of your prompts](/content/prompts/index.html)!

### February 14, 2025

[**Anthropic Prompt Caching Support**](/content/changelog/20250214-anthropic-prompt-caching/index.html)

We now support Anthropic’s **Prompt Caching** feature! Monitor and analyze cached prompt tokens and costs alongside regular requests through Helicone’s observability tools. Enable prompt caching through your existing Anthropic integration with Helicone - no additional configuration needed.

- Learn more about integrating with Anthropic in our [Anthropic integration guide](https://docs.helicone.ai/getting-started/integration-method/anthropic).
- Learn more about Anthropic Prompt Caching in [Prompt caching with Claude](https://www.anthropic.com/news/prompt-caching).

### January 31, 2025

[**Perplexity AI + Helicone**

We’re excited to announce our integration with Perplexity AI, bringing powerful observability tools to your Perplexity model implementations! Get started with just two simple steps:

1. Generate a write-only API key in your Helicone account.
2. Update your Perplexity AI base URL to:

```
https://perplexity.helicone.ai
```

That’s all it takes! Now you can monitor, analyze, and optimize your Perplexity AI models with Helicone’s comprehensive insights.

For more details, check out our [Perplexity AI integration guide](https://docs.helicone.ai/getting-started/integration-method/perplexity).

### January 30, 2025

[**Nebius Token Factory + Helicone**

We’re excited to announce our integration with Nebius Token Factory, bringing powerful observability tools to your Nebius model implementations! Get started with just two simple steps:

1. Generate a write-only API key in your Helicone account.
2. Update your Nebius Token Factory base URL to:

```
https://nebius.helicone.ai
```

That’s all it takes! Now you can monitor, analyze, and optimize your Nebius Token Factory models with Helicone’s comprehensive insights.

For more details, check out our [Nebius Token Factory integration guide](https://docs.helicone.ai/getting-started/integration-method/nebius).

### January 24, 2025

[**Helicone Prompt Editor: Your Ultimate AI Prompt Workspace**](/content/changelog/20250124-prompt-editor/index.html)

Elevate your prompt development with our new best-in-the-world editor, designed for peak efficiency and creativity. **Now Live for all Prompts & Experiments users!**

Your browser does not support the video tag.

### Smart Editing Capabilities

- **Advanced Text Editors**
  - Intelligent auto-complete
  - Magic Toolbar with powerful shortcuts
    - `⌘ E`: Quickly add variables
    - `⌘ J`: Insert XML delimiters
    - `⌘ K`: Perform instant edits
- **Workflow Optimization**
  - Side-by-side prompt-response view
  - Instant Save & Run for real-time feedback

### Comprehensive Control & Flexibility

- **Version Management**
  - Fast version switching
  - One-click version promotion
  - Simple deployment via prompt ID
- **Data & Model Support**
  - Import variables from live production data
  - Quickly link to an experiment
  - Message pairs and prefill support
  - Full compatibility with leading AI models:
    - Anthropic
    - OpenAI
    - Google
    - Meta
    - DeepSeek
    - And more!

### January 16, 2025

[**DeepSeek AI + Helicone**

We’re excited to announce our integration with DeepSeek AI, bringing powerful observability tools to your DeepSeek model implementations! Get started with just two simple steps:

1. Generate a write-only API key in your Helicone account.
2. Update your DeepSeek AI base URL to:

```
https://deepseek.helicone.ai
```

That’s all it takes! Now you can monitor, analyze, and optimize your DeepSeek AI models with Helicone’s comprehensive insights.

For more details, check out our [DeepSeek AI integration guide](https://docs.helicone.ai/getting-started/integration-method/deepseek).

### December 19, 2024

[**User Histograms: Analyze LLM Usage Patterns**

We’re excited to introduce User Histograms, a powerful new visualization tool that helps you understand user behavior patterns across your LLM applications.

### Key Features

- **Distribution Visualization**: See how your users are distributed across different metrics like token usage, costs, and request volumes
- **Percentile Analysis**: Quickly identify power users and understand usage patterns at different percentiles
- **Interactive Filtering**: Filter and segment your user data to focus on specific time periods or user groups

### Use Cases

1. **Usage Pattern Analysis**
   - Identify usage clusters and understand how different user segments interact with your LLM applications
   - Spot outliers and investigate unusual usage patterns
2. **Cost Optimization**
   - Understand cost distribution across your user base
   - Make informed decisions about pricing tiers and usage limits
3. **Capacity Planning**
   - Analyze token usage patterns to better predict and plan for scaling
   - Understand peak usage patterns across your user base

To access User Histograms, navigate to the Users tab in your Helicone dashboard and click on the Histograms view.

### December 10, 2024

[**🎉 Experiments is here!**

We are thrilled to announce that [Experiments](/content/us.helicone.ai/experiments/index.html) is out of beta.

Experiments is designed to help you tune your LLM prompt, test it on production data, and verify your iterations with quantifiable data.

### Main use cases

#### 1\. Continuous Improvement

Analyze production edge cases to refine your application’s performance.

#### 2\. Pre-deployment Testing

Benchmark new releases rigorously before rolling out to production environments.

#### 3\. Structured Testing

Implement LLM-as-a-judge or custom evaluation metrics, then compare prompt variations side-by-side with quick, actionable feedback loop.

#### 4\. Prompt Optimization

Determine the best prompt for production by running evaluators to prevent performance regressions.

For detailed documentation, refer to our [updated docs](https://docs.helicone.ai/features/experiments).

### December 6, 2024

[**Support for AWS Bedrock Models**](/content/changelog/20241206-bedrock-integration/index.html)

We’re excited to announce support for tracking [AWS Bedrock models](https://aws.amazon.com/bedrock/) requests through Helicone

### How to track your requests?

To track your Bedrock requests through Helicone, you can set the Bedrock client’s endpoint to use the Helicone Proxy.

```
endpoint="https://bedrock.helicone.ai/v1/<region>"
```

For detailed API documentation, please refer to our updated [docs](https://docs.helicone.ai/integrations/bedrock/javascript).

### November 12, 2024

[**Cerebras: New Model Provider Integration**](/content/changelog/20241112-cerebras-support/index.html)

## Cerebras Integration

We’re excited to announce the addition of Cerebras as a new model provider on our platform. This integration expands our suite of available AI models and provides more options for our users.

### Getting Started

To start using Cerebras models, create an account on [Cerebras](https://cerebras.ai/) and then create a new API key. Once you have your API key, you can add it to your Helicone configuration as a base\_url.

```
base_url="https://cerebras.helicone.ai/v1"
```

For detailed API documentation, please refer to our updated [docs](https://docs.helicone.ai/getting-started/integration-method/cerebras).

### October 24, 2024

[**Webhooks: Real-Time Integration and Automation**

We are excited to announce the addition of webhooks to our platform, enhancing real-time integration and automation capabilities. With this update, you can:

- Set up and monitor webhooks for real-time data processing.
- Seamlessly integrate webhook routes with your applications.
- Test webhooks locally using tools like ngrok.
- Randomly sample webhook events and filter them by specific properties to tailor your data processing needs.
- Utilize webhooks for evaluations and score tracking, enabling more precise and automated performance assessments.

For detailed instructions, please refer to our [Webhooks Setup Guide](https://docs.helicone.ai/features/webhooks).

### October 23, 2024

[**Prompt UI Refresh**

We’ve refreshed the Prompts interface to align with our new UI style — now simpler, more productive, and consistent throughout. Key improvements include:

- Full-width interface
- Ability to view production inputs in a prompt template
- Easily rollback to a previous prompt
- Ability to edit prompts and push to production from the UI

Check it out in the [Prompts tab](/content/prompts/index.html) in Helicone!

### October 23, 2024

[**New Claude 3.5 Sonnet (claude-3-5-sonnet-20241022-v2): Full Cost Support and Tracking**

We’re excited to announce immediate support for Anthropic’s latest Claude 3.5 Sonnet model (claude-3-5-sonnet-20241022-v2), released in October 2024.

### What’s New in This Version?

- Improved performance across various tasks
- Enhanced capabilities in analysis, coding, and creative writing
- Better contextual understanding and response relevance

### Performance Tracking and Cost Management

Our platform now offers:

- Comprehensive performance tracking for claude-3-5-sonnet-20241022-v2
- Updated pricing calculator for accurate cost estimation

### How to Use

Refer to our [Anthropic Integration Guide](https://docs.helicone.ai/integrations/anthropic/javascript) for details on how to use the new model with Helicone

[Learn More About Claude 3.5 Sonnet](https://aws.amazon.com/blogs/aws/upgraded-claude-3-5-sonnet-from-anthropic-available-now-computer-use-public-beta-and-claude-3-5-haiku-coming-soon-in-amazon-bedrock/)

### October 4, 2024

[**🎉 Prompt Experiments V2 Launch! 🎉**](/content/changelog/20241004-prompt-experiments/index.html)

Discover **Helicone’s experiments**, a new spreadsheet-like interface designed for efficient LLM prompt experimentation. Easily manage multiple prompt variations, run flexible experiments, and gain data-driven insights to optimize your AI prompts.

Your browser does not support the video tag.

**Get early access now 👉 [helicone.ai/experiments](/content/experiments/index.html)**

### October 3, 2024

[**Redesigned Requests Page for Enhanced LLM Observability**

We’re excited to announce a major redesign of our Requests page, enhancing the user experience and efficiency for AI LLM observability.

### Key Improvements

- **Streamlined Navigation:** Quick toggle between requests without closing the drawer, allowing for faster review and comparison.
- **Compact Information Display:** More data visible at a glance with a sleeker, more compact row design.
- **Reduced Visual Clutter:** A cleaner interface that focuses on essential information.
- **Enhanced Time Selector:** Improved configuration options and quick select features for more precise data filtering.
- **Unobstructed Page Navigation:** Chat widget no longer blocks page navigation, ensuring a smoother user experience.

### Benefits for LLM Developers and Data Scientists

- **Efficient Prompt Analysis:** Easily view and compare prompts across multiple requests.
- **Improved Performance Monitoring:** Quickly identify trends and anomalies in your LLM applications.
- **Streamlined Workflow:** Navigate through large volumes of request data with ease.

This redesign reflects our commitment to providing the best tools for AI LLM observability. We’ve focused on enhancing the core features that matter most to our users, making it easier than ever to gain insights from your LLM application data.

We encourage you to explore the new Requests page and experience the improvements firsthand. Your feedback is valuable as we continue to refine and enhance Helicone’s observability platform.

### October 2, 2024

[**Introducing new NPM packages for Helicone**](/content/changelog/20241002-vectordb-and-tools/index.html)

We are thrilled to announce the addition of two essential npm packages: `@helicone/async` and `@helicone/helpers`. Additionally, we are also deprecating the `@helicone/helicone` package.

### Why These Changes?

- **Optimized Package Size:** The previous `@helicone/helicone` was wrapped around OpenAI, resulting in a bulky package size.
- **Enhanced Function Utilization:** Many functions within the old package were unused and outdated. The new approach ensures that only necessary functions are included and are up to date.

### Detailed Changes

- **Deprecated `@helicone/helicone`:**
  - This package is officially deprecated and will no longer receive updates.
  - Existing functions within this package will continue to operate as expected to ensure a smooth transition.
- **Added `@helicone/async`:**
  - **HeliconeAsyncLogger Class:** Previously part of `@helicone/helicone`, this class is now housed within `@helicone/async`. It retains all existing functionalities, offering robust asynchronous logging capabilities.
- **Added `@helicone/helpers`:**
  - **HeliconeManualLogger Class:** Moved from `@helicone/helicone` to `@helicone/helpers`, this class now adopts a more functional approach. Visit the [docs](https://docs.helicone.ai/getting-started/integration-method/custom) to learn more.
  - **Enhanced Features:**
    - **[Vector Database Support](https://docs.helicone.ai/integrations/vectordb/javascript):** Added request logging support for vector databases.
    - **[External Tools](https://docs.helicone.ai/integrations/tools/javascript):** You can also log your external tool calls.

### September 23, 2024

[**Summary Reports**

**Get weekly summary reports of your LLM usage**

We’ve launched a new feature that keeps you updated on your **LLM usage** with detailed **weekly reports** delivered directly to your inbox every **Monday at 10 AM UTC**. These reports provide a **comprehensive overview** of key metrics, including **total usage, cost analysis, number of requests, error rate, active users, threats, number of sessions**, and **average session costs**.

With these automated reports, you can easily **monitor your AI performance, optimize your usage**, and make **data-driven decisions** for your projects. Ensure you’re staying on top of your LLM utilization and maximizing the value of your resources.

Ready to get started? [**Configure your weekly summaries now**](https://us.helicone.ai/settings/reports).

### September 16, 2024

[**O1 Models: Support Added with Token and Cost Tracking**

## Immediate Support for OpenAI’s o1 Models

We’re excited to announce support for OpenAI’s new o1 models, along with comprehensive tracking of token counts and spending.

### What Are o1 Models?

OpenAI’s o1 models represent a significant advancement in language AI. They use reinforcement learning to perform complex reasoning tasks, generating an internal chain of thought before producing a final response. This leads to enhanced performance and new capabilities for your applications.

### Accurate Cost Tracking

Our platform now fully supports cost tracking for o1 model usage. Due to the unique way these models process information, it’s important to provide token counts for both input and output to ensure accurate cost calculations.

### How to Ensure Accurate Tracking

- **Using Integrations:** If you’re using integrations like Langchain, LlamaIndex, or LiteLLM, token usage is automatically tracked.
- **Streaming Usage:** For accurate cost calculation while streaming, refer to our guide on [Correct Cost Calculation While Streaming](https://docs.helicone.ai/use-cases/enable-stream-usage).

[Learn More About o1 Models](https://openai.com/index/introducing-openai-o1-preview/)

### September 12, 2024

[**Datasets**](/content/changelog/20240912-datasets/index.html)

Streamline your AI data organization and analysis with Helicone’s new Datasets feature. Designed for LLM developers and data scientists, this tool simplifies data handling for improved AI model performance.

Your browser does not support the video tag.

## Key Features of Helicone Datasets:

1. **Dataset Creation**: Quickly set up and organize your AI training data within the requests page.
2. **Export**: Easily export your data as JSONL for training or finetuning.
3. **Edit**: Edit your dataset and save it as a new version.

## Benefits for AI Development:

To begin using the Datasets feature:

1. Navigate to the Requests page in your Helicone dashboard.
2. Enter select mode by clicking the select icon in the top right corner.
3. Select the data points you want to include in your dataset.
4. Click on “Create Dataset” and give it a name.
5. Access your datasets from the new Datasets tab to export or edit as needed.

### September 11, 2024

[**Collapsible Sidebar**

Enhance your workflow with our new collapsible sidebar feature. Users can now easily toggle the sidebar visibility, maximizing screen real estate and improving focus. This update offers:

- One-click sidebar collapse/expand
- Increased workspace flexibility
- Improved screen space utilization
- Seamless transition between full and minimized views

Optimize your productivity by customizing your interface on demand. Experience a cleaner, more adaptable workspace with our latest sidebar enhancement.

### September 10, 2024

[**Slack Alerts**

**Real-Time Alerts Now Available in Slack for Faster Issue Resolution**

Stay on top of critical issues with Helicone’s latest update: **Slack Integration for Alerts**. In addition to email notifications, you can now receive real-time alerts directly in your Slack workspace for faster action when something goes wrong.

Your browser does not support the video tag.

To get started, visit the [Alerts page](https://us.helicone.ai/alerts) to create or edit an alert. Enhance your team’s productivity by responding to key notifications without delay.

### August 29, 2024

[**\#1 Product of the Day on Product Hunt**

**Helicone Reaches #1 on Product Hunt!**

This achievement reflects our team’s hard work and the incredible support from our community. We’re thrilled about the boost in visibility for our platform!

Highlights:

- #1 on Product Hunt’s daily leaderboard
- Positive feedback from the open-source community
- Surge in new user sign-ups and engagement

A huge thank you to everyone who upvoted, commented, and shared Helicone. Your support motivates us to keep improving!

For more on our Product Hunt journey, check out our blog posts:

- [How to Automate a Product Hunt Launch - Lessons from Helicone’s Success](/content/blog/product-hunt-automate/index.html)
- [Behind 900 pushups, lessons learned from being #1 Product of the Day](/content/blog/product-hunt-experience/index.html)

Links:

Product Hunt: [Helicone on Product Hunt](https://www.producthunt.com/posts/helicone-ai)

### August 25, 2024

[**Docker images on Docker Hub**

Docker images now available on Docker Hub
We’ve started publishing Docker images on Docker Hub.

This update simplifies Helicone deployment on platforms that don’t natively support the Google Container Registry. For detailed instructions, please refer to our updated [self-hosting guide](https://docs.helicone.ai/getting-started/self-deploy-docker).

Links:

Docker Hub: [helicone](https://hub.docker.com/u/helicone)

### August 12, 2024

[**New hpstatic Function for Static Prompts in LLM Applications**

We’ve added a new `hpstatic` function to our [Helicone Prompt Formatter (HPF) package](https://github.com/Helicone/prompts). This function allows users to create static prompts that don’t change between requests, which is particularly useful for system prompts or other constant text. The `hpstatic` function wraps the text in `<helicone-prompt-static>` tags, indicating to Helicone that this part of the prompt should not be treated as variable input.

Here’s a quick example of how to use `hpstatic`:

```typescript
import { hpf, hpstatic } from "@helicone/prompts";

const systemPrompt = hpstatic`You are a helpful assistant.`;
const userPrompt = hpf`Write a story about ${{ character }}`;

const chatCompletion = await openai.chat.completions.create(
  {
    messages: [\
      { role: "system", content: systemPrompt },\
      { role: "user", content: userPrompt },\
    ],
    model: "gpt-3.5-turbo",
  },
  {
    headers: {
      "Helicone-Prompt-Id": "prompt_story",
    },
  }
);
```

This new feature enhances our prompt management capabilities, allowing for more flexible and efficient prompt structuring in your applications.

[Start Using Static Prompts 🚀](https://docs.helicone.ai/features/prompts#static-prompts-with-hpstatic)

### August 9, 2024

[**Ragas Integration for RAG System Evaluation**

We’re excited to announce our integration with Ragas, an open-source framework for evaluating Retrieval-Augmented Generation (RAG) systems. This integration allows you to:

- Monitor and analyze the performance of your RAG pipelines
- Gain insights into RAG effectiveness using metrics like faithfulness, answer relevancy, and context precision
- Easily identify areas for improvement in your RAG systems

Check out this quick video overview of the Ragas integration:

Observability for Ragas - Evaluation Framework - YouTube

Tap to unmute

hello my name is Justin and I am superexcited to show you how to integratehelicone

[Observability for Ragas - Evaluation Framework](https://www.youtube.com/watch?v=SUOI5M5K934) [Helicone AI](https://www.youtube.com/channel/UCBnwtTyErqyXN55MuXc5lTw)

Helicone AI145 subscribers

0:03Time elapsed 3 seconds/2:10Time duration 2 minutes, 10 seconds

More videos

[Watch on YouTube](https://www.youtube.com/watch?v=SUOI5M5K934)

[8:38](https://www.youtube.com/watch?v=5fp6e5nhJRk&feature=endscreen)

[**RAGAS: How to Evaluate a RAG Application Like a Pro for Beginners** \\
2 years ago](https://www.youtube.com/watch?v=5fp6e5nhJRk&feature=endscreen)

[1:29:57](https://www.youtube.com/watch?v=J3M95E0LQl8&feature=endscreen)

[**Want to Master Gen AI Models? Watch This RAGAs Evaluation Now \| RAGAs Framework \| Satyajit Pattnaik** \\
1 year ago](https://www.youtube.com/watch?v=J3M95E0LQl8&feature=endscreen)

[10:22](https://www.youtube.com/watch?v=WY047RvG8K8&feature=endscreen)

[**Reranking in RAG \| RAG \#5 \| Visually Teaching** \\
1 day ago](https://www.youtube.com/watch?v=WY047RvG8K8&feature=endscreen)

[12:53](https://www.youtube.com/watch?v=BxdWlV5LI_4&feature=endscreen)

[**The Linux Kernel is Falling Apart.** \\
5 days ago](https://www.youtube.com/watch?v=BxdWlV5LI_4&feature=endscreen)

[8:36](https://www.youtube.com/watch?v=VxMWdoDn93E&feature=endscreen)

[**If Cops Ask "Where You Headed?" - Say THIS (Simple Phrase)** \\
7 days ago](https://www.youtube.com/watch?v=VxMWdoDn93E&feature=endscreen)

[9:29](https://www.youtube.com/watch?v=-N0v4F8rSEw&feature=endscreen)

[**9 Microsoft Copilot Features Your Company Paid For (That Most Professionals Never Use)** \\
3 weeks ago](https://www.youtube.com/watch?v=-N0v4F8rSEw&feature=endscreen)

[22:22](https://www.youtube.com/watch?v=dOKHuw52YTA&feature=endscreen)

[**Evaluate AI Agents in Python with Ragas** \\
1 year ago](https://www.youtube.com/watch?v=dOKHuw52YTA&feature=endscreen)

[12:12](https://www.youtube.com/watch?v=aeae-sITqEA&feature=endscreen)

[**RAGAS - Evaluate your LangChain RAG Pipelines** \\
2 years ago](https://www.youtube.com/watch?v=aeae-sITqEA&feature=endscreen)

[10:43](https://www.youtube.com/watch?v=cRz0BWkuwHg&feature=endscreen)

[**Key Metrics and Evaluation Methods for RAG** \\
1 year ago](https://www.youtube.com/watch?v=cRz0BWkuwHg&feature=endscreen)

[19:42](https://www.youtube.com/watch?v=-_52DIIOsCE&feature=endscreen)

[**AI Agent Evaluation with RAGAS** \\
2 years ago](https://www.youtube.com/watch?v=-_52DIIOsCE&feature=endscreen)

[4:42](https://www.youtube.com/watch?v=8iVHOkUrpSA&feature=endscreen)

[**How to monitor LLM requests from Open WebUI** \\
1 year ago](https://www.youtube.com/watch?v=8iVHOkUrpSA&feature=endscreen)

[23:13](https://www.youtube.com/watch?v=CkldFczgwIg&feature=endscreen)

[**Doctor Warns These 9 Medications May Cause Memory Loss After 60 - Dr. William Li** \\
2 days ago](https://www.youtube.com/watch?v=CkldFczgwIg&feature=endscreen)

To get started with the Ragas integration, visit our [documentation](https://docs.helicone.ai/other-integrations/ragas) for step-by-step instructions and code examples.

### August 6, 2024

[**Optimistic Updates & Asynchronous Loading in Requests Page**

We’ve improved data loading in the Requests page of the Helicone platform. By fetching metadata and request bodies separately and loading data asynchronously we’ve reduced the time it takes to render large tables by almost 6x, improving speed and UX.

### July 26, 2024

[**New Assistants UI Playground**

We’re thrilled to announce a major update to our Assistants UI Playground! Head to the [Playground](https://us.helicone.ai/playground) and click the “Try New Playground” button to explore the latest improvements:

- Streamed responses for real-time interaction
- Enhanced tool rendering for better visualization
- Improved reliability for a smoother experience

Coming soon:

- Expanded model support
- Advanced prompt management
- Integrated Markdown editor

Try out the new Playground today and elevate your LLM testing experience!

### July 24, 2024

[**Fireworks AI + Helicone**

We’re excited to announce our integration with Fireworks AI, the high-performance LLM platform! Enhance your AI applications with Helicone’s powerful observability tools in just two easy steps:

1. Generate a write-only API key in your Helicone account.
2. Update your Fireworks AI base URL to:

```
https://fireworks.helicone.ai
```

That’s all it takes! Now you can monitor, analyze, and optimize your Fireworks AI models with Helicone’s comprehensive insights.

For more details, check out our [Fireworks AI integration guide](https://docs.helicone.ai/getting-started/integration-method/fireworks).

### July 23, 2024

[**Dify + Helicone**

We’re thrilled to announce our integration with Dify, the open-source LLM app development platform! Now you can easily add Helicone’s powerful observability features to your Dify projects in just two simple steps:

1. Generate a write-only API key in your Helicone account.
2. Set your API base URL in Dify to:

```
https://oai.helicone.ai/<API_KEY>
```

That’s it! Enjoy comprehensive logs and insights for your Dify LLM applications.

Check out our [integration guide](https://docs.helicone.ai/integration-guide/dify) for more details.

### July 22, 2024

[**Prompts package**

We’re excited to announce the release of our new `@helicone/prompts` package! This lightweight library simplifies prompt formatting for Large Language Models, offering features like:

- Automated versioning with change detection
- Support for chat-like prompt templates
- Efficient variable handling and extraction

Check it out on [GitHub](https://github.com/Helicone/prompts) and enhance your LLM workflow today!