Core Concepts

Prompt Configs

A prompt config is a container for all versions of a specific prompt. It has:

Key: A unique identifier used in your code (e.g., welcome-message)
Name: A human-readable name for the dashboard
Description: Optional documentation about the prompt’s purpose
Variables: Placeholders that get filled in at execution time

Example:

# In your code, you reference the prompt by its key
response = client.prompts.execute(
    prompt_key="welcome-message",
    variables={"name": "Alice"}
)

Prompt Versions

Each prompt config can have multiple versions. A version includes:

Model: The LLM to use (e.g., gpt-5, claude-3-5-sonnet-20241022)
Prompt Template: The actual prompt text with variable placeholders
Version Name: A label for this version (e.g., v1, formal-tone)
Status: Published or draft

Example version:

Model: gpt-5
Prompt: Write a friendly welcome message for {{name}}.

Subject Identification

Subject identification is data you provide to ensure consistent version assignment. It’s used for:

Deterministic Routing: Same subject always gets the same version
Metrics Aggregation: Links metrics to the correct prompt version

Subject identification can be:

A simple string: "user_123"
A dictionary: {"user_id": "user_123", "tier": "premium"}

Always use the same subject when executing prompts and recording metrics for the same user or session. This ensures metrics are correctly attributed to prompt versions.

Example:

# Execute with subject identification
response = client.prompts.execute(
    prompt_key="welcome-message",
    subject={"user_id": "user_123"},
    variables={"name": "Alice"}
)

# Record metric with same subject
client.metrics.record(
    metric_key="satisfaction",
    value=4.5,
    subject={"user_id": "user_123"}  # Same subject!
)

A/B Testing

A/B testing (or multivariate testing) lets you compare multiple prompt versions:

Create multiple versions of a prompt
Set up a traffic config that defines the distribution (e.g., 50% v1, 50% v2)
Execute the prompt with subject identification
Users are deterministically assigned to versions based on subject hash

The same subject always gets the same version. This ensures users have a consistent experience across sessions.

Example flow:

# User 1 might get version A
response_1 = client.prompts.execute(
    prompt_key="product-pitch",
    subject={"user_id": "user_001"}
)

# User 2 might get version B
response_2 = client.prompts.execute(
    prompt_key="product-pitch",
    subject={"user_id": "user_002"}
)

# User 1 will ALWAYS get version A on subsequent calls
response_1_again = client.prompts.execute(
    prompt_key="product-pitch",
    subject={"user_id": "user_001"}
)
# response_1.version_id == response_1_again.version_id

Metrics

Metrics are measurements you track to evaluate prompt performance. They can be:

Float: Numerical values (e.g., satisfaction score, response time)
Boolean: True/false values (e.g., conversion, email opened)
Enum: Categorical values (e.g., sentiment: positive/negative/neutral)

Metrics are automatically aggregated per prompt version when you use the same subject. Example:

# Record different metric types
client.metrics.record(
    metric_key="satisfaction",
    value=4.5,  # Float
    subject={"user_id": "user_123"}
)

client.metrics.record(
    metric_key="converted",
    value=True,  # Boolean
    subject={"user_id": "user_123"}
)

client.metrics.record(
    metric_key="sentiment",
    value="positive",  # Enum
    subject={"user_id": "user_123"}
)

Variables

Variables are dynamic values you pass to prompts at execution time. They’re defined using double curly braces in your prompt template. Dashboard:

Write a {{tone}} welcome message for {{customer_name}} who just purchased {{product}}.

Code:

response = client.prompts.execute(
    prompt_key="welcome-message",
    variables={
        "tone": "professional",
        "customer_name": "Alice",
        "product": "Pro Plan"
    }
)

Traffic Configs

A traffic config defines how users are distributed across prompt versions:

{
  "v1": 50,  // 50% of users get v1
  "v2": 30,  // 30% get v2
  "v3": 20   // 20% get v3
}

Traffic configs enable:

Gradual rollouts: Start with 10% on new version, increase if metrics look good
A/B tests: Equal split (50/50) between two versions
Multivariate tests: Test 3+ versions simultaneously

Request IDs

Every prompt execution returns a request ID. This unique identifier:

Links prompts to metrics in the dashboard
Helps with debugging and support
Enables detailed audit trails

response = client.prompts.execute(prompt_key="welcome")
print(response.request_id)  # "req_abc123..."

Getting Started

Prompt Configs

Prompt Versions

Subject Identification

A/B Testing

Metrics

Variables

Traffic Configs

Request IDs

Next Steps

Python Quickstart

A/B Testing Guide

​Prompt Configs

​Prompt Versions

​Subject Identification

​A/B Testing

​Metrics

​Variables

​Traffic Configs

​Request IDs

​Next Steps

Python Quickstart

A/B Testing Guide

Prompt Configs

Prompt Versions

Subject Identification

A/B Testing

Metrics

Variables

Traffic Configs

Request IDs

Next Steps