> ## Documentation Index
> Fetch the complete documentation index at: https://docs.switchport.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B Testing

> Run A/B tests to compare prompt versions and optimize performance

## Overview

A/B testing (also called split testing) lets you compare multiple prompt versions to see which performs better. Switchport provides deterministic routing so users get a consistent experience.

## How It Works

1. **Create multiple versions** of a prompt in the dashboard
2. **Set up a traffic config** to distribute users across versions
3. **Execute prompts with user identification** to get deterministic version assignment
4. **Record metrics** to measure performance
5. **Analyze results** in the dashboard to identify winners

## Creating Multiple Versions

In the Switchport dashboard:

<Steps>
  <Step title="Navigate to your prompt">
    Go to **Prompts** and select the prompt you want to test
  </Step>

  <Step title="Create first version">
    * Click **Add Version**
    * Name: `v1` or `formal-tone`
    * Set model and prompt template
    * Click **Save** and **Publish**
  </Step>

  <Step title="Create second version">
    * Click **Add Version** again
    * Name: `v2` or `casual-tone`
    * Use different wording or approach
    * Click **Save** and **Publish**
  </Step>

  <Step title="Create traffic config">
    * Click **Traffic Config**
    * Set distribution (e.g., 50% v1, 50% v2)
    * Click **Activate**
  </Step>
</Steps>

## Executing Prompts with A/B Testing

The key to A/B testing is using **context** for deterministic routing:

```go theme={null}
package main

import (
	"fmt"
	"log"

	"github.com/switchport-ai/switchport-go/switchport"
)

func main() {
	client, err := switchport.NewClient("")
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)
	}

	// User 1 gets assigned a version
	response1, err := client.Prompts.Execute(
		"product-pitch",
		map[string]interface{}{"user_id": "user_001"},
		map[string]interface{}{"product": "Pro Plan"},
	)
	if err != nil {
		log.Fatalf("Failed to execute prompt: %v", err)
	}
	fmt.Printf("User 1 got: %s\n", response1.VersionName)

	// User 2 might get a different version
	response2, err := client.Prompts.Execute(
		"product-pitch",
		map[string]interface{}{"user_id": "user_002"},
		map[string]interface{}{"product": "Pro Plan"},
	)
	if err != nil {
		log.Fatalf("Failed to execute prompt: %v", err)
	}
	fmt.Printf("User 2 got: %s\n", response2.VersionName)

	// User 1 will ALWAYS get the same version
	response1Again, err := client.Prompts.Execute(
		"product-pitch",
		map[string]interface{}{"user_id": "user_001"},
		map[string]interface{}{"product": "Pro Plan"},
	)
	if err != nil {
		log.Fatalf("Failed to execute prompt: %v", err)
	}
	fmt.Printf("User 1 again: %s\n", response1Again.VersionName)

	// These will be equal:
	if response1.VersionID == response1Again.VersionID {
		fmt.Println("✓ Same user got same version (deterministic)")
	}
}
```

<Info>
  The same user always returns the same version. This ensures users have a consistent experience across sessions.
</Info>

## Recording Metrics for A/B Tests

To compare versions, record metrics with the **same subject**:

```go theme={null}
func sendWelcomeEmail(client *switchport.Client, user User) error {
	subject := map[string]interface{}{"user_id": user.ID}

	// Execute prompt (subject gets assigned to a version)
	response, err := client.Prompts.Execute(
		"welcome-email", subject,
		map[string]interface{}{
			"name":        user.Name,
			"signup_date": user.CreatedAt,
		},
	)
	if err != nil {
		return err
	}

	// Send the email
	if err := sendEmail(user.Email, response.Text); err != nil {
		return err
	}

	// Later, track if they converted
	_, err = client.Metrics.Record(
		"conversion",
		true,
		subject, // Same subject!
		nil,
	)
	return err
}
```

Switchport automatically:

* Links the metric to the version that user saw
* Aggregates metrics per version
* Calculates averages, success rates, and distributions

## Traffic Distribution

You can distribute traffic in various ways:

### 50/50 Split (Classic A/B)

```json theme={null}
{
  "v1": 50,
  "v2": 50
}
```

### Multivariate Testing (A/B/C)

```json theme={null}
{
  "v1": 33,
  "v2": 33,
  "v3": 34
}
```

### Gradual Rollout

Start with a small percentage on the new version:

```json theme={null}
{
  "v1": 90,
  "v2": 10
}
```

If metrics look good, increase the new version:

```json theme={null}
{
  "v1": 50,
  "v2": 50
}
```

Finally, roll out completely:

```json theme={null}
{
  "v2": 100
}
```

## Complete Example: Email A/B Test

```go theme={null}
package main

import (
	"log"

	"github.com/switchport-ai/switchport-go/switchport"
)

type User struct {
	ID        string
	Name      string
	Email     string
	CreatedAt string
}

func sendWelcomeEmail(client *switchport.Client, user User) error {
	// Execute prompt (user gets assigned to a version)
	response, err := client.Prompts.Execute(
		"welcome-email",
		map[string]interface{}{"user_id": user.ID},
		map[string]interface{}{
			"name":        user.Name,
			"signup_date": user.CreatedAt,
		},
	)
	if err != nil {
		return err
	}

	// Send the email
	if err := sendEmail(user.Email, response.Text); err != nil {
		return err
	}

	// Track which version they saw (for debugging)
	logEvent("email_sent", map[string]interface{}{
		"user_id": user.ID,
		"version": response.VersionName,
	})

	return nil
}

func trackEmailOpened(client *switchport.Client, userID string) error {
	_, err := client.Metrics.Record(
		"email_opened",
		true,
		map[string]interface{}{"user_id": userID},
		nil,
	)
	return err
}

func trackEmailClicked(client *switchport.Client, userID string) error {
	_, err := client.Metrics.Record(
		"email_clicked",
		true,
		map[string]interface{}{"user_id": userID},
		nil,
	)
	return err
}

func trackConversion(client *switchport.Client, userID string) error {
	_, err := client.Metrics.Record(
		"conversion",
		true,
		map[string]interface{}{"user_id": userID},
		nil,
	)
	return err
}

func sendEmail(email, content string) error {
	// Implementation...
	return nil
}

func logEvent(event string, data map[string]interface{}) {
	log.Printf("Event: %s, Data: %v", event, data)
}
```

## Analyzing Results

In the Switchport dashboard, you can view:

* **Metric averages per version**: See which version has higher satisfaction scores
* **Conversion rates**: Compare success rates for boolean metrics
* **Sample sizes**: Ensure statistical significance
* **Confidence intervals**: Understand the reliability of results

## Best Practices

<AccordionGroup>
  <Accordion title="Use consistent context">
    Always use the same subject (e.g., user ID) for a given subject across all prompt executions and metric recordings.
  </Accordion>

  <Accordion title="Start with 50/50 splits">
    For initial A/B tests, use even splits to gather data faster.
  </Accordion>

  <Accordion title="Run tests long enough">
    Ensure you have enough data for statistical significance before declaring a winner.
  </Accordion>

  <Accordion title="Test one thing at a time">
    Change only one variable between versions to understand what drives performance differences.
  </Accordion>

  <Accordion title="Use gradual rollouts">
    For new versions, start with a small percentage to minimize risk.
  </Accordion>

  <Accordion title="Define success metrics upfront">
    Decide what metrics matter before running the test to avoid cherry-picking results.
  </Accordion>

  <Accordion title="Monitor for regressions">
    Watch for unexpected drops in other metrics when optimizing for one specific metric.
  </Accordion>
</AccordionGroup>

## Common Use Cases

### Email Marketing

Test subject lines, tone, call-to-action wording:

```go theme={null}
// Different email styles for different user segments
response, err := client.Prompts.Execute(
	"marketing-email",
	map[string]interface{}{
		"user_id": user.ID,
		"segment": user.Segment,
	},
	map[string]interface{}{"product": "Summer Sale"},
)
```

Metrics: open rate, click rate, conversion rate

### Customer Support Chatbot

Test different conversation styles:

```go theme={null}
response, err := client.Prompts.Execute(
	"support-bot",
	map[string]interface{}{"user_id": user.ID},
	map[string]interface{}{"issue": userMessage},
)
```

Metrics: resolution rate, satisfaction score, escalation rate

### Product Descriptions

Test different description styles:

```go theme={null}
response, err := client.Prompts.Execute(
	"product-description",
	map[string]interface{}{
		"product_id":   product.ID,
		"user_segment": segment,
	},
	map[string]interface{}{"product_name": product.Name},
)
```

Metrics: conversion rate, time on page, add-to-cart rate

## Next Steps

<Columns cols={2}>
  <Card title="Examples" icon="code" href="/sdk/go/examples/advanced-usage">
    See complete A/B testing examples
  </Card>

  <Card title="Metrics Reference" icon="chart-line" href="/sdk/go/reference/metrics">
    Learn more about recording metrics
  </Card>
</Columns>