> ## Documentation Index
> Fetch the complete documentation index at: https://ritika-fa61c43f.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# CRM Data Pipeline Environment Configuration Reference

> Set environment variables, choose task difficulty, configure your LLM provider, and tune agent behavior for the CRM Data Pipeline benchmark.

The CRM Data Pipeline Environment is configured entirely through environment variables — there are no configuration files to edit beyond the optional `.env` file in the project root. This design keeps the server stateless and fully compatible with containerized deployments on HuggingFace Spaces, Docker Compose, and Kubernetes. This page covers every variable the server and inference script read, how to select task difficulty at reset time, how to point the inference script at any OpenAI-compatible LLM provider, and how the built-in step and submission limits work.

## Environment Variables

The table below lists every environment variable recognized by the server and the inference script. Variables marked **Yes** under Required must be set before the relevant component will function correctly.

| Variable | Required | Default | Description |
| - | - | - | - |
| `TASK_ID` | No | `t1` | Which task to load when `POST /reset` is called. Accepted values: `t1` (Easy), `t2` (Medium), `t3` (Hard). If not set, the server defaults to `t1`. |
| `HF_TOKEN` | Yes (for inference) | — | HuggingFace API token for authenticating against the HuggingFace Inference Router. Checked first; if absent, `API_KEY` is used as a fallback. |
| `API_KEY` | Alternative to `HF_TOKEN` | — | Any OpenAI-compatible API key. Used when `HF_TOKEN` is not set. Accepts keys from OpenAI or any other service that speaks the OpenAI API format. |
| `API_BASE_URL` | No | `https://router.huggingface.co/v1` | The base URL for the OpenAI-compatible LLM API endpoint. Override this to point the inference script at OpenAI, a local Ollama instance, or any other compatible provider. |
| `MODEL_NAME` | No | `Qwen/Qwen2.5-72B-Instruct` | The model identifier sent to the LLM API. Use the exact model name string required by your chosen provider. |
| `OPENENV_BASE_URL` | No | `http://localhost:8080` | The base URL of the running environment server. Used by `CRMDataPipelineEnvClient` and any custom agent code that connects to the environment. |

**Creating your `.env` file:**

Create a `.env` file in the project root and populate it with the variables you need. The `python-dotenv` library loaded by `inference.py` automatically sources this file at script startup without overriding any variables already present in the shell environment.

```bash .env theme={null}
# HuggingFace token — primary credential for inference.py LLM calls
HF_TOKEN=hf_your_token_here

# Uncomment to use an OpenAI-compatible key instead of HF_TOKEN
# API_KEY=sk-your-openai-key-here

# Task loaded on every /reset call (t1 = easy, t2 = medium, t3 = hard)
TASK_ID=t1

# LLM provider endpoint — change this to switch providers
API_BASE_URL=https://router.huggingface.co/v1

# Model identifier for the LLM provider
MODEL_NAME=Qwen/Qwen2.5-72B-Instruct

# Environment server URL used by CRMDataPipelineEnvClient
OPENENV_BASE_URL=http://localhost:8080
```

> [!WARNING]
>
> Never commit your `.env` file to version control. The `.gitignore` in this repository already excludes it, but double-check before pushing. Your `HF_TOKEN` or `API_KEY` grants full API access under your account and should be treated as a secret.

## Choosing a Task

The `TASK_ID` environment variable is the single control knob for task difficulty. You can change the active task between episodes simply by updating the variable and calling `/reset` again, without restarting the server.

> [tab] Shell Export
>
> ```bash theme={null}
> # Easy: single-source web forms normalization
> # Agent cleans one dataset: fix date formats, lowercase emails, strip whitespace.
> export TASK_ID=t1
>
> # Medium: two-source deduplication
> # Agent standardizes web_forms and legacy_db, then merges and deduplicates.
> export TASK_ID=t2
>
> # Hard: three-way enterprise merge
> # Agent standardizes salesforce, web_leads, and legacy_db, then uses EXECUTE_SQL
> # to perform a complex aggregation join while filtering bot-injected outlier rows.
> export TASK_ID=t3
>
> uv run -m server.app
> ```

> [tab] Docker Run
>
> ```bash theme={null}
> # Easy
> docker run -p 8080:8080 -e TASK_ID=t1 crm-pipeline-env
>
> # Medium
> docker run -p 8080:8080 -e TASK_ID=t2 crm-pipeline-env
>
> # Hard
> docker run -p 8080:8080 -e TASK_ID=t3 crm-pipeline-env
> ```

> [tab] .env File
>
> ```bash .env theme={null}
> # Set once in your .env file; inference.py will pick it up automatically
> TASK_ID=t3
> ```

**What each task tests:**

> [!NOTE]
>
> Task difficulty is cumulative — each harder task is a strict superset of the operations required by the easier ones. If your agent solves `t3` reliably, it will solve `t1` and `t2` as well.

**Task 1 — Easy (`t1`): Web Forms Normalization**

The agent operates on a single `web_forms` source. The data contains mixed-case email addresses, phone numbers with inconsistent formatting (parentheses, dashes, dots, extensions), dates in multiple formats (`MM/DD/YYYY`, natural language, ISO 8601), null values, duplicate entries, and a small number of bot-injected rows identifiable by `customer_id = '???'` or `email LIKE '%bot%'`. The agent must profile the data, apply column standardizations, deduplicate by email, filter outliers with `EXECUTE_SQL`, and submit a clean `web_forms_clean` table.

**Task 2 — Medium (`t2`): Legacy DB Deduplication**

The agent now handles two sources: `web_forms` and `legacy_db`. Both require full standardization before they can be merged. After cleaning each source independently, the agent must `UNION ALL` them via `EXECUTE_SQL` into a `merged_output` table and deduplicate the combined set using `EXACT_EMAIL`. The challenge here is that `legacy_db` may use different column naming conventions, requiring the agent to inspect schemas carefully before acting.

**Task 3 — Hard (`t3`): Three-Way Enterprise Merge**

The most complex task. The agent standardizes three sources — `salesforce`, `web_leads`, and `legacy_db` — each with different schemas (notably `legacy_db` uses `contact_email` and `home_phone` instead of `email` and `phone`). The agent must craft a multi-source `EXECUTE_SQL` aggregation query that renames columns, filters bots, strips the `OLD-` prefix from `legacy_id` values, and resolves duplicate email conflicts using `MAX()` or `GROUP BY` semantics. A poorly constructed SQL query that leaves extra rows incurs a `-0.15` penalty per surplus row in the grader.

## LLM Provider Configuration

The inference script is compatible with any provider that exposes an OpenAI-compatible `/v1/chat/completions` endpoint. Set `API_BASE_URL` and `MODEL_NAME` to point the script at your preferred provider.

```bash HuggingFace Router (Default) theme={null}
# Uses Qwen2.5-72B-Instruct via the HuggingFace Inference Router.
# Requires a valid HF_TOKEN with Inference API access.
export HF_TOKEN=hf_your_token_here
export API_BASE_URL=https://router.huggingface.co/v1
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
```

```bash OpenAI Directly theme={null}
# Uses GPT-4o via the official OpenAI API.
# Replace API_KEY with your OpenAI project key.
export API_KEY=sk-proj-your-openai-key
export API_BASE_URL=https://api.openai.com/v1
export MODEL_NAME=gpt-4o
```

```bash Local Ollama theme={null}
# Uses a locally running Ollama instance — no API key required.
# Ensure `ollama serve` is running and the model is pulled: `ollama pull llama3.1`.
export API_BASE_URL=http://localhost:11434/v1
export MODEL_NAME=llama3.1
# API_KEY can be any non-empty string for Ollama's OpenAI-compat layer
export API_KEY=ollama
```

```bash vLLM / Custom Endpoint theme={null}
# Any OpenAI-compatible vLLM or LiteLLM proxy deployment.
export API_KEY=your-vllm-key
export API_BASE_URL=https://your-vllm-host.example.com/v1
export MODEL_NAME=meta-llama/Meta-Llama-3.1-70B-Instruct
```

> [!TIP]
>
> When using the HuggingFace Router, keep `temperature` low (the inference script defaults to `0.2`) to produce deterministic JSON action outputs. Higher temperatures increase JSON parse failures, which trigger the rule-based fallback. Confirm that your chosen model and provider support the `response_format: json_object` parameter to enforce structured output.

The `inference.py` script reads provider configuration from environment variables at startup:

```python theme={null}
import os
from openai import OpenAI

API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
API_BASE_URL = os.getenv("API_BASE_URL") or "https://router.huggingface.co/v1"
MODEL_NAME = os.getenv("MODEL_NAME") or "Qwen/Qwen2.5-72B-Instruct"

openai_client = OpenAI(
    api_key=API_KEY,
    base_url=API_BASE_URL
)
```

`HF_TOKEN` is checked first; `API_KEY` is the fallback. If neither is set, all LLM calls will raise an `AuthenticationError` and the script will fall back to the deterministic rule-based baseline for every step.

## Step Limits and Submission Guards

Two constants govern episode pacing and protect the grader from degenerate submissions.

**`MAX_STEPS_PER_TASK = 15`** (set in `inference.py`)

The episode loop runs for at most 15 steps per task before forcibly terminating the episode. This budget is sufficient for the most complex Task 3 multi-source pipeline (which the deterministic baseline solves in 7–8 steps) while preventing runaway agents from consuming excessive API quota. If you are writing a custom agent that needs more steps for exploratory strategies, increase this constant directly in your copy of `inference.py`.

**`MIN_STEPS_BEFORE_SUBMIT = 3`** (enforced by the environment server)

The environment enforces a minimum of 3 data cleaning steps before it accepts a `SUBMIT_PIPELINE` action. If an agent sends `SUBMIT_PIPELINE` as step 1 or step 2, the server returns:

```json theme={null}
{
  "observation": {
    "last_action_feedback": "Early submission blocked. You must perform at least 3 cleaning steps (Profile, Standardize, etc.) before submitting.",
    "done": false
  },
  "reward": -0.15,
  "done": false
}
```

The `-0.15` penalty is applied to the step reward, the episode continues (it is not terminated), and the agent must continue acting. This guard prevents trivially short episodes that would score near zero and skew evaluation metrics.

**Reward shaping summary:**

| Action | Reward |
| - | - |
| `PROFILE_SOURCE` | `+0.02` (flat) |
| `STANDARDIZE_COLUMN` | `+0.01` to `+0.05` (proportional to fraction of rows changed) |
| `HANDLE_MISSING` (DROP\_ROW) | `+0.01` to `+0.10` (proportional to null count resolved) |
| `HANDLE_MISSING` (FILL\_VALUE) | `+0.01` to `+0.05` |
| `DEDUPLICATE` | `+0.0` to `+0.20` (proportional to fraction of rows removed) |
| `EXECUTE_SQL` (valid SELECT) | `+0.10` |
| `EXECUTE_SQL` (forbidden / syntax error) | `-0.10` |
| `SUBMIT_PIPELINE` (premature, step \< 3) | `-0.15` |
| `SUBMIT_PIPELINE` (valid) | `+0.0` to `+0.20` (schema column match ratio × 0.2) |
| Unknown action type | `-0.05` |
| Any exception during action handling | `-0.05` |

## openenv.yaml Reference

The `openenv.yaml` file in the repository root is the machine-readable task catalogue consumed by OpenEnv tooling for environment registration and automated evaluation harness configuration. You do not need to edit this file unless you are extending the environment with new tasks.

```yaml openenv.yaml theme={null}
id: crm-data-pipeline
name: CRM Data Pipeline Agent
version: 1.0.0
description: |
  An environment simulating a CRM Data Engineer. The agent receives messy
  customer data from various sources (Web, Salesforce, Legacy DB) and must
  standardize, clean, and merge them into a pristine, analytics-ready
  pipeline table.
tags: ["etl", "data-engineering", "crm", "pandas"]
endpoints: {}
tasks:
  - id: t1
    description: "Task 1 (Easy): Normalize the single web_forms dataset. Fix date formats, lowercase emails, strip spaces."
    difficulty: easy
  - id: t2
    description: "Task 2 (Medium): Deduplicate the legacy_db dataset. Identify matching customers with messy info and merge them."
    difficulty: medium
  - id: t3
    description: "Task 3 (Hard): Resolve a 3-way merge conflict. Standardize and merge Salesforce, Web Leads, and Legacy DBs."
    difficulty: hard
```

**Field reference:**

| Field | Value | Description |
| - | - | - |
| `id` | `crm-data-pipeline` | Globally unique identifier for this environment within the OpenEnv registry. Used as a key in leaderboard entries and benchmark reports. |
| `name` | `CRM Data Pipeline Agent` | Human-readable display name shown in the OpenEnv dashboard. |
| `version` | `1.0.0` | Semantic version of the environment specification. Increment the minor version when you add new tasks; increment the major version when the action or observation schema changes in a breaking way. |
| `description` | (multi-line) | Prose description of the environment's purpose and scope. |
| `tags` | `["etl", "data-engineering", "crm", "pandas"]` | Searchable taxonomy tags used by the OpenEnv registry to group environments by domain and tooling. |
| `endpoints` | `{}` | Custom endpoint overrides for non-standard OpenEnv deployments. Empty here because all endpoints follow the standard `/reset`, `/step`, `/state` convention. |
| `tasks[].id` | `t1` / `t2` / `t3` | Task identifier — must exactly match the values accepted by the `TASK_ID` environment variable. |
| `tasks[].description` | (string) | Human-readable description of the task objective shown to agents and researchers. |
| `tasks[].difficulty` | `easy` / `medium` / `hard` | Normalized difficulty tier used by evaluation infrastructure to group results and compute per-difficulty aggregate scores. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.