> ## Documentation Index
> Fetch the complete documentation index at: https://ritika-fa61c43f.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Environment Architecture and Data Flow Explained

> A deep-dive into the CRM Pipeline's POMDP model, server components, file interconnections, reward signals, and full episode lifecycle.

The CRM Data Pipeline Environment is not a thin wrapper around a dataset — it is a fully stateful POMDP engine backed by a FastAPI server, isolated per-episode memory stores, a dense reward calculator, and a grader that performs row-exact DataFrame comparison against a hidden ground truth. This page dissects every layer of that architecture so you understand exactly what happens from the moment your agent calls `/reset` to the moment it receives a final score from `/grader/{episode_id}`.

## 1. System Architecture Overview

The runtime topology spans seven source files. The diagram below maps every component to its file and labels the direction of data flow.

```mermaid theme={null}
graph TD
    subgraph Agent Side
        A[LLM Agent\ninference.py]
        CL[CRMDataPipelineEnvClient\nclient.py]
    end

    subgraph Server Side
        APP[FastAPI Bootstrap\nserver/app.py]
        ENV[CRMDataPipelineEnv\nserver/environment.py]
        DG[DataGenerator\nserver/data_generator.py]
        GR[Grader Engine\nserver/graders.py]
        DB[(GLOBAL_TRUTH_STORE\nGLOBAL_ENV_STORE\ndicts keyed by UUID)]
    end

    subgraph Data Layer
        DF[(Pandas DataFrames\nin-memory sources dict)]
        SQ[(SQLite :memory:\nfor EXECUTE_SQL)]
    end

    subgraph Schema Layer
        MDL[Pydantic Models\nmodels.py]
    end

    A -->|builds CRMPipelineAction| CL
    CL -->|HTTP POST /step or /reset| APP
    APP -->|delegates to| ENV
    ENV -->|get_task_data on reset| DG
    DG -->|returns sources + hidden_truth| ENV
    ENV -->|stores truth snapshot| DB
    ENV -->|mutates| DF
    ENV -->|EXECUTE_SQL opens| SQ
    ENV -->|builds| MDL
    APP -->|imports| GR
    GR -->|reads GLOBAL_TRUTH_STORE| DB
    GR -->|reads env.final_df| ENV
    APP -->|returns HTTP response| CL
    CL -->|parses CRMPipelineObservation| A

    style A fill:#1e3a8a,stroke:#3b82f6,color:#fff
    style CL fill:#1e3a8a,stroke:#3b82f6,color:#fff
    style APP fill:#0f172a,stroke:#64748b,color:#fff
    style ENV fill:#0f172a,stroke:#64748b,color:#fff
    style DG fill:#0f172a,stroke:#64748b,color:#fff
    style GR fill:#0f172a,stroke:#64748b,color:#fff
    style DB fill:#292524,stroke:#78716c,color:#fff
    style DF fill:#064e3b,stroke:#10b981,color:#fff
    style SQ fill:#064e3b,stroke:#10b981,color:#fff
    style MDL fill:#451a03,stroke:#f97316,color:#fff
```

Each server-side component has a single, well-defined responsibility:

* **`server/app.py`** — FastAPI bootstrap. Calls `create_fastapi_app(CRMDataPipelineEnv, ...)` from `openenv.core`, registers a custom `/grader/{episode_id}` POST endpoint, and exposes a health-check `GET /`.
* **`server/environment.py`** — Core POMDP engine. Owns `reset()`, `step()`, all action handlers, and the `_build_observation()` factory. Holds the two global state stores.
* **`server/data_generator.py`** — Stochastic dataset factory. Called once per `/reset`. Uses Faker + NumPy + random to produce corrupted source frames and clean hidden-truth frames.
* **`server/graders.py`** — Scoring logic. `evaluate_dataframes()` performs a left-join between truth and agent output on `customer_id`, checks column-by-column exact string equality, and applies a bot-penalty.
* **`models.py`** — All Pydantic schemas: `CRMPipelineAction`, `CRMPipelineObservation`, `CRMPipelineState`, plus every Enum (`PipelineActionType`, `StandardizationStrategy`, `MissingStrategy`, `ConflictRule`, `DeduplicationStrategy`).
* **`client.py`** — Typed REST client. Extends `EnvClient` from `openenv.core`. Implements `_step_payload()` to serialise actions and `_parse_result()` to deserialise observations.
* **`inference.py`** — Agent driver. Instantiates `CRMDataPipelineEnvClient`, runs the observe-act loop, calls the OpenAI-compatible LLM with retry logic, and falls back to `RuleBasedBaseline` if the LLM fails.

## 2. File Interconnections Map

```mermaid theme={null}
graph LR
    INF[inference.py] -->|imports| CL[client.py]
    INF -->|imports| MDL[models.py]
    CL -->|imports| MDL
    APP[server/app.py] -->|imports| ENV[server/environment.py]
    APP -->|imports| GR[server/graders.py]
    APP -->|imports| MDL
    ENV -->|imports| MDL
    ENV -->|imports| DG[server/data_generator.py]
    GR -->|imports| ENV
    DG -->|uses faker, pandas, numpy| EXT([External: faker / pandas / numpy])
    MDL -->|uses pydantic| PYDANTIC([External: pydantic])
```

The table below describes the precise role of every import relationship:

| File | Imports From | Purpose |
| - | - | - |
| `inference.py` | `client.py`, `models.py` | Instantiates `CRMDataPipelineEnvClient`; constructs `CRMPipelineAction` objects; drives the observe-act loop with LLM + rule-based fallback |
| `client.py` | `models.py` | Typed wrapper around `openenv.core.EnvClient`; serialises actions via `_step_payload()`; deserialises responses via `_parse_result()` into `CRMPipelineObservation` |
| `server/app.py` | `server/environment.py`, `server/graders.py`, `models.py` | FastAPI bootstrap; creates the ASGI app via `create_fastapi_app`; registers the `/grader/{episode_id}` endpoint; wires `GLOBAL_TRUTH_STORE` and `GLOBAL_ENV_STORE` references |
| `server/environment.py` | `models.py`, `server/data_generator.py` | Owns the POMDP state machine; calls `get_task_data()` on reset; executes all action handlers; writes to the global stores; computes dense rewards |
| `server/graders.py` | `server/environment.py` | Calls `env.get_episode_truth()` and `env.get_final_dataframe()`; implements `evaluate_dataframes()` with Pandas merge and vectorised column matching |
| `server/data_generator.py` | *(external)* faker, pandas, numpy | Generates stochastic corrupted DataFrames and clean hidden-truth snapshots for all three task difficulties |
| `models.py` | *(external)* pydantic | Defines all Enum types and Pydantic models for the full action/observation/state schema; gracefully stubs `openenv.core` base classes if the package is unavailable |

## 3. Episode Lifecycle

The following sequence diagram shows the complete message flow for a single episode, from the initial `/reset` through the step loop to the final grader invocation.

```mermaid theme={null}
sequenceDiagram
    participant Agent as LLM Agent<br/>(inference.py)
    participant Client as CRMDataPipelineEnvClient<br/>(client.py)
    participant API as FastAPI Server<br/>(server/app.py)
    participant Env as CRMDataPipelineEnv<br/>(server/environment.py)
    participant DG as DataGenerator<br/>(server/data_generator.py)
    participant GT as GLOBAL_TRUTH_STORE<br/>(dict)
    participant GR as Grader<br/>(server/graders.py)

    Agent->>Client: env.reset()
    Client->>API: POST /reset
    API->>Env: env.reset()
    Env->>DG: get_task_data(task_id)
    DG-->>Env: {sources, hidden_truth, schema, conflict_rules}
    Env->>Env: episode_id = uuid4()
    Env->>GT: GLOBAL_TRUTH_STORE[episode_id] = hidden_truth.copy()
    Env->>Env: self._sources = sources.copy()
    Env-->>API: CRMStepResult(observation, reward=0.0, done=False)
    API-->>Client: JSON response
    Client-->>Agent: StepResult(observation)

    loop Cleaning Steps (step_count < MAX_STEPS)
        Agent->>Agent: build prompt from observation
        Agent->>Agent: call LLM or fallback baseline
        Agent->>Client: env.step(CRMPipelineAction)
        Client->>API: POST /step {action_type, source, ...}
        API->>Env: env.step(action)
        Env->>Env: state.step_count += 1
        Env->>Env: dispatch to _handle_* method
        Env->>Env: mutate self._sources DataFrame
        Env->>Env: compute dense reward
        Env->>Env: _build_observation(done, reward)
        Env-->>API: CRMStepResult(observation, reward, done)
        API-->>Client: JSON response
        Client-->>Agent: StepResult(observation, reward, done)
    end

    Agent->>Client: SUBMIT_PIPELINE{final_source}
    Client->>API: POST /step {action_type: SUBMIT_PIPELINE}
    API->>Env: env.step(action)
    Env->>Env: check step_count >= MIN_STEPS_BEFORE_SUBMIT (3)
    Env->>Env: self.final_df = self._sources[final_source]
    Env->>Env: schema_match_ratio = matched_cols / total_cols
    Env->>Env: reward = 0.2 × schema_match_ratio
    Env-->>API: CRMStepResult(done=True, reward)
    API-->>Client: JSON
    Client-->>Agent: StepResult(done=True)

    Agent->>API: POST /grader/{episode_id}?final_source=...&task_id=...
    API->>GT: truth_df = GLOBAL_TRUTH_STORE[episode_id][truth_key]
    API->>Env: agent_df = env.final_df
    API->>GR: evaluate_dataframes(truth_df, agent_df)
    GR->>GR: left-join on customer_id
    GR->>GR: column-by-column exact string match
    GR->>GR: score = correct_rows / num_truth_rows
    GR->>GR: penalty = max(0, len(agent_df) - num_truth_rows) × 0.15
    GR-->>API: final_score ∈ [0.01, 0.99]
    API-->>Agent: {episode_id, task_id, score, rows}
```

**1. POST /reset — Episode Initialisation**

The server reads the `TASK_ID` environment variable (default `t1`) and calls `get_task_data(task_id)` from `server/data_generator.py`. This function uses Faker with `seed(None)` to generate fresh stochastic data on every call. The environment deep-copies all source DataFrames into `self._sources` (so mutations don't leak) and deep-copies the hidden truth into `GLOBAL_TRUTH_STORE[episode_id]` (so it can never be modified by agent actions). A UUID is minted via `uuid.uuid4()` to isolate this episode from all others running concurrently.

**2. POST /step (loop) — Action Dispatch**

Each call to `/step` increments `state.step_count` by one, then dispatches to the correct private handler based on `action.action_type`. The handler mutates `self._sources` in place (e.g., calling `df[col].str.lower().str.strip()` for `LOWERCASE_STRIP`) and returns a float reward. After the handler returns, `_build_observation()` constructs a `CRMPipelineObservation` populated with the current task objective, available source names, the most recent data view, the quality report (if profiling has run), and the last action feedback string.

**3. SUBMIT\_PIPELINE — Validation Gate**

Before the episode can end, `step_count` must be at least `MIN_STEPS_BEFORE_SUBMIT = 3`. If your agent tries to submit earlier, it receives `reward = -0.15` and `done = False` — the episode continues. When submission is valid, `self.final_df` is set to `self._sources[action.final_source]`, a heuristic schema-match reward (`0.2 × schema_match_ratio`) is returned, and `done = True`.

**4. POST /grader/\{episode\_id} — Final Scoring**

The `/grader` endpoint is called with the `episode_id`, `final_source`, and `task_id`. It looks up the immutable truth snapshot from `GLOBAL_TRUTH_STORE`, retrieves `env.final_df` from `GLOBAL_ENV_STORE`, and calls `evaluate_dataframes()`. The grader left-joins on `customer_id`, normalises every column to lowercase stripped strings, and checks for exact equality. The final score is `correct_rows / num_truth_rows` minus a bot-excess penalty of `0.15` per surplus row.

## 4. Global State Architecture

Two module-level dictionaries in `server/environment.py` manage concurrent episode isolation:

```python theme={null}
# Keyed by episode UUID → immutable ground truth snapshot
GLOBAL_TRUTH_STORE: dict = {}

# Keyed by episode UUID → live CRMDataPipelineEnv instance
GLOBAL_ENV_STORE: dict = {}
```

**Why do these exist?** The OpenEnv FastAPI server may handle multiple concurrent agent sessions. Each `/reset` call can come from a different agent (or a different parallel evaluation run). Without isolation, one agent's DataFrame mutations would bleed into another agent's session. By keying both stores on a UUID minted at reset time, every episode gets its own copy of the source data and its own immutable truth snapshot.

**How isolation works in practice:**

```python theme={null}
# On reset — deep-copy so mutations can't reach the original
GLOBAL_TRUTH_STORE[episode_id] = {k: v.copy() for k, v in task_data["hidden_truth"].items()}
GLOBAL_ENV_STORE[episode_id]   = self

# Source frames are also copied before being handed to the agent
self._sources = {k: v.copy() for k, v in task_data["sources"].items()}
```

The truth snapshot is written once and never touched again — all action handlers only ever modify `self._sources`, which lives on the `CRMDataPipelineEnv` instance stored in `GLOBAL_ENV_STORE`. The grader reads the truth snapshot directly from `GLOBAL_TRUTH_STORE` at scoring time, guaranteeing the comparison is always against the original uncorrupted data.

> [!NOTE]
>
> In production deployments, these in-memory dictionaries should be replaced with a Redis-backed session store or a database to support truly stateless horizontal scaling. For single-node evaluation the current design is sufficient.

## 5. Reward Signal Architecture

The environment implements a **dense heuristic reward** strategy to overcome the sparse-reward problem inherent in data transformation tasks. Rewards are shaped at three levels: immediate per-action signals, a submission gate penalty, and a terminal schema-match bonus.

| Action | Trigger Condition | Reward |
| - | - | - |
| `VIEW_SOURCE` | Always | `0.0` (no reward — observation only) |
| `PROFILE_SOURCE` | Any valid source | `+0.02` flat |
| `STANDARDIZE_COLUMN` | Successful transformation | `+0.01` base `+ improvement_ratio × 0.10`, capped at `+0.05` |
| `STANDARDIZE_COLUMN` | Exception raised | `-0.05` |
| `HANDLE_MISSING` (DROP\_ROW) | Null rows removed | `(null_count / total_rows) × 0.10`, floor `+0.01` |
| `HANDLE_MISSING` (FILL\_VALUE) | Nulls filled | `(null_count / total_rows) × 0.05`, floor `+0.01` |
| `HANDLE_MISSING` (FILL\_MEAN / FILL\_MODE) | Strategy sent but unimplemented | `-0.05` (raises exception, caught by outer handler) |
| `HANDLE_MISSING` | Exception raised | `-0.05` |
| `DEDUPLICATE` | Rows removed | `(removed / start_len) × 0.20` — ranges from `0.0` (no dups found) to `+0.20` (all rows removed) |
| `DEDUPLICATE` | Exception raised | `-0.05` |
| `MERGE_SOURCES` | Successful merge | `+0.05` |
| `MERGE_SOURCES` | Join key missing | `-0.05` |
| `EXECUTE_SQL` | SELECT executed successfully | `+0.10` |
| `EXECUTE_SQL` | Non-SELECT or forbidden keyword | `-0.10` |
| `EXECUTE_SQL` | SQL syntax error | `-0.10` |
| `SUBMIT_PIPELINE` | `step_count < 3` | `-0.15` (blocked, episode continues) |
| `SUBMIT_PIPELINE` | Valid submission | `0.2 × (matched_schema_cols / total_schema_cols)` |
| Unknown `action_type` | Any | `-0.05` |

The `improvement_ratio` for `STANDARDIZE_COLUMN` is computed without leaking ground truth — it measures how many values in the column contained patterns that indicate they needed cleaning (e.g., uppercase characters, non-digit characters in phone numbers, `/`-separated dates) before the transformation was applied.

The final grader score from `/grader/{episode_id}` is independent of the step-level rewards. It is a row-exact accuracy ratio: `correct_rows / num_truth_rows` minus a `0.15 × excess_rows` penalty for any surplus rows (bots, duplicates) left in the final DataFrame.

## 6. SQL Execution and Safety Guardrails

The `EXECUTE_SQL` action is the most powerful tool in your agent's arsenal — and the most constrained. When invoked, `_handle_sql()` in `server/environment.py` does the following:

**1. Spin up an in-memory SQLite connection**

```python theme={null}
conn = sqlite3.connect(":memory:")
```

A fresh connection is created for every single `EXECUTE_SQL` call. There is no persistent SQL state between actions.

**2. Register all current DataFrames as SQL tables**

```python theme={null}
for name, df in self._sources.items():
    clean_df = df.copy()
    for col in clean_df.columns:
        if clean_df[col].dtype == object:
            clean_df[col] = clean_df[col].astype(str)
    clean_df.to_sql(name, conn, index=False, if_exists='replace')
```

Every source currently in `self._sources` becomes a queryable table — including any previously materialised `merged_output` or `sql_output` tables.

**3. Enforce the SQL safety parser**

```python theme={null}
query_stripped = action.query.strip().upper()

# Rule 1: Must start with SELECT
if not query_stripped.startswith("SELECT "):
    return -0.10  # Blocked

# Rule 2: Forbidden mutation keywords
forbidden = ["DROP ", "DELETE ", "UPDATE ", "INSERT ",
             "ALTER ", "CREATE ", "TRUNCATE "]
if any(cmd in query_stripped for cmd in forbidden):
    return -0.10  # Blocked
```

The parser operates on the **uppercased, stripped** query string. Only `SELECT` queries are permitted. Any query containing `DROP`, `DELETE`, `UPDATE`, `INSERT`, `ALTER`, `CREATE`, or `TRUNCATE` is rejected with a `-0.10` penalty before it reaches SQLite.

**4. Execute and materialise the result**

```python theme={null}
result_df = pd.read_sql_query(action.query, conn)
out_name = action.output_table if action.output_table else "sql_output"
self._sources[out_name] = result_df
```

On success, the result DataFrame is written back into `self._sources` under the `output_table` name you specified. This table is immediately available for subsequent actions and for the `SUBMIT_PIPELINE` step.

> [!NOTE]
>
> The safety parser checks for forbidden keywords anywhere in the query string (e.g., `SELECT * FROM web_forms WHERE customer_id NOT IN (SELECT id FROM DROP_table)` would be blocked because `DROP_` appears in the uppercased string). Write your WHERE clauses and subquery aliases to avoid incidental matches with forbidden keywords.

## 7. Complete Schema Reference

All action/observation/state types are defined in `models.py`. Every field is optional unless marked required (`...`). The complete enum value sets are listed below for quick reference.

### PipelineActionType

| Value | Description |
| - | - |
| `VIEW_SOURCE` | Retrieve a markdown table preview of the first 3 rows of a source |
| `PROFILE_SOURCE` | Generate a statistical quality report (null counts, column types) |
| `STANDARDIZE_COLUMN` | Apply a deterministic transformation to a single column |
| `HANDLE_MISSING` | Resolve null values in a column using a fill or drop strategy |
| `MERGE_SOURCES` | Outer-join two sources on a shared key into `merged_output` |
| `DEDUPLICATE` | Remove duplicate rows from a source using a semantic key |
| `EXECUTE_SQL` | Execute a SELECT query over all current sources via SQLite |
| `SUBMIT_PIPELINE` | End the episode and trigger final grader evaluation |

### StandardizationStrategy

| Value | Effect | Implemented |
| - | - | - |
| `LOWERCASE_STRIP` | `.str.lower().str.strip()` on the column | ✅ |
| `EXTRACT_NUMBERS` | Strips non-digit characters, handles `ext.`/`x` suffixes, prepends `+` | ✅ |
| `TO_DATETIME_ISO` | `pd.to_datetime(errors='coerce')` → `YYYY-MM-DDTHH:MM:SS`, invalid → `""` | ✅ |
| `PHONE_E164` | Defined in enum; not yet implemented in handler — falls back to `+0.01` reward with no column change | ⚠️ |

### MissingStrategy

| Value | Effect | Implemented |
| - | - | - |
| `DROP_ROW` | `df.dropna(subset=[col])` — removes rows where the column is null | ✅ |
| `FILL_VALUE` | `df[col].fillna(action.fallback_value)` — fills nulls with a literal string | ✅ |
| `FILL_MEAN` | Defined in enum; not implemented — raises exception → `-0.05` penalty | ⚠️ |
| `FILL_MODE` | Defined in enum; not implemented — raises exception → `-0.05` penalty | ⚠️ |

### DeduplicationStrategy

| Value | Effect |
| - | - |
| `EXACT_EMAIL` | `df.drop_duplicates(subset=["email"])` — exact string match on email |
| `FUZZY_NAME_PHONE` | Normalises name (lowercase strip) and phone (digits only), then `drop_duplicates` on both |

### ConflictRule

Used with the `MERGE_SOURCES` action to resolve column conflicts when both sources have a non-null value for the same column in the same row.

| Value | Behaviour |
| - | - |
| `PREFER_S1` | Keep the value from `source` (the primary source) when both are non-null |
| `PREFER_S2` | Keep the value from `source2` (the secondary source) when both are non-null |
| `COALESCE` | Keep the first non-null value — equivalent to `s1.combine_first(s2)` |

### CRMPipelineAction Fields

```python theme={null}
action_type:               PipelineActionType   # required
source:                    str | None           # primary source name
source2:                   str | None           # secondary source (MERGE_SOURCES only)
column:                    str | None           # column to mutate
standardization_strategy:  StandardizationStrategy | None
missing_strategy:          MissingStrategy | None
deduplication_strategy:    DeduplicationStrategy | None
fallback_value:            str | None           # literal fill value for FILL_VALUE
join_key:                  str | None           # join column for MERGE_SOURCES
conflict_rule:             ConflictRule | None  # conflict resolution for MERGE_SOURCES
final_source:              str | None           # table name to submit (SUBMIT_PIPELINE)
query:                     str | None           # SQL string (EXECUTE_SQL)
output_table:              str | None           # output table name (EXECUTE_SQL)
```

### CRMPipelineObservation Fields

```python theme={null}
done:                  bool          # True when episode has ended
reward:                float | None  # reward from the last action
current_task_objective: str          # human-readable task description
schema_target:         dict[str,str] # expected column → type map for final output
available_sources:     list[str]     # all table names currently in self._sources
current_view:          str           # markdown table preview from last VIEW_SOURCE
data_quality_report:   str | None    # null/type report from last PROFILE_SOURCE
last_action_feedback:  str           # human-readable result of the last action
conflict_rules:        dict[str,str] | None  # CONFLICT_RULES dict (Task 3 only)
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.