/reset to the moment it receives a final score from /grader/{episode_id}.
1. System Architecture Overview
The runtime topology spans seven source files. The diagram below maps every component to its file and labels the direction of data flow. Each server-side component has a single, well-defined responsibility:server/app.py— FastAPI bootstrap. Callscreate_fastapi_app(CRMDataPipelineEnv, ...)fromopenenv.core, registers a custom/grader/{episode_id}POST endpoint, and exposes a health-checkGET /.server/environment.py— Core POMDP engine. Ownsreset(),step(), all action handlers, and the_build_observation()factory. Holds the two global state stores.server/data_generator.py— Stochastic dataset factory. Called once per/reset. Uses Faker + NumPy + random to produce corrupted source frames and clean hidden-truth frames.server/graders.py— Scoring logic.evaluate_dataframes()performs a left-join between truth and agent output oncustomer_id, checks column-by-column exact string equality, and applies a bot-penalty.models.py— All Pydantic schemas:CRMPipelineAction,CRMPipelineObservation,CRMPipelineState, plus every Enum (PipelineActionType,StandardizationStrategy,MissingStrategy,ConflictRule,DeduplicationStrategy).client.py— Typed REST client. ExtendsEnvClientfromopenenv.core. Implements_step_payload()to serialise actions and_parse_result()to deserialise observations.inference.py— Agent driver. InstantiatesCRMDataPipelineEnvClient, runs the observe-act loop, calls the OpenAI-compatible LLM with retry logic, and falls back toRuleBasedBaselineif the LLM fails.
2. File Interconnections Map
The table below describes the precise role of every import relationship:3. Episode Lifecycle
The following sequence diagram shows the complete message flow for a single episode, from the initial/reset through the step loop to the final grader invocation.
1
POST /reset — Episode Initialisation
The server reads the
TASK_ID environment variable (default t1) and calls get_task_data(task_id) from server/data_generator.py. This function uses Faker with seed(None) to generate fresh stochastic data on every call. The environment deep-copies all source DataFrames into self._sources (so mutations don’t leak) and deep-copies the hidden truth into GLOBAL_TRUTH_STORE[episode_id] (so it can never be modified by agent actions). A UUID is minted via uuid.uuid4() to isolate this episode from all others running concurrently.2
POST /step (loop) — Action Dispatch
Each call to
/step increments state.step_count by one, then dispatches to the correct private handler based on action.action_type. The handler mutates self._sources in place (e.g., calling df[col].str.lower().str.strip() for LOWERCASE_STRIP) and returns a float reward. After the handler returns, _build_observation() constructs a CRMPipelineObservation populated with the current task objective, available source names, the most recent data view, the quality report (if profiling has run), and the last action feedback string.3
SUBMIT_PIPELINE — Validation Gate
Before the episode can end,
step_count must be at least MIN_STEPS_BEFORE_SUBMIT = 3. If your agent tries to submit earlier, it receives reward = -0.15 and done = False — the episode continues. When submission is valid, self.final_df is set to self._sources[action.final_source], a heuristic schema-match reward (0.2 × schema_match_ratio) is returned, and done = True.4
POST /grader/{episode_id} — Final Scoring
The
/grader endpoint is called with the episode_id, final_source, and task_id. It looks up the immutable truth snapshot from GLOBAL_TRUTH_STORE, retrieves env.final_df from GLOBAL_ENV_STORE, and calls evaluate_dataframes(). The grader left-joins on customer_id, normalises every column to lowercase stripped strings, and checks for exact equality. The final score is correct_rows / num_truth_rows minus a bot-excess penalty of 0.15 per surplus row.4. Global State Architecture
Two module-level dictionaries inserver/environment.py manage concurrent episode isolation:
/reset call can come from a different agent (or a different parallel evaluation run). Without isolation, one agent’s DataFrame mutations would bleed into another agent’s session. By keying both stores on a UUID minted at reset time, every episode gets its own copy of the source data and its own immutable truth snapshot.
How isolation works in practice:
self._sources, which lives on the CRMDataPipelineEnv instance stored in GLOBAL_ENV_STORE. The grader reads the truth snapshot directly from GLOBAL_TRUTH_STORE at scoring time, guaranteeing the comparison is always against the original uncorrupted data.
In production deployments, these in-memory dictionaries should be replaced with a Redis-backed session store or a database to support truly stateless horizontal scaling. For single-node evaluation the current design is sufficient.
5. Reward Signal Architecture
The environment implements a dense heuristic reward strategy to overcome the sparse-reward problem inherent in data transformation tasks. Rewards are shaped at three levels: immediate per-action signals, a submission gate penalty, and a terminal schema-match bonus.
The
improvement_ratio for STANDARDIZE_COLUMN is computed without leaking ground truth — it measures how many values in the column contained patterns that indicate they needed cleaning (e.g., uppercase characters, non-digit characters in phone numbers, /-separated dates) before the transformation was applied.
The final grader score from /grader/{episode_id} is independent of the step-level rewards. It is a row-exact accuracy ratio: correct_rows / num_truth_rows minus a 0.15 × excess_rows penalty for any surplus rows (bots, duplicates) left in the final DataFrame.
6. SQL Execution and Safety Guardrails
TheEXECUTE_SQL action is the most powerful tool in your agent’s arsenal — and the most constrained. When invoked, _handle_sql() in server/environment.py does the following:
1
Spin up an in-memory SQLite connection
EXECUTE_SQL call. There is no persistent SQL state between actions.2
Register all current DataFrames as SQL tables
self._sources becomes a queryable table — including any previously materialised merged_output or sql_output tables.3
Enforce the SQL safety parser
SELECT queries are permitted. Any query containing DROP, DELETE, UPDATE, INSERT, ALTER, CREATE, or TRUNCATE is rejected with a -0.10 penalty before it reaches SQLite.4
Execute and materialise the result
self._sources under the output_table name you specified. This table is immediately available for subsequent actions and for the SUBMIT_PIPELINE step.The safety parser checks for forbidden keywords anywhere in the query string (e.g.,
SELECT * FROM web_forms WHERE customer_id NOT IN (SELECT id FROM DROP_table) would be blocked because DROP_ appears in the uppercased string). Write your WHERE clauses and subquery aliases to avoid incidental matches with forbidden keywords.7. Complete Schema Reference
All action/observation/state types are defined inmodels.py. Every field is optional unless marked required (...). The complete enum value sets are listed below for quick reference.
PipelineActionType
StandardizationStrategy
MissingStrategy
DeduplicationStrategy
ConflictRule
Used with theMERGE_SOURCES action to resolve column conflicts when both sources have a non-null value for the same column in the same row.