Task Catalogue: Three Difficulty Tiers of CRM Cleaning
Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.
The CRM Data Pipeline benchmark exposes three tasks of escalating complexity, each spawned fresh on every /reset call using Faker with a randomised seed. No two episodes share the same data — your agent must generalise a cleaning strategy, not memorise a specific dataset. The difficulty progression mirrors real enterprise data engineering escalation: a single messy source, then two sources with conflicting records, then a three-way merge with divergent schemas and per-column conflict resolution rules.
Calling SUBMIT_PIPELINE before completing at least 3 action steps in any task incurs a hard penalty of -0.15 and the episode does not end. The minimum-steps gate (MIN_STEPS_BEFORE_SUBMIT = 3) is enforced in server/environment.py regardless of how clean your data already is. Always run at least one PROFILE_SOURCE and one STANDARDIZE_COLUMN before submitting.
Task 1 confronts your agent with a single source — web_forms — that has been heavily corrupted by the correlate_truth() function in server/data_generator.py. The source starts with 40 real customer rows whose ground-truth values are clean, ISO-formatted, E.164-formatted, and lowercase. Before your agent sees them, correlate_truth() runs every row through a randomised corruption pipeline and appends 8 bot rows. Your agent must undo all of that corruption.
The web_forms DataFrame your agent receives contains approximately 40 real rows (some will have fuzzy duplicates appended, making the raw row count higher) plus 8 synthetic bot rows injected at the end and then shuffled. The following corruptions are applied stochastically per row:Missing Data (20% probability per field)
Approximately 20% of rows will have email set to None and independently ~20% will have phone set to None.String Chaos (50% probability per field)
dirty["name"] = f" {dirty['name']} " or dirty["name"].upper()dirty["email"] = dirty["email"].upper()
Names gain leading and trailing whitespace, or are uppercased entirely. Emails are uppercased in roughly half of all rows.Date Chaos (tiered random)
Probability Range
Corruption Applied
> 0.85
"??/??/????" — completely invalid placeholder
> 0.70
"2023-13-12" — invalid month 13
> 0.55
None — date dropped entirely
≤ 0.55
"YYYY/MM/DD" — valid date but /-separated, not ISO 8601
Phone Chaos (tiered random, only when phone is non-null)
Probability Range
Corruption Applied
> 0.85
"12345678901 ext. 42" — extension suffix appended
> 0.70
"(123) 456-78901" — US parenthetical format
> 0.55
"Invalid Number" — literal string
≤ 0.55
Raw digit string without + prefix
Fuzzy Duplicate Injection (25% probability per row)After corrupting a row, correlate_truth() rolls a 25% chance to append a near-duplicate: the last name is kept, the first name is replaced with a random Faker first name, and the email field is set to None. These fuzzy duplicates test your agent’s DEDUPLICATE action.
Leading/trailing whitespace stripped; any casing is accepted
email
string
Lowercase, leading/trailing whitespace stripped
signup_date
string
ISO 8601 (YYYY-MM-DDTHH:MM:SS) or empty string ""
phone
string
E.164 format (+{country_code}{number}) or empty string ""
The grader compares each column by lowercasing and stripping both the truth value and your submitted value before checking equality, so case differences in name will not fail rows. However, signup_date and phone must match the exact format — 2023/01/15 will not match 2023-01-15T00:00:00.
The EXTRACT_NUMBERS handler strips extension suffixes, removes all non-digit characters, and prepends +. An all-zeros phone (+00000000000) from a bot row will survive this step but will be filtered in the SQL step.
Calls df.drop_duplicates(subset=["email"]). Fuzzy duplicates that have email = None are retained but will not match real customers in the grader join. Filter them next.
7
Filter bots and nulls via SQL
{ "action_type": "EXECUTE_SQL", "query": "SELECT * FROM web_forms WHERE email IS NOT NULL AND email NOT LIKE '%promo-alerts%' AND email NOT LIKE '%click-rewards%' AND email NOT LIKE '%freewinners%' AND email NOT LIKE '%bulk-mail%' AND phone != '+00000000000'", "output_table": "web_forms_clean"}
Earns +0.10. This is your bot-removal step. Alternatively, filter on customer_id NOT LIKE 'BOT%'.
Task 2 introduces a second data source. Your agent now has web_forms (the same corrupted dataset from Task 1) and legacy_db (a second, independently corrupted dataset containing 20 base customers plus 5 bot rows). The ground truth is a merged, deduplicated set of all unique customers across both sources — keyed to the merged_output table name.
web_forms — identical generation to Task 1: 40 real rows corrupted with the full chaos engine, 8 bot rows injected.legacy_db — generated by a second call to create_base_truth(20) followed by correlate_truth(). This produces ~20 base rows (with potential fuzzy duplicate appendages) plus 5 bot rows, then shuffled. The column names are identical to web_forms (customer_id, name, email, signup_date, phone) — but the data quality issues are independently randomised, so a customer that has a clean email in web_forms may have a corrupted email in legacy_db.
The challenge in Task 2 is not just cleaning two sources — it is producing the correct merged set size. The truth contains up to 60 unique customers (40 from web_forms + 20 from legacy_db, minus any email-level overlap). If your agent over-deduplicates (removing real rows) or under-deduplicates (keeping both copies of the same customer), the grader’s row count penalty activates.The evaluate_dataframes() grader still joins on customer_id, which means both sources’ records must survive the merge with their original customer_id values intact — your agent cannot drop customer_id or change it during the merge.
Each profile earns +0.02. Examine the null counts for both sources before standardising.
2
Standardize web_forms columns
Apply the same four-step standardization from Task 1 to web_forms: email LOWERCASE_STRIP, name LOWERCASE_STRIP, phone EXTRACT_NUMBERS, signup_date TO_DATETIME_ISO.
3
Filter web_forms bots via SQL
{ "action_type": "EXECUTE_SQL", "query": "SELECT * FROM web_forms WHERE customer_id NOT LIKE 'BOT%' AND email IS NOT NULL", "output_table": "web_forms_clean"}
4
Standardize legacy_db columns
Apply the same four-step standardization to legacy_db: email LOWERCASE_STRIP, name LOWERCASE_STRIP, phone EXTRACT_NUMBERS, signup_date TO_DATETIME_ISO.
5
Filter legacy_db bots via SQL
{ "action_type": "EXECUTE_SQL", "query": "SELECT * FROM legacy_db WHERE customer_id NOT LIKE 'BOT%' AND email IS NOT NULL", "output_table": "legacy_db_clean"}
6
UNION both sources
{ "action_type": "EXECUTE_SQL", "query": "SELECT * FROM web_forms_clean UNION ALL SELECT * FROM legacy_db_clean", "output_table": "merged_output"}
UNION ALL preserves all rows including cross-source duplicates. You’ll deduplicate in the next step.
The UNION ALL + DEDUPLICATE pattern is more reliable than MERGE_SOURCES for Task 2 because DEDUPLICATE uses Pandas drop_duplicates() which handles None emails gracefully (they are not treated as matching duplicates), whereas the MERGE_SOURCES outer join can produce Cartesian-product issues when the join key has many nulls.
Task 3 is the full-complexity benchmark. Your agent must unify three sources — salesforce, web_leads, and legacy_db — each with a different column schema, independently injected bots, and explicit conflict resolution rules that dictate which source wins when the same customer has different values across sources.
The three sources are generated from the same 60-customer base truth, then individually subsampled, corrupted, and augmented with bots:salesforce — 90% sample (~54 rows) + 5 bots
Column
Notes
customer_id
CUST_{n} format — canonical identifier
email
Lowercase — this is the authoritative email
phone
Present ~70% of the time (None otherwise)
web_leads — 80% sample (~48 rows) + 5 bots
Column
Notes
customer_id
Nullable ~20% of the time — some web sign-ups don’t yet have a CRM ID
email
Lowercase — matches Salesforce email
phone
Fresh phone number (independently generated via fake.msisdn()), present ~60% of the time
legacy_db — 85% sample (~51 rows) + 5 bots
Column
Notes
legacy_id
Uses OLD-{n} prefix instead of CUST_{n} (e.g., OLD-1000 maps to CUST_1000)
contact_email
UPPERCASE — must be lowercased before any join
home_phone
Original phone from base truth — older than web_leads
The CONFLICT_RULES dictionary from server/data_generator.py is exposed to your agent in every observation as conflict_rules:
CONFLICT_RULES = { "email": "prefer_salesforce", # SF email is canonical "phone": "prefer_web_leads", # Web leads has fresher phone data "customer_id": "prefer_salesforce",}
The hidden truth is built by applying these rules:
for r in truth: resolved_phone = wl_phones.get(r["customer_id"], r["phone"]) # WL > SF resolved_truth.append({ "customer_id": r["customer_id"], # prefer_salesforce "email": r["email"], # prefer_salesforce "phone": resolved_phone, # prefer_web_leads })
wl_phones maps every customer_id to the fresh phone generated for the web_leads source. If web_leads had a phone for that customer, the truth phone is the web_leads phone. If not, it falls back to the Salesforce phone.
Bot domains across all sources: promo-alerts.net, click-rewards.xyz, freewinners.info, bulk-mail.co.Filtering bots requires checking both the customer_id / legacy_id prefix (BOT) and the email domain. Note that legacy_db bot emails are uppercase (BULK@FREEWINNERS.INFO) — standardise contact_email before filtering.
Calling SUBMIT_PIPELINE before 3 steps in Task 3 incurs -0.15 and the episode continues. Task 3 requires the most preparation of any task — budget at least 7-8 actions for a complete pipeline.
The recommended approach uses a single EXECUTE_SQL action with a GROUP BY email aggregation over a UNION ALL of all three standardised sources:
SELECT MAX(customer_id) AS customer_id, email, MAX(phone) AS phoneFROM ( -- Salesforce: canonical email, customer_id, and SF phone SELECT customer_id, email, phone FROM salesforce WHERE customer_id NOT LIKE 'BOT%' AND email NOT LIKE '%promo-alerts%' AND email NOT LIKE '%click-rewards%' AND email NOT LIKE '%freewinners%' AND email NOT LIKE '%bulk-mail%' UNION ALL -- Web Leads: fresh phone, filter bot domains and null customer_id SELECT customer_id, email, phone FROM web_leads WHERE customer_id IS NOT NULL AND email NOT LIKE '%promo-alerts%' AND email NOT LIKE '%click-rewards%' AND email NOT LIKE '%freewinners%' AND email NOT LIKE '%bulk-mail%' UNION ALL -- Legacy DB: remap legacy_id → customer_id, lowercase email already applied SELECT REPLACE(legacy_id, 'OLD-', 'CUST_') AS customer_id, contact_email AS email, home_phone AS phone FROM legacy_db WHERE legacy_id NOT LIKE 'BOT%' AND contact_email NOT LIKE '%PROMO-ALERTS%' AND contact_email NOT LIKE '%BULK-MAIL%') subGROUP BY email
The GROUP BY email with MAX(customer_id) implements the prefer_salesforce rule — because CUST_ sorts lexicographically after OLD-, MAX() picks the Salesforce-format ID. The MAX(phone) implements a loose approximation of prefer_web_leads (fresh phones generated by Faker are not necessarily lexicographically larger, so this is imperfect — the deterministic RuleBasedBaseline in inference.py achieves perfection through the specific SQL shown there).
Remember to standardise legacy_db.contact_email to lowercase with STANDARDIZE_COLUMNbefore executing the SQL join. The SQL LIKE patterns above are case-sensitive in SQLite by default for ASCII characters, so an uppercase BULK@FREEWINNERS.INFO domain will not be caught by a lowercase LIKE '%bulk-mail%' filter unless the column has already been lowercased.
The contact_email column is UPPERCASE by design. Failing to lowercase it before the SQL join will cause the GROUP BY email to treat ALICE@EXAMPLE.COM and alice@example.com as different customers, exploding the row count.
5
Execute the 3-way UNION + GROUP BY SQL merge
{ "action_type": "EXECUTE_SQL", "query": "SELECT MAX(customer_id) AS customer_id, email, MAX(phone) AS phone FROM (SELECT customer_id, email, phone FROM salesforce WHERE customer_id NOT LIKE 'BOT%' AND email NOT LIKE '%promo-alerts%' UNION ALL SELECT customer_id, email, phone FROM web_leads WHERE customer_id IS NOT NULL AND email NOT LIKE '%promo-alerts%' UNION ALL SELECT REPLACE(legacy_id, 'OLD-', 'CUST_') AS customer_id, contact_email AS email, home_phone AS phone FROM legacy_db WHERE legacy_id NOT LIKE 'BOT%') GROUP BY email", "output_table": "merged_output"}
On success, earns +0.10. The merged_output table now contains one row per unique email address with the conflict-resolved customer_id and phone.
The grader will compare merged_output against the hidden truth built from _apply_conflict_rules() in data_generator.py.
The inference.pyRuleBasedBaseline for Task 3 achieves a perfect score on every episode. If your LLM agent is underperforming on Task 3, compare its SQL output row count against len(truth_df) using PROFILE_SOURCE on merged_output after the SQL step. Excess rows always point to bot rows that slipped through your WHERE filters.
{"docsConfig":{"theme":"aspen","$schema":"https://mintlify.com/docs.json","name":"CRM Data Pipeline","colors":{"primary":"#f8c61e","light":"#fbf9ec","dark":"#4e412f"},"favicon":"https://media.brand.dev/87b75398-73c9-4b33-bea3-6d01873af484.jpg","navbar":{"primary":{"type":"github","href":"https://github.com/GaurRitika/CRM-Datapipeline-env"}},"styling":{"latex":false},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"ritika-fa61c43f","hasAudioNarration":false,"actualSubdomain":"ritika-fa61c43f","customDomains":[],"gitSource":{"type":"github","owner":"gaurritika","repo":"docs","deployBranch":"main","contentDirectory":"","isPrivate":false,"mountPath":"","label":""},"entitlements":{"AI_CHAT":{"status":"DISABLED"}},"buildId":"6aea20fdf3fc705a21a57590:success","clientVersion":"0.0.4094","searchFilterCounts":[{"language":"default","tag":"Untitled Product","count":27},{"language":"default","tag":"Documentation","count":19},{"language":"default","tag":"API Reference","count":8},{"language":"default","tag":"REST Endpoints","count":5},{"language":"default","tag":"Data Engine","count":5},{"language":"default","tag":"Core Concepts","count":4},{"language":"default","tag":"Guides","count":4},{"language":"default","tag":"Data Models","count":3},{"language":"default","tag":"Overview","count":3},{"language":"default","tag":"Getting Started","count":3}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[],"page":{"pagePath":"task-catalogue","slug":"/task-catalogue","pageMetadata":{"title":"Task Catalogue: Three Difficulty Tiers of CRM Cleaning","sidebarTitle":"Task Catalogue","description":"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.","href":"/task-catalogue"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={code:`code`,li:`li`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,ul:`ul`,..._provideComponents(),...props.components},{CodeBlock,Heading,Note,Step,Steps,Tab,Table,Tabs,Tip,Warning}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!Step)_missingMdxReference(`Step`,true);if(!Steps)_missingMdxReference(`Steps`,true);if(!Tab)_missingMdxReference(`Tab`,true);if(!Table)_missingMdxReference(`Table`,true);if(!Tabs)_missingMdxReference(`Tabs`,true);if(!Tip)_missingMdxReference(`Tip`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`The CRM Data Pipeline benchmark exposes three tasks of escalating complexity, each spawned fresh on every `,_jsx(_components.code,{children:`/reset`}),` call using Faker with a randomised seed. No two episodes share the same data — your agent must generalise a cleaning strategy, not memorise a specific dataset. The difficulty progression mirrors real enterprise data engineering escalation: a single messy source, then two sources with conflicting records, then a three-way merge with divergent schemas and per-column conflict resolution rules.`]}),`\n`,_jsx(Warning,{children:_jsxs(_components.p,{children:[`Calling `,_jsx(_components.code,{children:`SUBMIT_PIPELINE`}),` before completing at least `,_jsx(_components.strong,{children:`3 action steps`}),` in any task incurs a hard penalty of `,_jsx(_components.strong,{children:`-0.15`}),` and the episode does `,_jsx(_components.strong,{children:`not`}),` end. The minimum-steps gate (`,_jsx(_components.code,{children:`MIN_STEPS_BEFORE_SUBMIT = 3`}),`) is enforced in `,_jsx(_components.code,{children:`server/environment.py`}),` regardless of how clean your data already is. Always run at least one `,_jsx(_components.code,{children:`PROFILE_SOURCE`}),` and one `,_jsx(_components.code,{children:`STANDARDIZE_COLUMN`}),` before submitting.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`task-comparison-at-a-glance`,children:`Task Comparison at a Glance`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Property`}),_jsx(_components.th,{children:`Task 1 — Easy`}),_jsx(_components.th,{children:`Task 2 — Medium`}),_jsx(_components.th,{children:`Task 3 — Hard`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Task ID`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`t1`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`t2`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`t3`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Data Sources`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`web_forms`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`web_forms`}),`, `,_jsx(_components.code,{children:`legacy_db`})]}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`salesforce`}),`, `,_jsx(_components.code,{children:`web_leads`}),`, `,_jsx(_components.code,{children:`legacy_db`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Real Rows (approx.)`})}),_jsx(_components.td,{children:`~40 base + fuzzy dups`}),_jsx(_components.td,{children:`~40 + ~20 base`}),_jsx(_components.td,{children:`~60 across all three`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Bot Rows`})}),_jsx(_components.td,{children:`8 (web_forms)`}),_jsx(_components.td,{children:`8 (web_forms) + 5 (legacy_db) = 13 total`}),_jsx(_components.td,{children:`5 per source (15 total)`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Schema Divergence`})}),_jsx(_components.td,{children:`None — single source`}),_jsx(_components.td,{children:`Moderate — same columns, different data quality`}),_jsx(_components.td,{children:`Severe — different column names, uppercase emails, old ID format`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Conflict Rules`})}),_jsx(_components.td,{children:`None`}),_jsx(_components.td,{children:`None`}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`prefer_salesforce`}),` for email/customer_id, `,_jsx(_components.code,{children:`prefer_web_leads`}),` for phone`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Ground Truth Key`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`web_forms`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`merged_output`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`merged_output`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Grader Join Key`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Target Schema Columns`})}),_jsx(_components.td,{\"data-numeric\":`true`,children:`5`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`5`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`3`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Primary Challenge`})}),_jsx(_components.td,{children:`Date/phone/email normalisation + bot removal`}),_jsx(_components.td,{children:`Cross-source deduplication`}),_jsx(_components.td,{children:`3-way JOIN + conflict resolution + bot filtering`})]})]})]}),`\n`,_jsxs(Tabs,{children:[_jsxs(Tab,{title:`Task 1 — Easy`,id:`task-1-%E2%80%94-easy`,\"data-child-heading-ids\":`[\"task-1%3A-web-forms-normalization\",\"input-data-characteristics\",\"bot-row-schema\",\"ground-truth-target-schema\",\"scoring\",\"recommended-action-sequence\"]`,children:[_jsx(Heading,{level:`2`,id:`task-1-web-forms-normalization`,children:`Task 1: Web Forms Normalization`}),_jsxs(_components.p,{children:[`Task 1 confronts your agent with a single source — `,_jsx(_components.code,{children:`web_forms`}),` — that has been heavily corrupted by the `,_jsx(_components.code,{children:`correlate_truth()`}),` function in `,_jsx(_components.code,{children:`server/data_generator.py`}),`. The source starts with 40 real customer rows whose ground-truth values are clean, ISO-formatted, E.164-formatted, and lowercase. Before your agent sees them, `,_jsx(_components.code,{children:`correlate_truth()`}),` runs every row through a randomised corruption pipeline and appends 8 bot rows. Your agent must undo all of that corruption.`]}),_jsx(Heading,{level:`3`,id:`input-data-characteristics`,children:`Input Data Characteristics`}),_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`web_forms`}),` DataFrame your agent receives contains approximately `,_jsx(_components.strong,{children:`40 real rows`}),` (some will have fuzzy duplicates appended, making the raw row count higher) plus `,_jsx(_components.strong,{children:`8 synthetic bot rows`}),` injected at the end and then shuffled. The following corruptions are applied stochastically per row:`]}),_jsx(_components.p,{children:_jsx(_components.strong,{children:`Missing Data (20% probability per field)`})}),_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` random.random() `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`>`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 0.8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: dirty[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` None`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` random.random() `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`>`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 0.8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: dirty[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` None`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Approximately 20% of rows will have `,_jsx(_components.code,{children:`email`}),` set to `,_jsx(_components.code,{children:`None`}),` and independently ~20% will have `,_jsx(_components.code,{children:`phone`}),` set to `,_jsx(_components.code,{children:`None`}),`.`]}),_jsx(_components.p,{children:_jsx(_components.strong,{children:`String Chaos (50% probability per field)`})}),_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`dirty[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"name\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\" `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`dirty[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'name'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` or`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` dirty[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"name\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`].upper()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`dirty[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` dirty[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`].upper()`})]}),`\n`]})})}),_jsx(_components.p,{children:`Names gain leading and trailing whitespace, or are uppercased entirely. Emails are uppercased in roughly half of all rows.`}),_jsx(_components.p,{children:_jsx(_components.strong,{children:`Date Chaos (tiered random)`})}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Probability Range`}),_jsx(_components.th,{children:`Corruption Applied`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`> 0.85`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`\"??/??/????\"`}),` — completely invalid placeholder`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`> 0.70`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`\"2023-13-12\"`}),` — invalid month 13`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`> 0.55`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`None`}),` — date dropped entirely`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`≤ 0.55`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`\"YYYY/MM/DD\"`}),` — valid date but `,_jsx(_components.code,{children:`/`}),`-separated, not ISO 8601`]})]})]})]}),_jsx(_components.p,{children:_jsx(_components.strong,{children:`Phone Chaos (tiered random, only when phone is non-null)`})}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Probability Range`}),_jsx(_components.th,{children:`Corruption Applied`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`> 0.85`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`\"12345678901 ext. 42\"`}),` — extension suffix appended`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`> 0.70`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`\"(123) 456-78901\"`}),` — US parenthetical format`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`> 0.55`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`\"Invalid Number\"`}),` — literal string`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`≤ 0.55`})}),_jsxs(_components.td,{children:[`Raw digit string without `,_jsx(_components.code,{children:`+`}),` prefix`]})]})]})]}),_jsx(_components.p,{children:_jsx(_components.strong,{children:`Fuzzy Duplicate Injection (25% probability per row)`})}),_jsxs(_components.p,{children:[`After corrupting a row, `,_jsx(_components.code,{children:`correlate_truth()`}),` rolls a 25% chance to append a near-duplicate: the last name is kept, the first name is replaced with a random Faker first name, and the `,_jsx(_components.code,{children:`email`}),` field is set to `,_jsx(_components.code,{children:`None`}),`. These fuzzy duplicates test your agent’s `,_jsx(_components.code,{children:`DEDUPLICATE`}),` action.`]}),_jsx(Heading,{level:`3`,id:`bot-row-schema`,children:`Bot Row Schema`}),_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`7`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"BOT{5-digit-number}\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# e.g. BOT42381`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"name\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Support Team\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# or Newsletter Bot, Promo Sender, Deals Alert`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"user42@promo-alerts.net\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"signup_date\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"1970-01-01\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# epoch — suspicious sentinel`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"+00000000000\"`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` # all-zeros — clear anomaly`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`Bot domains are: `,_jsx(_components.code,{children:`promo-alerts.net`}),`, `,_jsx(_components.code,{children:`click-rewards.xyz`}),`, `,_jsx(_components.code,{children:`freewinners.info`}),`, `,_jsx(_components.code,{children:`bulk-mail.co`}),`.`]}),_jsx(Heading,{level:`3`,id:`ground-truth-target-schema`,children:`Ground Truth Target Schema`}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Column`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Format Requirement`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[`Original `,_jsx(_components.code,{children:`CUST_{n}`}),` identifier, unchanged`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`name`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`Leading/trailing whitespace stripped; any casing is accepted`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`email`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[_jsx(_components.strong,{children:`Lowercase`}),`, leading/trailing whitespace stripped`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`signup_date`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[_jsx(_components.strong,{children:`ISO 8601`}),` (`,_jsx(_components.code,{children:`YYYY-MM-DDTHH:MM:SS`}),`) or empty string `,_jsx(_components.code,{children:`\"\"`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`phone`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[_jsx(_components.strong,{children:`E.164 format`}),` (`,_jsx(_components.code,{children:`+{country_code}{number}`}),`) or empty string `,_jsx(_components.code,{children:`\"\"`})]})]})]})]}),_jsxs(_components.p,{children:[`The grader compares each column by lowercasing and stripping both the truth value and your submitted value before checking equality, so case differences in `,_jsx(_components.code,{children:`name`}),` will not fail rows. However, `,_jsx(_components.code,{children:`signup_date`}),` and `,_jsx(_components.code,{children:`phone`}),` must match the exact format — `,_jsx(_components.code,{children:`2023/01/15`}),` will not match `,_jsx(_components.code,{children:`2023-01-15T00:00:00`}),`.`]}),_jsx(Heading,{level:`3`,id:`scoring`,children:`Scoring`}),_jsxs(_components.p,{children:[`Your final score is computed by `,_jsx(_components.code,{children:`evaluate_dataframes(truth_df, agent_df)`}),` in `,_jsx(_components.code,{children:`server/graders.py`}),`:`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`17`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`17`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Left-join truth to agent on customer_id`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`merged `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` pd.merge(truth_df, agent_df.drop_duplicates(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`subset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]),`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:` on`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`suffixes`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"_truth\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"_agent\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`), `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`how`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"left\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Per-column exact match (case-insensitive, strip-normalised)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` col `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` truth_df.columns:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` col `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`continue`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` s_truth `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` merged[`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`_truth\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`].fillna(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`).astype(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:`str`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`).str.strip().str.lower()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` s_agent `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` merged[`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`_agent\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`].fillna(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`).astype(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:`str`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`).str.strip().str.lower()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` merged[`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`_match\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (s_truth `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`==`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` s_agent)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Row is correct only if ALL columns match`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`score `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` merged[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"row_correct\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`].sum() `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`/`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` len`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(truth_df)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Bot/duplicate penalty`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`penalty `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` max`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:`len`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(agent_df) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`-`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` len`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(truth_df)) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`*`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:` 0.15`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`final_score `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` min`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0.99`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:`max`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0.01`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, score `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`-`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` penalty))`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Leaving even one bot row in your submission costs you `,_jsx(_components.code,{children:`0.15`}),` of score. Failing to deduplicate fuzzy rows means extra rows are penalised.`]}),_jsx(Heading,{level:`3`,id:`recommended-action-sequence`,children:`Recommended Action Sequence`}),_jsxs(Steps,{children:[_jsxs(Step,{title:`Profile the source`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"PROFILE_SOURCE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Earns `,_jsx(_components.code,{children:`+0.02`}),`. The quality report will show null counts per column and data types, giving your agent the context it needs to prioritise transformations.`]})]}),_jsxs(Step,{title:`Standardize email (lowercase + strip)`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`6`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"LOWERCASE_STRIP\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`Calls `,_jsx(_components.code,{children:`df[\"email\"].str.lower().str.strip()`}),`. Earns up to `,_jsx(_components.code,{children:`+0.05`}),` based on how many values were uppercase.`]})]}),_jsx(Step,{title:`Standardize name (strip whitespace)`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`6`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"name\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"LOWERCASE_STRIP\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})})}),_jsxs(Step,{title:`Standardize phone (extract to E.164)`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`6`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXTRACT_NUMBERS\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`EXTRACT_NUMBERS`}),` handler strips extension suffixes, removes all non-digit characters, and prepends `,_jsx(_components.code,{children:`+`}),`. An all-zeros phone (`,_jsx(_components.code,{children:`+00000000000`}),`) from a bot row will survive this step but will be filtered in the SQL step.`]})]}),_jsxs(Step,{title:`Standardize signup_date to ISO 8601`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`6`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"signup_date\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"TO_DATETIME_ISO\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`Uses `,_jsx(_components.code,{children:`pd.to_datetime(errors='coerce')`}),` — invalid dates like `,_jsx(_components.code,{children:`??/??/????`}),` and `,_jsx(_components.code,{children:`2023-13-12`}),` become `,_jsx(_components.code,{children:`NaT`}),` and are then written as `,_jsx(_components.code,{children:`\"\"`}),`.`]})]}),_jsxs(Step,{title:`Deduplicate by email`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"DEDUPLICATE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"deduplication_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXACT_EMAIL\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`Calls `,_jsx(_components.code,{children:`df.drop_duplicates(subset=[\"email\"])`}),`. Fuzzy duplicates that have `,_jsx(_components.code,{children:`email = None`}),` are retained but will not match real customers in the grader join. Filter them next.`]})]}),_jsxs(Step,{title:`Filter bots and nulls via SQL`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXECUTE_SQL\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"query\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SELECT * FROM web_forms WHERE email IS NOT NULL AND email NOT LIKE '%promo-alerts%' AND email NOT LIKE '%click-rewards%' AND email NOT LIKE '%freewinners%' AND email NOT LIKE '%bulk-mail%' AND phone != '+00000000000'\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"output_table\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms_clean\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`Earns `,_jsx(_components.code,{children:`+0.10`}),`. This is your bot-removal step. Alternatively, filter on `,_jsx(_components.code,{children:`customer_id NOT LIKE 'BOT%'`}),`.`]})]}),_jsxs(Step,{title:`Submit the pipeline`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SUBMIT_PIPELINE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"final_source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms_clean\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Earns `,_jsx(_components.code,{children:`0.2 × schema_match_ratio`}),` as a step reward. The full terminal score comes from the grader.`]})]})]})]}),_jsxs(Tab,{title:`Task 2 — Medium`,id:`task-2-%E2%80%94-medium`,\"data-child-heading-ids\":`[\"task-2%3A-legacy-db-deduplication\",\"data-sources\",\"ground-truth-construction\",\"ground-truth-target-schema-2\",\"what-makes-this-harder\",\"recommended-action-sequence-2\"]`,children:[_jsx(Heading,{level:`2`,id:`task-2-legacy-db-deduplication`,children:`Task 2: Legacy DB Deduplication`}),_jsxs(_components.p,{children:[`Task 2 introduces a second data source. Your agent now has `,_jsx(_components.code,{children:`web_forms`}),` (the same corrupted dataset from Task 1) and `,_jsx(_components.code,{children:`legacy_db`}),` (a second, independently corrupted dataset containing 20 base customers plus 5 bot rows). The ground truth is a `,_jsx(_components.strong,{children:`merged, deduplicated`}),` set of all unique customers across both sources — keyed to the `,_jsx(_components.code,{children:`merged_output`}),` table name.`]}),_jsx(Heading,{level:`3`,id:`data-sources`,children:`Data Sources`}),_jsxs(_components.p,{children:[_jsx(_components.strong,{children:_jsx(_components.code,{children:`web_forms`})}),` — identical generation to Task 1: 40 real rows corrupted with the full chaos engine, 8 bot rows injected.`]}),_jsxs(_components.p,{children:[_jsx(_components.strong,{children:_jsx(_components.code,{children:`legacy_db`})}),` — generated by a second call to `,_jsx(_components.code,{children:`create_base_truth(20)`}),` followed by `,_jsx(_components.code,{children:`correlate_truth()`}),`. This produces ~20 base rows (with potential fuzzy duplicate appendages) plus 5 bot rows, then shuffled. The column names are identical to `,_jsx(_components.code,{children:`web_forms`}),` (`,_jsx(_components.code,{children:`customer_id`}),`, `,_jsx(_components.code,{children:`name`}),`, `,_jsx(_components.code,{children:`email`}),`, `,_jsx(_components.code,{children:`signup_date`}),`, `,_jsx(_components.code,{children:`phone`}),`) — but the data quality issues are independently randomised, so a customer that has a clean email in `,_jsx(_components.code,{children:`web_forms`}),` may have a corrupted email in `,_jsx(_components.code,{children:`legacy_db`}),`.`]}),_jsx(Heading,{level:`3`,id:`ground-truth-construction`,children:`Ground Truth Construction`}),_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# From server/data_generator.py — generate_medium_task()`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`df_truth `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` pd.concat([`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` web_forms_truth, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# 40 clean rows`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` pd.DataFrame(truth2) `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# 20 clean rows`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]).drop_duplicates(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`subset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]).reset_index(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`drop`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`True`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`The hidden truth is the outer union of both sources’ clean rows, deduplicated by `,_jsx(_components.code,{children:`email`}),`. This means:`]}),_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[`If a customer appears in both sources (same email), only `,_jsx(_components.strong,{children:`one`}),` row is in the truth.`]}),`\n`,_jsx(_components.li,{children:`If a customer appears in only one source, that row is still in the truth.`}),`\n`,_jsxs(_components.li,{children:[`The truth is keyed to `,_jsx(_components.code,{children:`merged_output`}),` — your agent must create a table by that exact name.`]}),`\n`]}),_jsx(Heading,{level:`3`,id:`ground-truth-target-schema-2`,children:`Ground Truth Target Schema`}),_jsx(_components.p,{children:`Same as Task 1:`}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Column`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Format Requirement`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`CUST_{n}`}),` format`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`name`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`Stripped`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`email`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`Lowercase, stripped`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`signup_date`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[`ISO 8601 or `,_jsx(_components.code,{children:`\"\"`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`phone`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[`E.164 or `,_jsx(_components.code,{children:`\"\"`})]})]})]})]}),_jsx(Heading,{level:`3`,id:`what-makes-this-harder`,children:`What Makes This Harder`}),_jsxs(_components.p,{children:[`The challenge in Task 2 is not just cleaning two sources — it is `,_jsx(_components.strong,{children:`producing the correct merged set size`}),`. The truth contains up to 60 unique customers (40 from web_forms + 20 from legacy_db, minus any email-level overlap). If your agent over-deduplicates (removing real rows) or under-deduplicates (keeping both copies of the same customer), the grader’s row count penalty activates.`]}),_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`evaluate_dataframes()`}),` grader still joins on `,_jsx(_components.code,{children:`customer_id`}),`, which means both sources’ records must survive the merge with their original `,_jsx(_components.code,{children:`customer_id`}),` values intact — your agent cannot drop `,_jsx(_components.code,{children:`customer_id`}),` or change it during the merge.`]}),_jsx(Heading,{level:`3`,id:`recommended-action-sequence-2`,children:`Recommended Action Sequence`}),_jsxs(Steps,{children:[_jsxs(Step,{title:`Profile both sources`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"PROFILE_SOURCE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"PROFILE_SOURCE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"legacy_db\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Each profile earns `,_jsx(_components.code,{children:`+0.02`}),`. Examine the null counts for both sources before standardising.`]})]}),_jsx(Step,{title:`Standardize web_forms columns`,children:_jsxs(_components.p,{children:[`Apply the same four-step standardization from Task 1 to `,_jsx(_components.code,{children:`web_forms`}),`: email `,_jsx(_components.code,{children:`LOWERCASE_STRIP`}),`, name `,_jsx(_components.code,{children:`LOWERCASE_STRIP`}),`, phone `,_jsx(_components.code,{children:`EXTRACT_NUMBERS`}),`, signup_date `,_jsx(_components.code,{children:`TO_DATETIME_ISO`}),`.`]})}),_jsx(Step,{title:`Filter web_forms bots via SQL`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXECUTE_SQL\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"query\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SELECT * FROM web_forms WHERE customer_id NOT LIKE 'BOT%' AND email IS NOT NULL\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"output_table\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_forms_clean\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})})}),_jsx(Step,{title:`Standardize legacy_db columns`,children:_jsxs(_components.p,{children:[`Apply the same four-step standardization to `,_jsx(_components.code,{children:`legacy_db`}),`: email `,_jsx(_components.code,{children:`LOWERCASE_STRIP`}),`, name `,_jsx(_components.code,{children:`LOWERCASE_STRIP`}),`, phone `,_jsx(_components.code,{children:`EXTRACT_NUMBERS`}),`, signup_date `,_jsx(_components.code,{children:`TO_DATETIME_ISO`}),`.`]})}),_jsx(Step,{title:`Filter legacy_db bots via SQL`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXECUTE_SQL\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"query\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SELECT * FROM legacy_db WHERE customer_id NOT LIKE 'BOT%' AND email IS NOT NULL\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"output_table\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"legacy_db_clean\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})})}),_jsxs(Step,{title:`UNION both sources`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXECUTE_SQL\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"query\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SELECT * FROM web_forms_clean UNION ALL SELECT * FROM legacy_db_clean\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"output_table\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"merged_output\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[_jsx(_components.code,{children:`UNION ALL`}),` preserves all rows including cross-source duplicates. You’ll deduplicate in the next step.`]})]}),_jsxs(Step,{title:`Deduplicate the merged table by email`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"DEDUPLICATE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"merged_output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"deduplication_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXACT_EMAIL\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`This calls `,_jsx(_components.code,{children:`drop_duplicates(subset=[\"email\"])`}),` on the combined table, keeping one row per unique email address.`]})]}),_jsx(Step,{title:`Submit`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SUBMIT_PIPELINE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"final_source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"merged_output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})})})]}),_jsx(Tip,{children:_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`UNION ALL`}),` + `,_jsx(_components.code,{children:`DEDUPLICATE`}),` pattern is more reliable than `,_jsx(_components.code,{children:`MERGE_SOURCES`}),` for Task 2 because `,_jsx(_components.code,{children:`DEDUPLICATE`}),` uses Pandas `,_jsx(_components.code,{children:`drop_duplicates()`}),` which handles `,_jsx(_components.code,{children:`None`}),` emails gracefully (they are not treated as matching duplicates), whereas the `,_jsx(_components.code,{children:`MERGE_SOURCES`}),` outer join can produce Cartesian-product issues when the join key has many nulls.`]})})]}),_jsxs(Tab,{title:`Task 3 — Hard`,id:`task-3-%E2%80%94-hard`,\"data-child-heading-ids\":`[\"task-3%3A-3-way-enterprise-merge\",\"source-schemas-and-generation-parameters\",\"conflict-resolution-rules\",\"bot-injection-details\",\"ground-truth-target-schema-3\",\"execute_sql-strategy-for-the-3-way-join\",\"recommended-action-sequence-3\"]`,children:[_jsx(Heading,{level:`2`,id:`task-3-3-way-enterprise-merge`,children:`Task 3: 3-Way Enterprise Merge`}),_jsxs(_components.p,{children:[`Task 3 is the full-complexity benchmark. Your agent must unify three sources — `,_jsx(_components.code,{children:`salesforce`}),`, `,_jsx(_components.code,{children:`web_leads`}),`, and `,_jsx(_components.code,{children:`legacy_db`}),` — each with a `,_jsx(_components.strong,{children:`different column schema`}),`, `,_jsx(_components.strong,{children:`independently injected bots`}),`, and `,_jsx(_components.strong,{children:`explicit conflict resolution rules`}),` that dictate which source wins when the same customer has different values across sources.`]}),_jsx(Heading,{level:`3`,id:`source-schemas-and-generation-parameters`,children:`Source Schemas and Generation Parameters`}),_jsx(_components.p,{children:`The three sources are generated from the same 60-customer base truth, then individually subsampled, corrupted, and augmented with bots:`}),_jsxs(_components.p,{children:[_jsx(_components.strong,{children:_jsx(_components.code,{children:`salesforce`})}),` — 90% sample (~54 rows) + 5 bots`]}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Column`}),_jsx(_components.th,{children:`Notes`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`CUST_{n}`}),` format — canonical identifier`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`email`})}),_jsx(_components.td,{children:`Lowercase — this is the authoritative email`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`phone`})}),_jsxs(_components.td,{children:[`Present ~70% of the time (`,_jsx(_components.code,{children:`None`}),` otherwise)`]})]})]})]}),_jsxs(_components.p,{children:[_jsx(_components.strong,{children:_jsx(_components.code,{children:`web_leads`})}),` — 80% sample (~48 rows) + 5 bots`]}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Column`}),_jsx(_components.th,{children:`Notes`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})}),_jsx(_components.td,{children:`Nullable ~20% of the time — some web sign-ups don’t yet have a CRM ID`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`email`})}),_jsx(_components.td,{children:`Lowercase — matches Salesforce email`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`phone`})}),_jsxs(_components.td,{children:[_jsx(_components.strong,{children:`Fresh phone number`}),` (independently generated via `,_jsx(_components.code,{children:`fake.msisdn()`}),`), present ~60% of the time`]})]})]})]}),_jsxs(_components.p,{children:[_jsx(_components.strong,{children:_jsx(_components.code,{children:`legacy_db`})}),` — 85% sample (~51 rows) + 5 bots`]}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Column`}),_jsx(_components.th,{children:`Notes`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`legacy_id`})}),_jsxs(_components.td,{children:[`Uses `,_jsx(_components.code,{children:`OLD-{n}`}),` prefix instead of `,_jsx(_components.code,{children:`CUST_{n}`}),` (e.g., `,_jsx(_components.code,{children:`OLD-1000`}),` maps to `,_jsx(_components.code,{children:`CUST_1000`}),`)`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`contact_email`})}),_jsxs(_components.td,{children:[_jsx(_components.strong,{children:`UPPERCASE`}),` — must be lowercased before any join`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`home_phone`})}),_jsx(_components.td,{children:`Original phone from base truth — older than web_leads`})]})]})]}),_jsx(Heading,{level:`3`,id:`conflict-resolution-rules`,children:`Conflict Resolution Rules`}),_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`CONFLICT_RULES`}),` dictionary from `,_jsx(_components.code,{children:`server/data_generator.py`}),` is exposed to your agent in every observation as `,_jsx(_components.code,{children:`conflict_rules`}),`:`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`5`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#D4D4D4`},children:`CONFLICT_RULES`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"prefer_salesforce\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# SF email is canonical`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"prefer_web_leads\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Web leads has fresher phone data`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"prefer_salesforce\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsx(_components.p,{children:`The hidden truth is built by applying these rules:`}),_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`7`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` r `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` truth:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` resolved_phone `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` wl_phones.get(r[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`], r[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`]) `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# WL > SF`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` resolved_truth.append({`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: r[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`], `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# prefer_salesforce`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: r[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`], `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# prefer_salesforce`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: resolved_phone, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# prefer_web_leads`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` })`})}),`\n`]})})}),_jsxs(_components.p,{children:[_jsx(_components.code,{children:`wl_phones`}),` maps every `,_jsx(_components.code,{children:`customer_id`}),` to the fresh phone generated for the `,_jsx(_components.code,{children:`web_leads`}),` source. If web_leads had a phone for that customer, the truth phone is the web_leads phone. If not, it falls back to the Salesforce phone.`]}),_jsx(Heading,{level:`3`,id:`bot-injection-details`,children:`Bot Injection Details`}),_jsx(_components.p,{children:`Bots are injected into all three sources. Each source gets 5 bot rows:`}),_jsx(CodeBlock,{filename:``,numberOfLines:`8`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`8`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Salesforce bots`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"BOT{5-digit}\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"user42@promo-alerts.net\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"+00000000000\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Web Leads bots`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"customer_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`None`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"user99@click-rewards.xyz\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`None`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:`# Legacy DB bots`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"legacy_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"BOT{5-digit}\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"contact_email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"BULK@FREEWINNERS.INFO\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"home_phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"+00000000000\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Bot domains across all sources: `,_jsx(_components.code,{children:`promo-alerts.net`}),`, `,_jsx(_components.code,{children:`click-rewards.xyz`}),`, `,_jsx(_components.code,{children:`freewinners.info`}),`, `,_jsx(_components.code,{children:`bulk-mail.co`}),`.`]}),_jsxs(_components.p,{children:[`Filtering bots requires checking both the `,_jsx(_components.code,{children:`customer_id`}),` / `,_jsx(_components.code,{children:`legacy_id`}),` prefix (`,_jsx(_components.code,{children:`BOT`}),`) and the email domain. Note that legacy_db bot emails are uppercase (`,_jsx(_components.code,{children:`BULK@FREEWINNERS.INFO`}),`) — standardise `,_jsx(_components.code,{children:`contact_email`}),` before filtering.`]}),_jsx(Heading,{level:`3`,id:`ground-truth-target-schema-3`,children:`Ground Truth Target Schema`}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Column`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Format Requirement`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`customer_id`})}),_jsx(_components.td,{children:`string`}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`CUST_{n}`}),` format (from Salesforce, not `,_jsx(_components.code,{children:`OLD-{n}`}),`)`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`email`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`Lowercase (from Salesforce — canonical)`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`phone`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`E.164 (from Web Leads when available, else Salesforce)`})]})]})]}),_jsxs(_components.p,{children:[`There are only `,_jsx(_components.strong,{children:`3 columns`}),` in the Task 3 truth. Your final `,_jsx(_components.code,{children:`merged_output`}),` must have exactly these three columns.`]}),_jsx(Heading,{level:`3`,id:`execute_sql-strategy-for-the-3-way-join`,children:`EXECUTE_SQL Strategy for the 3-Way JOIN`}),_jsx(Warning,{children:_jsxs(_components.p,{children:[`Calling `,_jsx(_components.code,{children:`SUBMIT_PIPELINE`}),` before 3 steps in Task 3 incurs `,_jsx(_components.strong,{children:`-0.15`}),` and the episode continues. Task 3 requires the most preparation of any task — budget at least 7-8 actions for a complete pipeline.`]})}),_jsxs(_components.p,{children:[`The recommended approach uses a single `,_jsx(_components.code,{children:`EXECUTE_SQL`}),` action with a `,_jsx(_components.code,{children:`GROUP BY email`}),` aggregation over a `,_jsx(_components.code,{children:`UNION ALL`}),` of all three standardised sources:`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`37`,language:`sql`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`sql`,children:_jsxs(_components.code,{language:`sql`,numberOfLines:`37`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`SELECT`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` MAX`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(customer_id) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`AS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` customer_id,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email,`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` MAX`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(phone) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`AS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` phone`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`FROM`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` -- Salesforce: canonical email, customer_id, and SF phone`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` SELECT`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` customer_id, email, phone`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` FROM`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` salesforce`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` WHERE`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` customer_id `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` 'BOT%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%promo-alerts%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%click-rewards%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%freewinners%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%bulk-mail%'`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` UNION ALL`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` -- Web Leads: fresh phone, filter bot domains and null customer_id`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` SELECT`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` customer_id, email, phone`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` FROM`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` web_leads`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` WHERE`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` customer_id `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`IS NOT NULL`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%promo-alerts%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%click-rewards%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%freewinners%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%bulk-mail%'`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` UNION ALL`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#6A9955`},children:` -- Legacy DB: remap legacy_id → customer_id, lowercase email already applied`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` SELECT`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` REPLACE`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(legacy_id, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'OLD-'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`'CUST_'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`AS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` customer_id,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` contact_email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`AS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` home_phone `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`AS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` phone`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` FROM`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` legacy_db`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` WHERE`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` legacy_id `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` 'BOT%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` contact_email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%PROMO-ALERTS%'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` AND`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` contact_email `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`NOT`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:` LIKE`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '%BULK-MAIL%'`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`) sub`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#569CD6`},children:`GROUP BY`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` email`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`GROUP BY email`}),` with `,_jsx(_components.code,{children:`MAX(customer_id)`}),` implements the `,_jsx(_components.code,{children:`prefer_salesforce`}),` rule — because `,_jsx(_components.code,{children:`CUST_`}),` sorts lexicographically after `,_jsx(_components.code,{children:`OLD-`}),`, `,_jsx(_components.code,{children:`MAX()`}),` picks the Salesforce-format ID. The `,_jsx(_components.code,{children:`MAX(phone)`}),` implements a loose approximation of `,_jsx(_components.code,{children:`prefer_web_leads`}),` (fresh phones generated by Faker are not necessarily lexicographically larger, so this is imperfect — the deterministic `,_jsx(_components.code,{children:`RuleBasedBaseline`}),` in `,_jsx(_components.code,{children:`inference.py`}),` achieves perfection through the specific SQL shown there).`]}),_jsx(Note,{children:_jsxs(_components.p,{children:[`Remember to standardise `,_jsx(_components.code,{children:`legacy_db.contact_email`}),` to lowercase with `,_jsx(_components.code,{children:`STANDARDIZE_COLUMN`}),` `,_jsx(_components.strong,{children:`before`}),` executing the SQL join. The SQL `,_jsx(_components.code,{children:`LIKE`}),` patterns above are case-sensitive in SQLite by default for ASCII characters, so an uppercase `,_jsx(_components.code,{children:`BULK@FREEWINNERS.INFO`}),` domain will not be caught by a lowercase `,_jsx(_components.code,{children:`LIKE '%bulk-mail%'`}),` filter unless the column has already been lowercased.`]})}),_jsx(Heading,{level:`3`,id:`recommended-action-sequence-3`,children:`Recommended Action Sequence`}),_jsxs(Steps,{children:[_jsxs(Step,{title:`Profile all three sources`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`3`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`3`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"PROFILE_SOURCE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"salesforce\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"PROFILE_SOURCE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_leads\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"PROFILE_SOURCE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"legacy_db\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Profile each source first to understand null distributions. Each earns `,_jsx(_components.code,{children:`+0.02`}),`.`]})]}),_jsx(Step,{title:`Standardize Salesforce columns`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"salesforce\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"LOWERCASE_STRIP\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"salesforce\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXTRACT_NUMBERS\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})})}),_jsx(Step,{title:`Standardize Web Leads columns`,children:_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_leads\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"LOWERCASE_STRIP\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"web_leads\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXTRACT_NUMBERS\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})})}),_jsxs(Step,{title:`Standardize Legacy DB columns — critical step`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"legacy_db\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"contact_email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"LOWERCASE_STRIP\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"STANDARDIZE_COLUMN\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"legacy_db\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"column\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"home_phone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"standardization_strategy\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXTRACT_NUMBERS\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`contact_email`}),` column is UPPERCASE by design. Failing to lowercase it before the SQL join will cause the `,_jsx(_components.code,{children:`GROUP BY email`}),` to treat `,_jsx(_components.code,{children:`ALICE@EXAMPLE.COM`}),` and `,_jsx(_components.code,{children:`alice@example.com`}),` as different customers, exploding the row count.`]})]}),_jsxs(Step,{title:`Execute the 3-way UNION + GROUP BY SQL merge`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"EXECUTE_SQL\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"query\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SELECT MAX(customer_id) AS customer_id, email, MAX(phone) AS phone FROM (SELECT customer_id, email, phone FROM salesforce WHERE customer_id NOT LIKE 'BOT%' AND email NOT LIKE '%promo-alerts%' UNION ALL SELECT customer_id, email, phone FROM web_leads WHERE customer_id IS NOT NULL AND email NOT LIKE '%promo-alerts%' UNION ALL SELECT REPLACE(legacy_id, 'OLD-', 'CUST_') AS customer_id, contact_email AS email, home_phone AS phone FROM legacy_db WHERE legacy_id NOT LIKE 'BOT%') GROUP BY email\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"output_table\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"merged_output\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsxs(_components.p,{children:[`On success, earns `,_jsx(_components.code,{children:`+0.10`}),`. The `,_jsx(_components.code,{children:`merged_output`}),` table now contains one row per unique email address with the conflict-resolved `,_jsx(_components.code,{children:`customer_id`}),` and `,_jsx(_components.code,{children:`phone`}),`.`]})]}),_jsxs(Step,{title:`Submit the pipeline`,children:[_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"action_type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"SUBMIT_PIPELINE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:`\"final_source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"merged_output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`The grader will compare `,_jsx(_components.code,{children:`merged_output`}),` against the hidden truth built from `,_jsx(_components.code,{children:`_apply_conflict_rules()`}),` in `,_jsx(_components.code,{children:`data_generator.py`}),`.`]})]})]}),_jsx(Tip,{children:_jsxs(_components.p,{children:[`The `,_jsx(_components.code,{children:`inference.py`}),` `,_jsx(_components.code,{children:`RuleBasedBaseline`}),` for Task 3 achieves a perfect score on every episode. If your LLM agent is underperforming on Task 3, compare its SQL output row count against `,_jsx(_components.code,{children:`len(truth_df)`}),` using `,_jsx(_components.code,{children:`PROFILE_SOURCE`}),` on `,_jsx(_components.code,{children:`merged_output`}),` after the SQL step. Excess rows always point to bot rows that slipped through your WHERE filters.`]})})]})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Task Catalogue: Three Difficulty Tiers of CRM Cleaning","sidebarTitle":"Task Catalogue","description":"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.","href":"/task-catalogue"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Task Comparison at a Glance","slug":"task-comparison-at-a-glance","depth":2,"children":[]},{"title":"Task 1: Web Forms Normalization","slug":"task-1-web-forms-normalization","depth":2,"children":[{"title":"Input Data Characteristics","slug":"input-data-characteristics","depth":3,"children":[],"tabId":"task-1-%E2%80%94-easy"},{"title":"Bot Row Schema","slug":"bot-row-schema","depth":3,"children":[],"tabId":"task-1-%E2%80%94-easy"},{"title":"Ground Truth Target Schema","slug":"ground-truth-target-schema","depth":3,"children":[],"tabId":"task-1-%E2%80%94-easy"},{"title":"Scoring","slug":"scoring","depth":3,"children":[],"tabId":"task-1-%E2%80%94-easy"},{"title":"Recommended Action Sequence","slug":"recommended-action-sequence","depth":3,"children":[],"tabId":"task-1-%E2%80%94-easy"}],"tabId":"task-1-%E2%80%94-easy"},{"title":"Task 2: Legacy DB Deduplication","slug":"task-2-legacy-db-deduplication","depth":2,"children":[{"title":"Data Sources","slug":"data-sources","depth":3,"children":[],"tabId":"task-2-%E2%80%94-medium"},{"title":"Ground Truth Construction","slug":"ground-truth-construction","depth":3,"children":[],"tabId":"task-2-%E2%80%94-medium"},{"title":"Ground Truth Target Schema","slug":"ground-truth-target-schema-2","depth":3,"children":[],"tabId":"task-2-%E2%80%94-medium"},{"title":"What Makes This Harder","slug":"what-makes-this-harder","depth":3,"children":[],"tabId":"task-2-%E2%80%94-medium"},{"title":"Recommended Action Sequence","slug":"recommended-action-sequence-2","depth":3,"children":[],"tabId":"task-2-%E2%80%94-medium"}],"tabId":"task-2-%E2%80%94-medium"},{"title":"Task 3: 3-Way Enterprise Merge","slug":"task-3-3-way-enterprise-merge","depth":2,"children":[{"title":"Source Schemas and Generation Parameters","slug":"source-schemas-and-generation-parameters","depth":3,"children":[],"tabId":"task-3-%E2%80%94-hard"},{"title":"Conflict Resolution Rules","slug":"conflict-resolution-rules","depth":3,"children":[],"tabId":"task-3-%E2%80%94-hard"},{"title":"Bot Injection Details","slug":"bot-injection-details","depth":3,"children":[],"tabId":"task-3-%E2%80%94-hard"},{"title":"Ground Truth Target Schema","slug":"ground-truth-target-schema-3","depth":3,"children":[],"tabId":"task-3-%E2%80%94-hard"},{"title":"EXECUTE_SQL Strategy for the 3-Way JOIN","slug":"execute_sql-strategy-for-the-3-way-join","depth":3,"children":[],"tabId":"task-3-%E2%80%94-hard"},{"title":"Recommended Action Sequence","slug":"recommended-action-sequence-3","depth":3,"children":[],"tabId":"task-3-%E2%80%94-hard"}],"tabId":"task-3-%E2%80%94-hard"}],"codeExamples":{}},"jsLinks":[],"subdomain":"ritika-fa61c43f","siteName":"CRM Data Pipeline","title":"Task Catalogue: Three Difficulty Tiers of CRM Cleaning - CRM Data Pipeline","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-07-28T13:56:04.919Z","originalFileLocation":"task-catalogue.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.\">\u003cmeta data-mint-head name=\"application-name\" content=\"CRM Data Pipeline\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/jpeg\" sizes=\"180x180\" href=\"https://media.brand.dev/87b75398-73c9-4b33-bea3-6d01873af484.jpg\">\u003clink data-mint-head rel=\"icon\" type=\"image/jpeg\" sizes=\"32x32\" href=\"https://media.brand.dev/87b75398-73c9-4b33-bea3-6d01873af484.jpg\">\u003clink data-mint-head rel=\"icon\" type=\"image/jpeg\" sizes=\"16x16\" href=\"https://media.brand.dev/87b75398-73c9-4b33-bea3-6d01873af484.jpg\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/jpeg\" href=\"https://media.brand.dev/87b75398-73c9-4b33-bea3-6d01873af484.jpg\">\u003cmeta data-mint-head property=\"og:title\" content=\"Task Catalogue: Three Difficulty Tiers of CRM Cleaning - CRM Data Pipeline\">\u003cmeta data-mint-head property=\"og:description\" content=\"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"CRM Data Pipeline\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://ritika-fa61c43f.mintlify.app/task-catalogue\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://ritika-fa61c43f.mintlify.site/_mintlify/api/og/v2?division=Overview\u0026amp;title=Task+Catalogue%3A+Three+Difficulty+Tiers+of+CRM+Cleaning\u0026amp;description=Detailed+breakdown+of+all+three+benchmark+tasks+%E2%80%94+Easy%2C+Medium%2C+and+Hard+%E2%80%94+with+data+schemas%2C+objectives%2C+and+scoring+criteria.\u0026amp;theme=3c8700722347fb26282ef816\u0026amp;signature=3d0e4efb17b2a87e7d21dc4ba32985dea634e35909317f53b02e36c0f4e99156\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Task Catalogue: Three Difficulty Tiers of CRM Cleaning - CRM Data Pipeline\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://ritika-fa61c43f.mintlify.site/_mintlify/api/og/v2?division=Overview\u0026amp;title=Task+Catalogue%3A+Three+Difficulty+Tiers+of+CRM+Cleaning\u0026amp;description=Detailed+breakdown+of+all+three+benchmark+tasks+%E2%80%94+Easy%2C+Medium%2C+and+Hard+%E2%80%94+with+data+schemas%2C+objectives%2C+and+scoring+criteria.\u0026amp;theme=3c8700722347fb26282ef816\u0026amp;signature=3d0e4efb17b2a87e7d21dc4ba32985dea634e35909317f53b02e36c0f4e99156\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"https://mintlify.s3.us-west-1.amazonaws.com/ritika-fa61c43f/_generated/favicon/browserconfig.xml?v=3\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"CRM Data Pipeline\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#f8c61e\">\u003clink data-mint-head rel=\"canonical\" href=\"https://ritika-fa61c43f.mintlify.app/task-catalogue\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/task-catalogue.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://ritika-fa61c43f.mintlify.app/#organization\",\"name\":\"CRM Data Pipeline\",\"url\":\"https://ritika-fa61c43f.mintlify.app\"},{\"@type\":\"WebSite\",\"@id\":\"https://ritika-fa61c43f.mintlify.app#website\",\"name\":\"CRM Data Pipeline\",\"url\":\"https://ritika-fa61c43f.mintlify.app\",\"publisher\":{\"@id\":\"https://ritika-fa61c43f.mintlify.app/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue#webpage\",\"url\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue\",\"name\":\"Task Catalogue: Three Difficulty Tiers of CRM Cleaning\",\"description\":\"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.\",\"dateModified\":\"2026-07-28T13:56:04.919Z\",\"isPartOf\":{\"@id\":\"https://ritika-fa61c43f.mintlify.app#website\"},\"breadcrumb\":{\"@id\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Overview\",\"item\":\"https://ritika-fa61c43f.mintlify.app/introduction\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Task Catalogue: Three Difficulty Tiers of CRM Cleaning\",\"item\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue#article\",\"headline\":\"Task Catalogue: Three Difficulty Tiers of CRM Cleaning\",\"name\":\"Task Catalogue: Three Difficulty Tiers of CRM Cleaning\",\"description\":\"Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.\",\"url\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue\",\"mainEntityOfPage\":{\"@id\":\"https://ritika-fa61c43f.mintlify.app/task-catalogue#webpage\"},\"dateModified\":\"2026-07-28T13:56:04.919Z\",\"publisher\":{\"@id\":\"https://ritika-fa61c43f.mintlify.app/#organization\"},\"isPartOf\":{\"@id\":\"https://ritika-fa61c43f.mintlify.app#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"CRM Data Pipeline\",\"creator\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","siteName":"CRM Data Pipeline","nav":{"global":null,"products":[{"product":"Untitled Product","tabs":[{"tab":"Documentation","groups":[{"group":"Overview","pages":[]},{"group":"Getting Started","pages":[]},{"group":"Core Concepts","pages":[]},{"group":"Data Engine","pages":[]},{"group":"Guides","pages":[]}]},{"tab":"Documentation","groups":[{"group":"Overview","pages":[{"title":"CRM Data Pipeline: OpenEnv Benchmark for AI Agents","sidebarTitle":"Introduction","href":"/introduction"},{"title":"Environment Architecture and Data Flow Explained","sidebarTitle":"Architecture","description":"A deep-dive into the CRM Pipeline's POMDP model, server components, file interconnections, reward signals, and full episode lifecycle.","href":"/architecture"},{"title":"Task Catalogue: Three Difficulty Tiers of CRM Cleaning","sidebarTitle":"Task Catalogue","href":"/task-catalogue"}]},{"group":"Getting Started","pages":[{"title":"CRM Data Pipeline Environment: Get Started in 5 Minutes","sidebarTitle":"Quick Start","description":"Spin up the CRM Data Pipeline environment server and run your first agent episode loop with real data cleaning actions in under 5 minutes.","href":"/quickstart"},{"title":"CRM Data Pipeline: Local and Docker Installation Guide","sidebarTitle":"Installation","href":"/installation"},{"title":"CRM Data Pipeline Environment Configuration Reference","sidebarTitle":"Configuration","href":"/configuration"}]},{"group":"Core Concepts","pages":[{"title":"POMDP Model: How the Environment Simulates Data Engineering","sidebarTitle":"POMDP Model","href":"/concepts/pomdp-model"},{"title":"Action Space: All 8 Pipeline Operations Explained","sidebarTitle":"Action Space","href":"/concepts/action-space"},{"title":"Observation Space: What Your Agent Sees After Each Step","sidebarTitle":"Observation Space","href":"/concepts/observation-space"},{"title":"Reward System: Dense Heuristic Reward Shaping Explained","sidebarTitle":"Reward System","href":"/concepts/reward-system"}]},{"group":"Data Engine","pages":[{"title":"CRM Data Sources: Schemas, Formats, and Corruption Patterns","sidebarTitle":"Data Sources","href":"/data-engine/data-sources"},{"title":"Data Generator: How Messy CRM Data Is Procedurally Created","sidebarTitle":"Data Generator","href":"/data-engine/data-generator"},{"title":"Standardization Strategies for CRM Column Cleaning","sidebarTitle":"Standardization","href":"/data-engine/standardization-strategies"},{"title":"CRM Deduplication Strategies: Exact and Fuzzy Matching","sidebarTitle":"Deduplication","href":"/data-engine/deduplication"},{"title":"SQL Execution: Querying CRM DataFrames with EXECUTE_SQL","sidebarTitle":"SQL Execution","href":"/data-engine/sql-execution"}]},{"group":"Guides","pages":[{"title":"Running the Baseline Inference Script on CRM Tasks","sidebarTitle":"Run Inference","href":"/guides/running-inference"},{"title":"Write a Custom Agent for the CRM Data Pipeline Env","sidebarTitle":"Write an Agent","href":"/guides/writing-an-agent"},{"title":"Understanding Grading and Scoring in the CRM Pipeline","sidebarTitle":"Grading \u0026 Scoring","href":"/guides/grading-scoring"},{"title":"Validate Your Submission Before Submitting to OpenEnv","sidebarTitle":"Submission Validation","href":"/guides/submission-validation"}]}]},{"tab":"API Reference","groups":[{"group":"REST Endpoints","pages":[]},{"group":"Data Models","pages":[]}]},{"tab":"API Reference","pages":[{"title":"API Reference","href":"/api/overview"}]},{"tab":"Untitled Tab"},{"tab":"Untitled Tab"}]},{"product":"RealTutor ai"}]},"basePath":"","title":"Task Catalogue: Three Difficulty Tiers of CRM Cleaning - CRM Data Pipeline","buildId":"dpl_42LwxxU1kGN114H4XzwGYXa9ZDah","deploymentId":"dpl_42LwxxU1kGN114H4XzwGYXa9ZDah","dynamics":[],"chunks":[]}