Agent 31 · Data Automation
DataIngest Agent
Ingests new data batches and heals schema drift.
ServiceInsightLangGraph
Found2
Fixed + verified2
Held for approval0
Recalln/a
One offline run on demo data with planted problems (recorded platform test, November 2026).
How DataIngest heals data, step by step
INData inNYC TLC yellow-taxi trip records
01Detect2 problems found
02Diagnose2 problem types named
03Plan2 safe to fix · 0 need a person
04Fix + verify2 fixed and re-checked
05ReportPlain-English report and Control Tower record
- Client problem
- Pipelines break when a source renames a column or stops sending.
- What it does
- Watches landing batches for drift and staleness.
- How it heals
- Maps renamed columns and ingests; alerts on stale feeds.
- Human gate
- Unknown new columns are logged for review.
- Typical buyer
- Mid-market data teams
- Built with
- LangGraph self-healing loop
- Demo data
- NYC TLC yellow-taxi trip records (public Parquet download) or synthetic.
- Tables
- Landing batches and lakehouse tables built by other agents
- Target outcome
- Target: zero manual fixes for known drift patterns.
- Path to production
- Service (Insight).
Issues detected on the demo data
| Issue | Risk | Proposed fix | Count |
|---|---|---|---|
| schema_drift Missing contract columns ['tpep_pickup_datetime', 'fare_amount']; extra ['fare', 'pickup_datetime'] | low | Map renamed columns back to the contract | 1 |
| stale_batch Latest trip 2026-08-22 is 39 days behind | medium | Quarantine batch and alert the source owner | 1 |
How we demo it
- Drop a batch with renamed columns.
- Run DataIngest.
- Show the ingest log.
Built on AWS
Runs in the platform container on EC2 Graviton (or on demand in Lambda), writes healed tables to the S3 lakehouse, uses Amazon Bedrock only to write explanations, and reports six metrics to CloudWatch.
Step into DigiQuench.
Agents that heal your data.
Watch an agent find planted problems, fix the safe ones and wait for your approval on the rest. Thirty minutes, live.