Getting Started¶

Welcome to ETLantic!

ETLantic catches incompatible data-pipeline wiring before you process data. Define typed datasets, transformations, and pipelines in Python; validate and plan them once; run locally or through optional engine plugins.

Project status: ETLantic 0.21.0 is stable within the documented single-tenant reference deployment boundary. Experimental features and broader deployment models remain outside that claim. See Capabilities for the shipped boundary and Evaluator brief for decision-makers. How to read status labels: Documentation Status.

Five-minute path¶

Installation — pip install etlantic==0.21.0
Quickstart — etlantic init, validate, plan, and run
First Pipeline — evolve the generated project
Engine selection — then an engine tutorial

CLI run vs in-memory demos

The Quickstart binds assets to JSON files, so etlantic run works without seeding. In-memory demos (PipelineRuntime.memory.seed) only share data inside one Python process—use examples/memory_customers.py from a checkout for that path. Prefer the same --profile for validate, plan, and run (development by default when omitted).

What You'll Learn¶

Install ETLantic from PyPI
Define typed data contracts and transformations
Wire a pipeline and validate it before execution
Run locally with durable JSON assets via etlantic init
Use the CLI for inspect / validate / plan / run
Tell shipped APIs from future design

Prerequisites¶

Python 3.11+
Basic type annotations
Familiarity with ETL concepts helps; orchestration experience is optional

Documentation Roadmap¶

Installation
Quickstart
Your First Pipeline
Engine selection, then Polars, Pandas, SQL, or PySpark
Diligence: Capabilities, Evaluator Brief, Compare
FAQ / Troubleshooting / Cookbook / Best practices
Project Structure (after a second pipeline)
Upgrade when moving between 0.x releases

The ETLantic Mental Model¶

Typed Python classes
      │
      ▼
Validation (catch bad wiring)
      │
      ▼
PipelinePlan (secret-free, deterministic)
      │
      ▼
Run locally  |  Compile (Airflow)  |  Generate contracts

ETLantic 0.21.0 can execute registered Python implementations with its local runtime and optional Polars/Pandas/SQL/PySpark plugins, compile plans to Airflow DAGs via etlantic-airflow, execute plans through the Prefect local MVP, and compile supported portable transformation families without native engine implementations.

Next Step¶

Continue with Installation, then Quickstart.