Engine selection¶

Status: Available in ETLantic 0.21.0.

Start with core local Python. Add one engine at a time.

Choose a path¶

Goal	Install	Profile hint	Guide
Learn the model in memory	`etlantic==0.21.0`	`development`	Quickstart
JSON / CSV files	core only	file storage bindings	File storage
Fast local dataframes	`etlantic[polars]==0.21.0`	`dataframe_engine="polars"`	Polars tutorial
Pandas compatibility	`etlantic[pandas]==0.21.0`	`dataframe_engine="pandas"`	Pandas tutorial
Cross-engine Polars↔Pandas	`etlantic[dataframes]==0.21.0`	both plugins allowlisted	Interchange example
Keep work in SQL	`etlantic[sql]==0.21.0`	`sql_engine="sql"`	SQL tutorial
Local Spark batch	`etlantic[pyspark]==0.21.0`	`spark_engine="pyspark"`	PySpark tutorial
Emit Airflow DAGs	`etlantic[airflow]==0.21.0`	`orchestrator="airflow"`	Airflow tutorial
Prefect local scheduler	`etlantic[prefect]==0.21.0`	`orchestrator="prefect"`	Prefect example
Portable transforms (no native impl)	matching engine plugin	`portable_transform_policy="require"`	Portable transforms

One engine first. Do not combine SQL + Spark + dataframes until a single engine path works under validate and plan.
Pin the minor in 0.x. Keep core and every official plugin on the same release (for example etlantic==0.21.0 with etlantic-polars==0.21.0). Do not mix 0.20 plugins with 0.21 core.
Production profiles need allowlists. Create profiles/prod.json from the embedded JSON in Capabilities → CI starter, or from prod.example.json in a checkout. Trim the allowlist to engines you install (not installed with the PyPI wheel).
Airflow is compile-only. etlantic-airflow writes DAG artifacts; install Apache Airflow separately where DAGs load.
Memory demos need Python seeding. CLI run does not share process-local memory from a previous Python session.