Examples¶

Green path¶

Installation — pip install etlantic==0.21.0
Quickstart
First Pipeline
Engine selection
Runnable scripts below

Pages under Design Studies are aspirational stubs—not installable tutorials.

Runnable guides (docs)¶

Sample multi-file project — examples/sample_project/
File-backed pipeline — JSON and CSV
Ops examples — secrets, schema, SARIF
Polars
Pandas
SQL
PySpark
Airflow
Prefect direct execution — examples/prefect_run.py
Airflow Compile — examples/airflow_compile.py
Portable transforms — examples/portable_polars_kernel.py, portable_pandas_kernel.py, and portable_wave17.py
Polars ↔ Pandas interchange — examples/interchange_polars_pandas.py
SparkForge Adapter — tests/sparkforge/

Runnable scripts (repository `examples/`)¶

Clone required

examples/ is not installed with the PyPI wheel. Commands below need a git checkout (uv run …). Pip-only users: paste the Quickstart or open scripts on GitHub.

Scripts marked (CI) run in .github/workflows/checks.yml. Others are documented and copy-paste runnable locally. Repository index: examples/README.md on GitHub.

In-memory quickstart (CI)¶

uv run python examples/memory_customers.py

Portable kernels and 0.17 families (docs / local)¶

uv sync --group dataframes
uv run python examples/portable_polars_kernel.py
uv run python examples/portable_pandas_kernel.py
uv run python examples/portable_wave17.py

Polars ↔ Pandas interchange (CI)¶

uv sync --group dataframes
uv run python examples/interchange_polars_pandas.py

See Polars ↔ Pandas Interchange.

JSON and CSV storage (docs / local)¶

uv run python examples/file_storage.py

Dataframe parity (Polars / Pandas) (CI)¶

uv sync --group dataframes
uv run python examples/dataframe_parity.py polars
uv run python examples/dataframe_parity.py pandas

SQL to SQL (CI)¶

uv sync --group sql
uv run python examples/sql_to_sql.py
uv run python examples/sql_boundary_hybrid.py
uv run python examples/sql_transactional_write.py
uv run python examples/sql_failure_recovery.py

Defaults to in-memory SQLite for demos; set ETLANTIC_SQL_URL for PostgreSQL.

Local PySpark (CI)¶

uv sync --group pyspark
uv run python examples/pyspark_local.py

Airflow compile (CI)¶

uv sync --group airflow
uv run python examples/airflow_compile.py

Prefect local execution (CI)¶

uv sync --group prefect
uv run python examples/prefect_run.py

Design studies (aspirational)¶

The remaining pages in this section explore intended integrations. Each page opens with a Future design warning. They may contain APIs, packages, or commands that go beyond the shipped surface — prefer the runnable guides and examples/ for installable behavior.

Topic	Prefer instead
CSV and JSON through built-in storage	`examples/file_storage.py`
Pandas and Polars pipelines	`examples/dataframe_parity.py`
SQL execution and pushdown	`examples/sql_*.py`
PySpark batch	`examples/pyspark_local.py`
Airflow compilation	Airflow Compile
Graphviz DOT / HTML lineage	`etlantic.viz` / `etlantic viz`
SparkForge migration adapter	SparkForge Adapter
Structured Streaming	Experimental foundation only
Portable PySpark-style transformations	Portable Transformation

Do not use a design study as an installation or API reference. The capabilities page and API reference define the current boundary.