Runtime configuration (shipped)¶

Status: Available in ETLantic 0.21.0. Configure profiles, bindings, and engines in Python or JSON. Optional etlantic.toml may set default_profile and named profile references. Only the environment variables listed here are read by shipped code.

Profiles in Python¶

from etlantic import Profile

profile = Profile(
    name="production",
    security_mode="production",
    orchestrator="local",
    dataframe_engine="polars",  # or None / "pandas"
    sql_engine="sql",           # requires etlantic-sql
    spark_engine="pyspark",     # requires etlantic-pyspark
    validation_policy="strict",
    plugin_allowlist={
        "etlantic-polars": "==0.21.0",
        "etlantic-sql": "==0.21.0",
    },
    assets={"customer_source": "customers"},
)

Pass profile= to validate, plan, run, and compile_plan. Production fail-closed trust uses security_mode="production" only (not the profile name).

Optional project file¶

When etlantic.toml exists at the project root, ETLantic loads default_profile and optional [profiles] entries. See Configuration today. Absent that file, profiles resolve from profiles/{name}.json, built-ins, or explicit JSON paths.

Environment variables¶

Variable	Used by	Meaning
`ETLANTIC_SQL_URL`	`etlantic-sql`	SQLAlchemy URL (PostgreSQL reference; SQLite demo-only)
`ETLANTIC_SPARK_BACKEND`	`etlantic-pyspark` tests	Set to `sparkless` for JVM-free Spark tests
`ETLANTIC_SECRET_*`	`EnvSecretProvider`	Secret values when using the env provider with that prefix

Example:

export ETLANTIC_SQL_URL=postgresql+psycopg://user:pass@localhost:5432/etlantic
# demo only:
export ETLANTIC_SQL_URL=sqlite+pysqlite:///:memory:

Observability (optional)¶

Install the optional OpenTelemetry extra when available:

pip install 'etlantic[otel]'

Wire an observability provider through the public etlantic.observability protocol. JSON console logging is available without OTel. See Logging and the API reference.

Not shipped¶

Do not configure these as if they exist in 0.21:

ETLANTIC_CONFIG / ETLANTIC_PROFILE / ETLANTIC_PROJECT auto-loading
AWS Secrets Manager / Vault providers (OS keyring is optional via etlantic-keyring)

Proposed 1.0 names beyond the optional project toml live under Future Design: Configuration and Environment Variables.