Evaluator Brief¶
A one-page answer for enterprise evaluators and technical decision-makers.
What ETLantic is¶
A typed, contract-driven modeling layer for data pipelines in Python. You define datasets, transformations, and pipelines once; ETLantic validates and plans them; plugins execute.
It is not a dataframe engine, distributed scheduler, warehouse, or secret manager.
What is stable in bounded 0.21.0¶
| Area | Ready? |
|---|---|
Typed authoring (Data, Transformation, Pipeline) |
Yes |
Validation and secret-free PipelinePlan |
Yes |
| ODCS / DTCS / DPCS interchange | Yes |
| Local in-process runtime + run reports | Yes |
| Memory / callable / JSON / CSV / no-write storage | Yes |
| Env + mounted-file secrets | Yes |
| Polars / Pandas plugins | Yes (separate packages) |
SQL plugin (etlantic-sql) |
Yes (PostgreSQL reference) |
PySpark plugin (etlantic-pyspark) |
Yes (local provider; batch production path) |
| Structured Streaming | Experimental |
| Airflow / orchestrator compilation | Yes (etlantic-airflow) |
| DTCS 3.0 portable plan models/profiles | Yes (install dtcs>=0.13,<1; content floor dtcs 0.14.0) |
@Transformation.portable authoring |
Yes (0.11) |
Portable Polars compiler (kernel + relational /1) |
Yes (0.13+) |
Portable PySpark compiler (kernel + relational /1) |
Yes (0.13+) |
Portable Pandas compiler (kernel + relational /1, eager) |
Yes (0.14) |
Portable SQL compiler (kernel + relational /1) |
Yes (0.15) |
| Public portable transform conformance suite | Yes (0.14) |
| Multi-tenant durable orchestration | No |
| Formal SLA / support response times | No |
Security posture¶
- Plans never contain resolved secrets
- SQL plugins use structured compilation with identifier/parameter safety; untrusted raw SQL is out of scope
- Spark session credentials resolve at acquire time and never embed in plans
- Plugin allowlists / version pins are available via
Profile.plugin_allowlist(production profiles fail closed when empty) - 0.20–0.21 trust controls shipped: SafeIoPolicy, pre-import allowlist +
manifests, artifact/cache isolation keys, outbound default-deny,
unsafe-serialization prohibition, versioned
SecurityEvent, release SBOM digests and GitHub attestations - Report vulnerabilities privately; security fixes are supported on 0.21.x
Shipped vs residual (0.21)¶
| Concern | Status |
|---|---|
Secret-free plans/reports; security_mode |
Shipped |
| Production plugin allowlist (selection, not sandbox) | Shipped |
| Safe I/O, outbound default-deny, serialization ban | Shipped |
| Artifact/cache isolation keys (single-tenant reference) | Shipped |
| Release SBOM digests + GitHub attestations | Shipped |
| Cross-tenant / multi-tenant isolation guarantees | Residual / adopter-owned |
| Formal DoS capacity SLA | Residual (partial I/O budgets only) |
| Compliance-grade audit system of record | Adopter-owned (CLI reports are operational evidence) |
| HA/DR, SOC2/GDPR certs, identity/RBAC/SSO | Adopter-owned / out of scope |
| In-process multi-tenancy | Out of scope — use process isolation |
Read Security and the repository security policy. For the bounded reference topology and required controls, read Production Readiness.
Bounded production support (do not skip)¶
ETLantic 0.21.0 is production/stable for documented single-tenant reference deployments. Shipped trust controls do not make an arbitrary multi-tenant topology safe.
Residual items that block unrestricted enterprise-wide production claims:
| Residual | Why it matters |
|---|---|
| Provenance beyond allowlist/pins + release attestations | Broader supply-chain programs remain adopter-owned |
| Cross-tenant artifact/cache isolation | Single-tenant isolation keys are not a multi-tenant control plane |
| Formal DoS budgets / capacity SLA | Partial I/O budgets only |
| Compliance audit SoR | Durable/file reports are operational evidence, not compliance |
| In-process multi-tenancy | Explicitly out of scope—use process isolation |
Treat CLI run reports under .etlantic/reports/ as operational evidence (not
an audit system of record). Pass --ephemeral only when you want process-local
storage.
How to read status labels in deeper chapters: Documentation Status.
What remains outside the stable envelope¶
- Copying long Airflow design study tutorials into production—use
examples/airflow_compile.pyandetlantic-airflowinstead - Treating Structured Streaming APIs as stable (they are experimental)
- AWS Secrets Manager / Vault (not shipped); OS keyring is available via
etlantic-keyring - Process-local / durable file reports as an audit system of record
- Stable 1.0 compatibility guarantees
- Managed Databricks/EMR/Connect Spark providers
- Undocumented advanced portable profiles — Polars and PySpark ship the
documented 0.17 Wave 1 / Wave 2 families; Pandas and SQL remain at kernel +
portable-relational/1. Continuation profiles remain outside the advertised claim set. Keep a native@implementation(...)for profiles outside the advertised claim set, or forportable_transform_policy="native".
Enterprise readiness matrix¶
| Concern | Status in 0.21 |
|---|---|
| License | MIT (core and official plugins) |
| Supported versions / EOL | Current stable line is 0.21.x; see SECURITY.md |
| Compliance attestations (SOC2, GDPR cert) | Adopter-owned — not provided |
| Identity / RBAC / SSO | Out of scope — use process and network isolation |
| HA / DR / RPO / RTO | Adopter-owned topology |
| SBOM / signed provenance | Release CI emits SPDX SBOM digests and GitHub build provenance attestations |
| Audit system of record | Gap — durable/file reports are operational evidence only |
| Tested scale | Local/pilot workloads; no published capacity guarantees |
| Upgrade / rollback | Pin exact versions; see Migration 0.20 → 0.21 |
Recommended evaluation path¶
Follow this path after the green path (Install → Quickstart → First Pipeline → Engine selection), or as an enterprise diligence track:
- Installation —
pip install etlantic==0.21.0 - Quickstart (
etlantic init;examples/requires a checkout) - First Pipeline
- Engine selection
- Capabilities
- Optional Gate A: checkout
examples/interchange_polars_pandas.pywithetlantic-polars+etlantic-pandasat==0.21.0 - Optional engine examples from a checkout (portable kernels, SQL, PySpark, Airflow compile, Prefect)
- Migration 0.20 → 0.21 if upgrading; otherwise Upgrade hub
- Roadmap summary for sequencing
- Production path: create
profiles/prod.jsonfrom CI starter / prod.example.json and see Production profiles
Support channel¶
GitHub issues for bugs and questions. Include ETLantic version, Python version, and a minimal reproduction. Never include credentials or production data. See SUPPORT.md.