Diagnostics Reference

Diagnostics are structured findings produced while loading, inspecting, validating, planning, compiling, or executing a pipeline.

They are intended for people, CI systems, editors, and plugin tooling.

Diagnostic Model

A diagnostic should contain:

Diagnostic(
    code="PMPIPE201",
    severity=Severity.ERROR,
    message='Unknown input "cleaned_customer".',
    path=("pipeline", "publish", "input"),
    source=SourceLocation(...),
    help='Did you mean "cleaned_customers"?',
    related=(...),
    metadata={...},
)

Severity

Severity	Meaning
error	The operation cannot safely continue
warning	The model is valid but may be unsafe or surprising
info	Relevant explanatory information
hint	Optional improvement or editor assistance

Only errors make a validation report invalid by default.

Diagnostic Namespaces

ETLantic-owned codes should use stable categories:

PMSRCxxx   Source and import loading
PMTYPExxx  Type annotations and model introspection
PMDATAxxx  Data-contract integration
PMTRNxxx   Transformation definitions and implementations
PMXFORMxxx Portable transformation authoring, IR, compilers, and execution
PMPIPExxx  Pipeline topology and wiring
PMPLANxxx  Planning and capability resolution
PMPLUGxxx  Plugin trust / allowlist (e.g. PMPLUG401, PMPLUG402)
PMORCHxxx  Orchestration / compile diagnostics
PMSPARKxxx Spark capability and runtime diagnostics
PMDFxxx    Dataframe plugin diagnostics
PMXFORMxxx Portable transform authoring / compiler diagnostics
PMEXECxxx  Execution lifecycle
PMCFGxxx   Configuration and profiles
PMGENxxx   Contract and documentation generation
PMINTxxx   Internal framework invariants

Standards and plugins retain their own namespaces, such as ODCS, DTCS, DPCS, or a documented plugin prefix.

Practical code index

These codes are emitted by the 0.18 source tree. The message, path, metadata, and severity provide the case-specific detail. Maintainers can regenerate a machine-readable inventory with:

uv run python scripts/generate_diagnostics_catalog.py

The script prints every PMxxxx / related code literal found under src/etlantic. The curated tables below remain the human-oriented index.

Pipeline and planning

Code	Meaning
PMPIPE201	A pipeline member, connection, or referenced port is invalid or unresolved
PMPIPE210	Connected producer and consumer contracts are incompatible
PMPIPE220	An invalid-output port feeds a normal required input
PMPIPE301	Pipeline graph contains a cycle
PMPIPE302	The logical graph could not be built
PMPLAN201	An Extract/Load asset has no binding in the selected profile or registry
PMPLAN202	A node contract lacks a published ODCS identifier
PMPLAN301	A step has no implementation for the selected engine
PMPLAN401	No plugin capabilities are registered for the selected engine
PMPLAN402	A required capability is unsupported
PMPLAN403	Planning selected an allowed capability fallback

Plugin trust and portable transforms

Code	Meaning
PMPLUG401	A production profile has an empty plugin allowlist and fails closed
PMPLUG402	A discovered plugin is not allowlisted or does not match its version constraint
PMXFORM201	A declared portable output is missing from the return value
PMXFORM202	A portable definition returned an undeclared output
PMXFORM301	A compiler cannot satisfy a portable operation or capability requirement
PMXFORM302	Portable compilation is required but no suitable compiler is registered
PMXFORM501	Portable execution failed
PMXFORM801	Portable IR captured a callable
PMXFORM802	Portable IR contains a forbidden binary literal
PMXFORM803	Portable IR captured a secret value or reference
PMXFORM810	Portable plan exceeds the document-size budget
PMXFORM811	Portable plan exceeds the node-count budget
PMXFORM812	Portable plan exceeds the depth budget

Orchestration and execution

Code	Meaning
PMORCH300	The requested orchestrator compiler plugin is missing or compilation failed
PMORCH301	The orchestrator lacks a required capability
PMORCH340	An in-memory artifact cannot cross the external orchestration boundary
PMORCH341	An oversized inline artifact requires durable transport
PMORCH342	Artifact metadata appears to contain a secret
PMEXEC100	Pipeline.run() was called from an active event loop; use arun()
PMEXEC300	A runtime node failed
PMEXEC301	Failure policy continued or skipped a node after an upstream failure
PMEXEC320	A planned step lacks required transformation identity or registration
PMEXEC330	Runtime input or output validation failed
PMEXEC401	An environment-backed secret is unavailable
PMEXEC402	A file-backed secret cannot be loaded safely
PMEXEC420	A dataframe plugin is unavailable, or a JSON binding lacks a location
PMEXEC430	A SQL plugin is unavailable, or a CSV binding lacks a location
PMEXEC440	No Spark plugin is available for the selected engine

Backend-specific

Code	Meaning
PMDF410	A dataframe row failed contract validation
PMSPARK220	Spark schema compatibility produced a lossy or incompatible finding
PMSPARK221	Spark schema inspection failed
PMSPARK310	The profile's UDF policy forbids a planned Spark UDF strategy
PMSPARK320	A batch-only transformation was placed in a streaming region

Search the exact code in the source or include it in an issue when a code is not listed here. Do not suppress trust, secret, or semantic safety diagnostics.

Source Locations

When available, diagnostics should identify:

• File or URI
• Line and column
• Python object or class
• Contract path
• Pipeline node and port
• Generated artifact

Example:

src/pipelines/customer.py:42:9 PMPIPE201

The step "publish_customers" expects Customer, but received RawCustomer
from "load_customers.result".

help: connect the output of NormalizeCustomers or change the sink contract

Related Locations

A diagnostic may refer to more than one place:

• The consumer port
• The producer port
• The relevant contract declaration
• The selected implementation

Related locations make type and compatibility errors explainable without flattening them into a single message.

Reports

Operations return reports containing diagnostics:

report = CustomerPipeline.validate()

if not report.valid:
    report.raise_for_errors()

Reports should support:

• Filtering by severity or code
• Stable ordering
• JSON serialization
• Human rendering
• SARIF export
• Summary counts

Exceptions and Diagnostics

Expected user errors should become diagnostics. Exceptions are reserved for invalid API usage, I/O failures configured as fatal, plugin crashes, or broken framework invariants.

An exception raised by a convenience method should retain its report:

try:
    CustomerPipeline.plan(profile="production")
except PipelineValidationError as exc:
    print(exc.report)

Validation Diagnostics

Validation may report:

• Invalid data-contract types
• Missing transformation inputs
• Incompatible output and input contracts
• Cycles
• Duplicate identifiers
• Invalid parameters
• Missing sinks
• Unsupported subpipeline boundaries
• Invalid portable expression names, types, outputs, or bounded structure

Planning Diagnostics

Planning may report:

• No compatible implementation
• Missing plugin
• Unsupported capability
• Ambiguous binding
• Unsafe artifact boundary
• Unsupported orchestrator behavior
• Incompatible SQL or Spark dialect
• Unsupported portable operation, function, type, or semantic mode
• Ambiguous portable/native selection or prohibited fallback

Portable diagnostics use expression paths such as outputs.result.project.full_name and reserve these ranges:

PMXFORM1xx authoring and signatures
PMXFORM2xx names, types, contracts, and outputs
PMXFORM3xx compiler selection and capabilities
PMXFORM4xx lowering and semantic mismatch
PMXFORM5xx portable runtime execution
PMXFORM8xx security and bounded-input rejection
PMXFORM9xx internal invariants

Shipped authoring codes in 0.11 include:

Code	Meaning
PMXFORM101	Portable definition signature mismatch or excluded F.expr
PMXFORM110	Return value is not a FrameExpr / output mapping
PMXFORM201	Declared output missing from portable return value
PMXFORM202	Undeclared output returned
PMXFORM203	Single FrameExpr return with multiple outputs
PMXFORM801–803	Callable / binary / secret capture rejection
PMXFORM810–812	Document size / node count / depth budget exceeded
PMXFORM901	Unexpected plan protocol identity

Execution Diagnostics

Execution findings should distinguish:

• Failed
• Timed out
• Cancelled
• Skipped
• Retrying
• Abandoned
• Invalid input data
• Invalid output data

Runtime exceptions should be normalized without hiding the original exception.

Suppression

Suppressions should be explicit, narrow, and reviewable:

class CustomerPipeline(Pipeline):
    model_config = {
        "diagnostic_suppressions": {
            "PMPIPE410": "Legacy source retained during migration",
        }
    }

Suppressing errors that protect required semantics should not be allowed.

Machine-Readable Output

etlantic validate path/to/pipeline.py:CustomerPipeline --format json
etlantic validate path/to/pipeline.py:CustomerPipeline --format sarif

Machine output should use stable field names and diagnostic codes even when human wording improves.

Plugin Requirements

Plugins should:

• Emit structured diagnostics rather than printing
• Use stable documented codes
• Attach node and plugin identity
• Preserve causal exceptions
• Avoid leaking secrets
• Suggest remediation when possible

See Also

• Exceptions
• Pipeline Validation
• Error Handling