Workflow Plans and Execution Strategies¶

This document defines the architecture contract between the public workflow builders, durable Django task execution, and current or future Ray execution engines. The version 1 effective-plan materializer, identity pinning, dynamic/local selection metadata, Compiled Graph compatibility adapter, and Ray-free compiled-invocation lifecycle reducer are implemented. Static actors, a native Compiled Graph execution adapter, and a resident owner remain future strategy work.

The governing plan decision is ADR-0001. The first Compiled Graph ownership and reuse boundary is fixed separately by ADR-0002. Invocation state, deadlines, fallback, one-shot output ownership, and cleanup are fixed by ADR-0003.

Stable invariants¶

• One workflow run remains inside one durable RayTaskExecution lifecycle. Internal workflow nodes do not become independent Django tasks or durable retry records.
• WorkflowSignature is a reusable definition builder, not a persisted execution plan.
• An effective execution plan is versioned, immutable, canonical, secret-free, and independent of any Ray beta object.
• Invocation values are bound after plan materialization. Changing Kubernetes inventory, request identifiers, credentials, or other runtime data does not by itself change topology identity.
• Logical work cardinality and physical worker cardinality are separate facts.
• Compiled Graph is an execution strategy for an eligible plan, not a Django task type or a promise that arbitrary workflow definitions can be compiled.
• Strategy validation and selection complete before remote side effects begin. Automatic strategy fallback closes atomically when preparation starts, before an actor, channel, buffer, or other strategy resource may be allocated. Starting submission additionally forbids replay or automatic re-execution of that invocation.
• Session lifecycle actions use the complete durable run identity; invocation actions add a unique invocation_id. Plan or session equality never replaces either fence.
• Compiled invocation budgets are distinct absolute deadlines capped by the outer durable deadline. Phase changes never grant fresh relative time.
• Every compiled output slot has one owner and one terminal disposition before graph reuse. Native one-shot references never enter durable state.

Version 1 implementation boundary¶

Every WorkflowSignature.run() now materializes an effective plan before it submits a nested Ray task. The materializer resolves named per-step RuntimeEnv profiles once, freezes plan-relevant mappings, computes canonical JSON and a domain-separated SHA-256 fingerprint, and records deterministic eligibility diagnostics. Actual RuntimeEnv payloads remain in a separate in-memory execution binding because they can contain credentials; only a secret-free projection enters the plan.

Inside a durable task, the first plan is pinned on RayTaskExecution together with its bounded manifest, pinning attempt, and selection summary. A retry may rebuild the Python definition but must produce the same fingerprint and retry-safe environment bindings before any workflow leaf is submitted. A mismatch or retry-unsafe binding fails closed and requires a newly enqueued task. These fields remain observable when node progress is disabled:

• workflow_plan_fingerprint is the exact pinned retry identity and the required key for any future owner routing or prepared-strategy cache;
• workflow_plan_pinned_attempt identifies the attempt allowed to reuse runtime-only bindings without cross-attempt comparison;
• workflow_plan_json is the bounded canonical secret-free manifest; and
• workflow_plan_selection records the requested policy, selected baseline strategy, eligible strategies, and bounded rejections.

Local execution and ordinary dynamic Ray tasks are the only selectable strategies in this change. Current task-shaped leaves always receive an UNSUPPORTED_NODE_MODEL rejection for Compiled Graph. A dynamic map also receives DYNAMIC_TOPOLOGY, and unresolved code, RuntimeEnv, or platform identity adds its own rejection. A later strategy may route only by the exact fingerprint and must drain prepared state when it changes. The current cache_key(), owner assertion, and drain predicate are contract helpers; this release does not create a resident owner or operational graph cache.

Context-free WorkflowSignature.run() materializes its plan before the public path resolves a Ray executor. Version 1 therefore leaves compiler-owner topology and submission transport blank for that path and rejects Compiled Graph with OWNER_LIFETIME_MISMATCH; it does not infer direct-driver from process-global Ray state. Resolving and transporting an explicit direct-driver owner context belongs to the future execution adapter. Local and dynamic execution remain available.

The standard-library-only lifecycle protocol is executable independently of that adapter. It separates session health from invocation outcome, validates run and invocation identity on every event, issues one bounded deterministic action token at a time, and produces a versioned bounded snapshot without reading a clock or importing Ray. Its exact version is fingerprinted at strategy_requirements.compiled_graph.lifecycle_protocol_version. This does not make any current workflow compiled or promote a native compatibility row.

Vocabulary¶

These terms deliberately avoid using workflow as a synonym for every task. A Django task may contain no workflow, one workflow invocation, or eventually several repeated invocations of one effective plan.

Workload classification¶

The execution decision uses multiple axes rather than one broad task label:

Every plan also records these independent dimensions:

• topology class: static, dynamic, or fixed_width;
• expected invocation cardinality: once, a declared bound, or repeated;
• logical work-item cardinality and its bound, if known;
• task versus actor node model and synchronous versus asynchronous callable kind;
• physical stage and replica count;
• fan-out, admission, in-flight, queue, result-buffer, and retained-reference bounds;
• resources, placement, RuntimeEnv identity, transport, side effects, failure policy, owner lifetime, and durability boundary.

Kubernetes synchronization example¶

discover namespaces -> map(sync namespace) -> summarize is a dynamic workflow when the map creates one node per discovered namespace or resource. Namespaces being stable in practice does not make this topology static: the node count is still derived from invocation data.

A different plan can use a fixed number of actor replicas and pass the current namespace/resource inventory as invocation data. Each replica consumes a partition or bounded stream. That is a fixed_width physical topology even though the logical item count changes. It may become eligible for static actors or a compiled kernel if all other capability checks pass. The inventory, resource bodies, resource versions, API tokens, and credentials remain invocation data and are excluded from the plan fingerprint.

This distinction prevents two unsafe claims:

1. stable production inventory is not proof of static graph topology; and
2. changing inventory does not force topology invalidation when fixed-width nodes are intentionally designed to accept it as data.

Contract layers¶

Definition and plan template¶

The public builders remain convenient Python objects. A materializer traverses a definition and produces a plan template with stable operator IDs, input slots, callable references, and declared capabilities. Definition-time values are not automatically trusted as plan constants.

Current Step.bound_args, Step.bound_kwargs, and WorkflowSignature.run() arguments are lifted into named invocation slots by the compatibility adapter. A future explicit plan-constant API may embed a small canonical non-secret literal when changing that literal intentionally changes plan identity. The materializer must not guess that an arbitrary bound Python value is safe to persist or fingerprint.

Effective execution plan¶

Before the first remote submission, the materializer resolves the template against the deployment and produces an effective plan. At minimum it resolves:

• callable import paths, callable kinds, and an application definition/code revision;
• the outer RuntimeEnv identity and every per-step RuntimeEnv profile to canonical, secret-free specifications and content identities;
• Ray scheduling resources and placement requirements;
• physical actor stages and replica counts when present;
• transport, buffer, result-retention, admission, retry, cancellation, and owner requirements;
• Ray compatibility identity, operating platform capabilities, and strategy-specific settings that affect preparation or reuse.

Materialization deep-copies and normalizes every field. Mutating the source signature, settings dictionaries, or profile definitions afterward cannot change the effective plan.

Invocation envelope¶

An invocation envelope binds values to the effective plan. It carries durable run and invocation identity, input values or durable references, credentials supplied through the approved runtime channel, deadlines, and idempotency context. The envelope is not part of the plan fingerprint.

The input-slot schema, ordering, and binding rules are plan fields because changing them changes how values are interpreted. The values bound to those slots are not.

Version 1 effective-plan IR¶

The example below illustrates the normative field groups. It is not a public Python API or a database schema:

{
  "plan_format": "django-ray.workflow-plan",
  "plan_format_version": 1,
  "definition": {
    "name": "myapp.sync.fixed-width",
    "revision": "sha256:...",
    "build_revision": "build:0123456789abcdef",
    "container_image_digest": "sha256:..."
  },
  "topology": {
    "class": "fixed_width",
    "entry_ports": ["inventory", "sync-key"],
    "result_ports": ["summary"]
  },
  "nodes": [
    {
      "id": "partition",
      "operation": "call",
      "node_model": "actor",
      "callable": {
        "import_path": "myapp.workflows.partition_inventory",
        "kind": "sync"
      },
      "inputs": [{"port": "inventory", "source": "invocation:inventory"}],
      "outputs": ["partitions"],
      "stage": "partitioner"
    },
    {
      "id": "sync",
      "operation": "fixed_pool",
      "node_model": "actor",
      "callable": {
        "import_path": "myapp.workflows.sync_partition",
        "kind": "sync"
      },
      "inputs": [{"port": "items", "source": "node:partition:partitions"}],
      "outputs": ["results"],
      "stage": "sync-workers"
    }
  ],
  "edges": [
    {"source": "partition:partitions", "target": "sync:items", "transport": "object"}
  ],
  "physical_topology": {
    "stages": [
      {"id": "partitioner", "replicas": 1},
      {"id": "sync-workers", "replicas": 8}
    ]
  },
  "capabilities": {
    "invocations": {"cardinality": "repeated", "expected_count": 100},
    "logical_items": {"cardinality": "input_bounded", "maximum": 10000},
    "admission": {
      "maximum_in_flight": 1,
      "maximum_queued": 0,
      "maximum_buffered_results": 1
    },
    "effects": {"mode": "external_idempotent", "idempotency_slot": "sync-key"},
    "durability": {"boundary": "outer_task", "per_node_recovery": false},
    "owner": {"lifetime": "durable_run", "sharing": "isolated"}
  },
  "environments": {
    "outer": {"digest": "sha256:...", "reusable": true},
    "by_node": {"partition": "sha256:...", "sync": "sha256:..."}
  },
  "strategy_requirements": {
    "compiled_graph": {
      "settings_version": 1,
      "lifecycle_protocol_version": 1,
      "maximum_in_flight": 1,
      "maximum_buffered_results": 1,
      "owner_concurrency": 1,
      "transport": "cpu-shared-memory",
      "buffer_bytes": 1048576
    }
  },
  "compatibility": {
    "django_ray_plan_api": 1,
    "compiled_graph": {
      "schema_version": 2,
      "policy_version": 2,
      "reason": "CANDIDATE_REQUIRES_SMOKE",
      "topology": "nested-ray-task",
      "submission_transport": "direct-ray-core",
      "transport": "cpu-shared-memory",
      "runtime": {"ray_version": "2.56.1", "operating_system": "linux"}
    }
  }
}

The materialized representation must contain JSON values only. It must never contain a callable object, ObjectRef, actor handle, DAGNode, CompiledDAG, CompiledDAGRef, open client, file descriptor, coroutine, lock, or process-local identity.

Required field groups¶

Capability model¶

Capabilities are declarative constraints used for validation and selection. They do not prove that application code is safe, deterministic, or idempotent.

Strategy implementations may add versioned rejection rules, but they must not reinterpret the declared semantics. For example, an engine cannot silently treat unknown side effects as idempotent or an unbounded map as fixed width.

Plan fields versus invocation fields¶

The selected strategy, policy, plan fingerprint, and rejection diagnostics are durable run metadata, not node progress. They must remain available when node reporting is disabled, but the selection result is not folded back into the strategy-neutral semantic topology.

Canonical serialization and fingerprint¶

The version 1 fingerprint is computed from the effective plan's normative fields:

1. Validate against the exact plan_format_version schema and reject unknown normative fields.
2. Normalize import paths, enum values, resource keys, platform identifiers, input slots, and user-visible identifiers. Identifiers are Unicode NFC.
3. Preserve order where order has semantics, such as chain nodes, argument slots, result ports, and replica ranks. Sort schema-declared sets before serialization.
4. Reject non-finite numbers, ambiguous numeric forms, duplicate object keys, unserializable values, and values normalized through default=str.
5. Serialize the normalized model as canonical UTF-8 JSON with lexicographically ordered object keys and no insignificant whitespace. Numeric normalization must be deterministic across supported Python versions.
6. Compute SHA-256 over the ASCII domain separator django-ray.workflow-plan-v1\0 followed by the canonical bytes.
7. Render the identity as sha256:<lowercase hexadecimal digest>.

Two definitions that materialize to the same normative JSON must produce byte-for-byte equal canonical serialization and the same fingerprint. Non-semantic annotations may be stored beside the canonical plan, never inside the hashed payload.

The opt-in bounded-map result buffer does not introduce a new top-level plan field or change plan_format_version=1. An unselected map retains byte-for-byte the existing map node and empty physical_topology.actors/placement_relationships slots. A selected map links its existing actor_layout field to one entry in physical_topology.actors; that actor entry contains the versioned result-buffer protocol and ray.cloudpickle codec, item/in-flight/serialized-byte bounds, scheduler-visible CPU/memory/custom resources, placement, non-detached lifetime, fixed no-restart semantics, serial actor-call limits, direct two-return finalization, and best-effort cleanup contract. All of those fields are normative fingerprint inputs. A retry that changes any one of them is rejected before actor or leaf effects.

Changing the result buffer's exact maximum_pending_actor_calls bound from 1 to 2 therefore changes its canonical effective-plan identity. It does not increment plan_format_version or the result-buffer protocol version: the result-buffer work in GitHub issue #101 has not shipped in a release, and the version 1 actor contract already fingerprints the exact option and bound. No released persisted version 1 result-buffer plan is silently reinterpreted.

Invalidation inputs¶

A graph instance or prepared strategy state must be rejected and drained when any fingerprinted field changes. Important examples are:

• plan format, topology, ports, dependencies, callable path or kind;
• conservative application code/definition revision;
• explicit plan constant or binding schema;
• normalized resource, placement, stage, replica, actor, or concurrency layout;
• resolved outer or per-node RuntimeEnv content identity;
• transport, channel, serialization, buffer, retained-result, or compilation setting;
• lifecycle protocol version, side-effect, retry, cancellation, durability, owner, lifetime, or admission contract;
• a Ray, django-ray, optional dependency, operating-system, architecture, accelerator, or strategy capability change that the compatibility policy says is relevant.

Changing only invocation values, run identity, timestamps, observations, or progress does not invalidate the plan. A compatibility policy may deliberately declare a range of Ray patch versions equivalent, but that decision must be explicit, versioned, and tested; silently dropping Ray or platform identity is not acceptable.

Code and deployment identity¶

Callable paths do not prove which code will run. Version 1 therefore requires a conservative definition revision suitable for the deployment form:

The conservative revision may invalidate more often than perfect source-equivalence analysis would require. That is preferable to routing an invocation into actors that may contain incompatible code. Dynamic execution may remain available when a stable reusable identity is unavailable, but its rejection diagnostic must explain why static or compiled reuse is disabled.

Secret handling¶

Raw secrets are never plan fields and are never hashed. Hashing a low-entropy password, token, or namespace credential is not safe redaction.

When a secret changes process, actor, environment, or graph compatibility, the plan may contain only a stable provider reference and non-secret version/revision supplied by an approved secret system. The material value is injected after identity validation over a trusted runtime channel. If no non-secret compatibility identity exists, the plan is ineligible for reusable actors or compilation and receives a structured rejection. Existing RuntimeEnv dictionaries that embed secret values cannot be persisted in an effective plan merely because another surface redacts their display.

Credential-looking environment names are abstracted only by a declared provider and credential revision. All other values remain runtime-only unless an explicit environment_revision covers the entire configuration contract. Without that revision, reusable strategies receive UNRESOLVED_RUNTIME_ENV; with it, operators must change the revision for changes such as MODE=prod to MODE=dev. Token rotation can therefore remain stable without creating a secret-derived fingerprint.

Semantic Ray label selectors and fallback placement constraints require a separate non-secret scheduling_revision; an environment revision does not cover placement. Dashboard task names and annotation labels remain outside canonical plan identity.

Local working_dir and py_modules paths are always rejected for reusable strategies in version 1. django-ray computes a strong source snapshot, but Ray's local artifact lookup does not provide a matching end-to-end worker verification boundary. Remote artifacts therefore need an independently verifiable immutable identity before a resident or compiled strategy can reuse them.

Identity hashing is also budgeted. When an otherwise valid RuntimeEnv code tree or callable module exceeds the identity byte, file, or traversal budget, version 1 records a constant runtime-only identity and rejects reusable strategies; it does not reject the existing local or dynamic-task execution path.

Format compatibility and snapshot boundary¶

plan_format_version is an integer with fail-closed semantics:

• an executor must support the exact version before submission;
• unknown normative fields are rejected rather than ignored;
• changing fingerprint meaning, defaults, field interpretation, or required fields requires a new format version;
• non-semantic annotations may evolve outside the canonical payload;
• no executor may silently downgrade a plan or rewrite it after fingerprinting.

The effective plan is materialized and snapshotted before the first remote submission of a durable workflow run. The bounded, redacted manifest, canonical fingerprint, definition revision, and selection diagnostics must be persisted outside optional node progress. Version 1 detailed snapshots admit at most 64 logical plan nodes and 64 KiB of canonical JSON; repeated callable identities are deduplicated in a top-level table. Definitions that exceed either storage bound continue through local or dynamic-task execution with a bounded overflow snapshot, a digest covering the complete secret-free definition, observed counts, and PLAN_SNAPSHOT_OVERFLOW. They are not eligible for a reusable strategy. Large runtime inventories should still be invocation data consumed by bounded map_step, not duplicated static topology. The fingerprint, canonical manifest, the attempt that first pinned it, and current-attempt selection are stored on RayTaskExecution; selection diagnostics are not duplicated into progress snapshots. The detailed and overflow forms both retain bounded retry_safe and retry_unsafe_paths metadata for environment bindings whose raw values cannot enter the secret-free fingerprint.

When physical actor or placement detail contributes to the overflow, the persisted sentinel keeps empty detail arrays plus bounded actor, result-buffer, placement, and stage counts. snapshot.source_digest is computed from the complete secret-free manifest before compaction, so an omitted protocol, bound, resource, placement, lifetime, or restart change still changes the overflow fingerprint.

The 64 KiB limit bounds the durable canonical artifact, not all transient memory used while materializing a Python definition. The materializer still walks the definition and holds bounded builder metadata before it can emit the overflow snapshot. It is not an executor memory quota or a substitute for keeping runtime inventory out of topology.

The first successful materialization pins the plan identity for the RayTaskExecution attempt chain. A retry creates a new durable run identity and may rebuild the definition in a new process. Its materialized fingerprint must match the pinned identity before remote submission, but fingerprint equality alone is sufficient only when every RuntimeEnv binding is retry-safe. Opaque URIs, unsupported or malformed values, mutable dependency specifications, and environment or credential values without their required non-secret revision are retry-unsafe because distinct execution values can share the same redacted plan. Repeated use within the attempt that created the pin is allowed; a later attempt fails closed with bounded paths and remediation guidance.

Content-hashed local files and trees are retry-safe even though they remain ineligible for reusable strategies: their source bytes can be compared to the pin before upload. Immutable digests and correctly declared environment or credential revisions likewise restore retry safety for the values they cover. A rolling writer that left the pin attempt null may initialize it only for a retry-safe plan; retry-unsafe rows fail closed. A fingerprint mismatch always fails with an actionable plan-revision error. Submitting intentionally changed work requires a new task or a future explicit, audited migration policy.

This policy preserves the existing immutable outer RuntimeEnv intent and prevents a retry or resident owner from silently switching per-step profiles after a deployment. Queued work that has never materialized a plan continues to use the definition present when its first run begins.

Strategy eligibility and diagnostics¶

Every strategy returns a structured decision before preparation or submission:

{
  "plan_selection_format": "django-ray.workflow-plan-selection",
  "plan_selection_format_version": 1,
  "requested_policy": "auto",
  "selected_strategy": "dynamic_tasks",
  "eligible_strategies": ["local", "dynamic_tasks"],
  "rejections": [
    {
      "strategy": "compiled_graph",
      "code": "DYNAMIC_TOPOLOGY",
      "path": "topology.class",
      "message": "map node sync expands from invocation data"
    }
  ]
}

Diagnostics are bounded, deterministic for the same plan and capability set, ordered by strategy then code/path, and contain no input values or secret material. Each rejection has a stable machine-readable code, relevant plan path or node ID, and concise operator message. Multiple reasons are reported together where validation can continue safely.

The initial common rejection codes are:

An explicit strategy request fails during validation if rejected. auto may select another eligible strategy and records all relevant rejections only until preparation starts. Issuing the preparation action atomically closes strategy fallback before an actor, channel, buffer, or reservation can be created. Issuing the submission action also permanently forbids replay or automatic re-execution of that invocation. The selected strategy must then use its failure, cancellation, output-accounting, drain, quarantine, and teardown contract.

Eligibility summary¶

Ray Compiled Graph is currently beta and optimizes repeated execution of a static graph. Its current ownership, actor, capacity, result-consumption, and teardown limitations are strategy capability checks, not new task semantics. See the upstream Compiled Graph overview and troubleshooting limitations, plus django-ray's fail-closed Compiled Graph compatibility policy.

Compiled invocation lifecycle protocol¶

Lifecycle protocol version 1 is a deterministic reducer and adapter interface. The reducer owns transitions and never imports Ray, reads a clock, performs I/O, or stores an event history. It retains one bounded action token/generation. The adapter dispatches that exact action and returns an event carrying the same token and required identity; stale, duplicate, or out-of-order events are rejected.

Version 1 accepts exactly one owner process, one graph instance, one caller, owner adapter concurrency of one, one in-flight invocation, one buffered invocation result, and zero owner-side queued invocations. The exact adapter actions are VALIDATE, PREPARE, ADMIT, SUBMIT, CONSUME_OUTPUT, RELEASE_CAPACITY, CANCEL, DRAIN_INVOCATION, DRAIN_SESSION, CHECK_HEALTH, and TEARDOWN. Other capacity values or an unsupported action fail closed.

The executable LifecycleCapacity fields are session_owners, callers, maximum_in_flight, maximum_buffered_results, owner_concurrency, and queue_capacity, with respective values 1, 1, 1, 1, 1, and 0.

Session actions for validation, preparation, readiness, idle drain, quarantine, and teardown carry the complete four-field durable run identity. Invocation actions for admission, submission, callback, cancellation, result consumption, invocation drain, and cleanup add invocation_id. Session state and invocation state remain independent: an invocation outcome does not prove graph health, and teardown does not rewrite the invocation's primary outcome.

The protocol uses exactly seven absolute deadlines: outer, admission, submission, result, cancellation, drain, and teardown. Validation and preparation are capped only by outer. Every inner deadline is capped by the outer durable deadline, which wins ties. The adapter observes time and returns a deadline event; the reducer never reads a clock or refreshes a budget after a state change. Action-scoped DEADLINE_EXPIRED echoes its token. Tokenless OUTER_DEADLINE_EXPIRED supersedes pending work at the outer boundary and dispatches no late action.

There are two irreversible cutoffs:

1. issuing preparation start closes automatic strategy fallback before the adapter may allocate any resource; cleanup never reopens fallback; and
2. issuing submission start closes replay and automatic re-execution of that invocation, even if the adapter later proves rejection before execution.

A proven rejection may affect only the retry disposition of a future durable attempt. A submit timeout is otherwise ambiguous and records MAY_HAVE_STARTED plus APPLICATION_POLICY_REQUIRED; draining and a positive health result may still permit independent graph reuse, never replay. Owner loss after submission may have begun is indeterminate and non-adoptable.

Starting result retrieval spends a slot's one-shot opportunity; a timeout is not a poll and cannot issue a second get. A slot is NOT_CREATED only when submission is proven not to have started. Created outputs become CONSUMED, ADAPTER_RELEASED, LOST_WITH_OWNER, CLEANUP_UNCONFIRMED, or UNAVAILABLE_AFTER_TEARDOWN; the last three never authorize graph reuse. CLEANUP_UNCONFIRMED truthfully terminates output accounting when cleanup or teardown failure leaves release unconfirmed, while UNAVAILABLE_AFTER_TEARDOWN requires confirmed teardown. Reuse additionally requires a positive session-health classification. Primary outcome, effect state (NOT_STARTED or MAY_HAVE_STARTED), graph disposition, retry disposition, and cleanup diagnostics are separate facts. Retry disposition is AUTOMATIC_ALLOWED, APPLICATION_POLICY_REQUIRED, OPERATOR_RECONCILIATION_REQUIRED, or PROHIBITED; none rewinds the invocation. An adapter-reported DRAIN_FAILURE is distinct from deadline-driven DRAIN_TIMEOUT; both require teardown without replacing an earlier primary outcome.

Owner loss is classified at the submission cutoff. Before submission it leaves NOT_STARTED, NOT_CREATED, and AUTOMATIC_ALLOWED. After submission it leaves MAY_HAVE_STARTED, LOST_WITH_OWNER, and OPERATOR_RECONCILIATION_REQUIRED; that invocation is non-adoptable and the graph cannot be reused.

The versioned snapshot is secret-free, JSON-safe, limited to 64 output slots, eight cleanup diagnostic codes, and 16,384 JSON bytes, and contains no payload, result value, credential, exception object, Ray handle, channel, DAG node, or compiled reference. See ADR-0003: Compiled Invocation Lifecycle for the normative state and evidence boundaries.

The 64-slot reducer bound is accounting capacity for a versioned protocol, not native multi-output eligibility. The first adapter boundary remains one buffered invocation result until issue #116 supplies the additional Ray transport evidence.

The reducer does not prove Kubernetes side-effect safety or native multi-output and zero-copy behavior. Issue #115 owns idempotency, partial-write, cancellation, and reconciliation evidence for Kubernetes synchronization. Issue #116 owns exact transport and output evidence. They do not block the Ray-free reducer, but they block those capability claims in a native strategy.

Run, invocation, and observability identity¶

A durable workflow run identity contains at least:

(task_execution_pk, attempt_number, execution_generation, run_id)

Every invocation adds an invocation_id unique within that run. Every progress revision, strategy callback, result, cancellation, terminal flush, owner request, and cleanup action is scoped to this identity. The plan fingerprint and definition revision describe what is intended; run and invocation identity describe which execution may write current state.

The compatibility path for the version 1 progress schema is additive:

• current dynamic node IDs remain runtime expansion paths scoped to one invocation;
• progress snapshots carry a compact versioned run and plan summary; the selected strategy and full rejection diagnostics remain in the separate durable selection;
• a client resets rather than merges state when run or invocation identity changes;
• progress revisions restart only with a new run/invocation identity;
• node reporting may be disabled, but run identity, plan fingerprint, requested policy, selected strategy, and bounded rejection summary remain available through durable task observability;
• progress remains observational and is never a recovery log.

Attempt/generation write fencing is implemented separately; this document defines the identity that the persistence and strategy layers consume.

Current WorkflowSignature compatibility¶

The public API remains source compatible while the plan boundary is introduced:

The adapter preserves current argument ordering, bound-keyword precedence, ordered group/map results, local fallback, importability validation, RuntimeEnv override behavior, and one outer task durability boundary. Current Step dataclasses are frozen at the attribute level and now deep-freeze plan-relevant RuntimeEnv and Ray-option mappings. Bound application values remain invocation data and may contain ordinary mutable objects. The materialized plan, rather than the signature object, remains the durable immutable boundary.

The ordered-fold actor uses the physical-topology and actor-contract extensibility slots introduced in plan format version 1, so adding it does not reinterpret any existing plan field and does not require a PLAN_FORMAT_VERSION increment. Its own protocol and codec versions are explicit inside the actor contract. Overflow snapshots omit expanded actor details, retain separate result-buffer and result-fold counts, and preserve the complete omitted identity through snapshot.source_digest.

A dotted callable supplied only by a step RuntimeEnv does not need to import on the Django submitter. It materializes as worker-only code and remains dynamically executable, while reusable strategies receive UNRESOLVED_CODE_IDENTITY. Stateful callable instances and partials receive the same conservative rejection because their runtime-bound state is not a safe canonical plan field.

No current caller becomes compiled merely because its chain or group is logically static. Current steps are Ray task nodes. Static actors, an owner, and all eligibility constraints must first be introduced and validated.

Acceptance contract for plan snapshot implementation¶

Issue #84 and later implementations can cite these stable requirements:

1. PLAN-01 -- Pre-submit boundary: materialization, canonical validation, fingerprinting, and strategy eligibility complete before any remote submission, actor creation, channel allocation, or application side effect.
2. PLAN-02 -- Deep immutability: the effective plan contains immutable normalized data; mutation of plan-relevant Ray options, settings, or RuntimeEnv profiles after materialization has no effect. Bound application values remain outside the plan as invocation data and retain their existing Python types and mutability.
3. PLAN-03 -- Canonical equality: semantically equal definitions under the same compatibility inputs produce byte-equal canonical JSON and equal fingerprints across supported Python processes.
4. PLAN-04 -- Conservative invalidation: every topology, callable/revision, resource, placement, actor layout, resolved environment, transport, buffer, compile, lifecycle, owner, Ray, or platform input listed above changes identity unless a versioned compatibility rule explicitly proves equivalence.
5. PLAN-05 -- Resolved environments: named outer and per-step profiles are resolved at materialization; profile content changes cannot reuse the old fingerprint.
6. PLAN-06 -- Secret-free identity: canonical bytes, persisted manifests, diagnostics, logs, and fingerprints contain no secret material or digest of raw secret material. Unresolved secret compatibility rejects reusable strategies.
7. PLAN-07 -- Invocation separation: task arguments, lifted bound values, Kubernetes inventory, request IDs, credentials, run identity, and observed state are invocation fields. Changing only those values does not change the fingerprint.
8. PLAN-08 -- Stable code identity: package, image, archive, and development code use a documented conservative revision. A process-local or unverifiable identity is rejected for reusable strategies with UNRESOLVED_CODE_IDENTITY.
9. PLAN-09 -- Retry pinning: the first materialized fingerprint and attempt are pinned to the execution attempt chain. A retry mismatch or retry-unsafe environment fails before submission and cannot route to an old owner or silently adopt a new definition or execution binding.
10. PLAN-10 -- Structured diagnostics: validation returns stable code, plan path or node, and redacted message for every safely discoverable rejection. Explicit and automatic selection behavior follows the pre-preparation fallback rule and the submission replay cutoff.
11. PLAN-11 -- Durable summary: run identity, plan version/fingerprint, definition revision, requested policy, selected strategy, and bounded rejection summary remain observable when node progress reporting is disabled.
12. PLAN-12 -- API parity: existing step, chain, group, map_step, with_options, with_runtime_env, run, and local execution tests retain their documented result and argument semantics on the default dynamic strategy.
13. PLAN-13 -- Runtime-node scoping: dynamic map expansion IDs, Ray IDs, graph handles, progress revisions, results, and timings are scoped to run/invocation state and never alter or enter canonical plan identity.
14. PLAN-14 -- Bounded persistence: any stored plan manifest and diagnostic summary has explicit size/depth/count limits, redaction, and versioning. Runtime engine objects and unbounded node graphs are never persisted as plan metadata.
15. PLAN-15 -- Result-buffer identity: selecting the bounded-map result buffer populates only the version 1 physical-actor and actor-layout extensibility slots. Its protocol, codec, bounds, resources, placement, lifetime, restart, actor-call, finalization, and cleanup semantics are fingerprinted; an unselected map's canonical manifest is unchanged.
16. PLAN-16 -- Compiled lifecycle identity: every effective plan contains the exact supported strategy_requirements.compiled_graph.lifecycle_protocol_version. Changing lifecycle transition meaning, fallback or replay cutoffs, deadline precedence, output ownership, or snapshot interpretation changes that version and therefore the plan fingerprint. The Ray-free reducer rejects stale identity/action tokens and unsupported protocol versions without invoking a native engine.

These requirements do not choose a database model, resident-owner topology, or compiled adapter. Those decisions remain in their focused issues.

Non-goals¶

• Implement an effective-plan materializer, executor, owner, actor pool, or Compiled Graph integration in this documentation change.
• Add a database row for each logical work item or workflow node.
• Change the outer at-least-once durability and retry boundary.
• Infer that a side-effect declaration is true without application tests and controls.
• Make arbitrary Django tasks, dynamic maps, or process-local Python objects eligible for compilation.

Related documentation¶

• Architecture
• Ray-Native Workflows
• Performance
• Runtime Environments
• Retry and Error Handling
• ADR-0001: Workflow plans and execution strategies
• ADR-0002: Compiled session ownership and reuse
• ADR-0003: Compiled invocation lifecycle