Publishing Pipeline

The publishing workflow in pipeline.py runs five sequential phases:

collect → validate → config → extract → integrate

Phases

Phase	Role
collect	Gather scene data into named slots on PipelineContext
validate	Check collected data meets requirements; abort on failure
config	Modify scenes and settings temporarily for exports and renders
extract	Process/render/export the data to disk
integrate	Copy outputs into the versioned publish tree

Actions

Each phase is a list of @dataclass action instances (CollectAction, ValidateAction, ConfigAction, ExtractAction, IntegrateAction). Actions communicate through named slots on PipelineContext.

Every action implements execute(ctx) and may optionally override cleanup(ctx) (default: no-op) to release anything it acquired — a scratch directory, a changed scene setting, a lock. PublishPipeline.run() calls cleanup() once the pipeline is done, in reverse execution order (last action run, first cleaned up), for every action that actually executed. This runs both after a fully successful pass and when a later action raises — in the failure case, only the actions that already ran are cleaned up; the one that raised and anything after it are not. LoadPipeline.run() follows the same rule for its ordered [[load]] steps.

Running a pipeline

PublishPipeline.run(on_phase=None) runs all five phases in order. Pass on_phase — a callable taking the phase name — to be notified right before each phase starts (used by the GUI backend to emit progress signals between phases). Cleanup (see above) always runs once run() returns or raises, so callers don't need to handle it themselves.

Pipeline context

Publish and load pipelines share the same PipelineContext class — there are no separate publish/load contexts. Besides the slot dict it carries entity, task, product_type, dcc, and settings, plus fields for workfile tracking, layered products and loads:

Field	Set by	Meaning
source	PipelineContext.from_session()	Identifier of the workfile the pipeline runs from ("{entity_type}:{entity}:{task}:{layer}:{version}", e.g. "shot:sq010_sh020:comp:main:3"); written as source into the publish metadata. Empty when there is no open workfile (e.g. standalone publisher)
layer	Publish UI layer selector / Session.load_product()	Layer name to publish under or load; empty = main layer
version	Session.load_product()	Version to load; -1 (VERSION_LATEST) = most recent
extra	Publish actions (e.g. ConfigAssetPipelineAsset, CollectProductFile)	Pass-through product values core doesn't interpret, e.g. {collection = "Prop-Tree"}; IntegrateFile writes them as the product entry's nested extra table (core fields like version, layer, note, source stay top-level). Load actions read them from the entry, e.g. LoadBlenderLink the collection name
ref_id	Session.load_product()	Reference id the loaded scene objects are tagged with

PipelineContext.from_session(pipeline_key) builds a context from the active session's entity, task and workfile; the values are captured once, so actions that change the session during the run (e.g. ActionIncrementWorkfile) don't affect it. Workfile entries in kpipe-metadata.toml record the same source for the version they were saved from, so every increment links back to its basis.

The publish UIs (Blender N-panel and standalone publisher) show a layer selector for every pipeline unless its config sets layers = false. Existing layer names are suggested via Entity.get_product_layers(product_type); typing a new name creates a new layer. A pipeline can instead fix its layer with layer = "<name>" (together with layers = false).

Action discovery

Actions are discovered at runtime via load_custom_actions(). Later entries override earlier ones:

1. kpipe/actions/ (in packages/core) — built-in actions (loaded on import kpipe)
2. kpipe_<dcc>/…/actions/ (in packages/<dcc>) — DCC-specific actions (loaded by the extension's register(); Blender passes package= so they are imported as submodules of the extension and use relative imports like from ..media_blender import … — Blender's extension policy forbids sys.path changes and new top-level modules)
3. {KPIPE_ROOT}/actions/ — project-level custom actions (loaded lazily on first PublishPipeline.from_config())

Configuring a pipeline

Pipelines are defined per product-type and DCC in configs/<bundle>/kpipe-publish.toml:

[pipeline.render_blender_cycles]
dcc = "blender"
product_type = "render_exr"

  [pipeline.render_blender_cycles.settings]
  use_selection = false

  [[pipeline.render_blender_cycles.collect]]
  action = "CollectBlenderObjects"
  slot = "beauty"
  collection_names = ["Render_Beauty"]

  [[pipeline.render_blender_cycles.integrate]]
  action = "IntegrateImageSequence"
  slot = "beauty"