Skip to content

Publishing Pipeline

The publishing workflow in pipeline.py runs five sequential phases:

collect → validate → config → extract → integrate

Phases

Phase Role
collect Gather scene data into named slots on PipelineContext
validate Check collected data meets requirements; abort on failure
config Modify scenes and settings temporarily for exports and renders
extract Process/render/export the data to disk
integrate Copy outputs into the versioned publish tree

Actions

Each phase is a list of @dataclass action instances (CollectAction, ValidateAction, ConfigAction, ExtractAction, IntegrateAction). Actions communicate through named slots on PipelineContext.

Every action implements execute(ctx) and may optionally override cleanup(ctx) (default: no-op) to release anything it acquired — a scratch directory, a changed scene setting, a lock. PublishPipeline.run() calls cleanup() once the pipeline is done, in reverse execution order (last action run, first cleaned up), for every action that actually executed. This runs both after a fully successful pass and when a later action raises — in the failure case, only the actions that already ran are cleaned up; the one that raised and anything after it are not. LoadPipeline.run() follows the same rule for its ordered [[load]] steps.

Running a pipeline

PublishPipeline.run(on_phase=None) runs all five phases in order. Pass on_phase — a callable taking the phase name — to be notified right before each phase starts (used by the GUI backend to emit progress signals between phases). Cleanup (see above) always runs once run() returns or raises, so callers don't need to handle it themselves.

Pipeline context

Publish and load pipelines share the same PipelineContext class — there are no separate publish/load contexts. Besides the slot dict it carries entity, task, product_type, dcc, and settings, plus three fields for layered products and loads:

Field Set by Meaning
layer Publish UI layer selector / Session.load_product() Layer name to publish under or load; empty = main layer
version Session.load_product() Version to load; -1 (VERSION_LATEST) = most recent
ref_id Session.load_product() Reference id the loaded scene objects are tagged with

The publish UIs (Blender N-panel and standalone publisher) show a layer selector for every pipeline unless its config sets layers = false. Existing layer names are suggested via Entity.get_product_layers(product_type); typing a new name creates a new layer.

Action discovery

Actions are discovered at runtime via load_custom_actions(). Later entries override earlier ones:

  1. kpipe/actions/ (in packages/core) — built-in actions (loaded on import kpipe)
  2. kpipe_<dcc>/…/actions/ (in packages/<dcc>) — DCC-specific actions (loaded by the extension's register())
  3. {KPIPE_ROOT}/actions/ — project-level custom actions (loaded lazily on first PublishPipeline.from_config())

Configuring a pipeline

Pipelines are defined per product-type and DCC in configs/<bundle>/kpipe-publish.toml:

[pipeline.render_blender_cycles]
dcc = "blender"
product_type = "render_exr"

  [pipeline.render_blender_cycles.settings]
  use_selection = false

  [[pipeline.render_blender_cycles.collect]]
  action = "CollectBlenderObjects"
  slot = "beauty"
  collection_names = ["Render_Beauty"]

  [[pipeline.render_blender_cycles.integrate]]
  action = "IntegrateImageSequence"
  slot = "beauty"