Deployment, Versioning, and Rollback

Workflow separates definition content from the binding that selects a definition for new instances. Deploy creates a snapshot. Publish, StartGrayScale, CompleteGrayScale, and Rollback mutate WorkflowDeployment. Running instances retain their DefinitionId.

1. Model

The same key, tenant, and checksum returns the existing version. A new version is max+1. Concurrent deploy still needs unique-constraint evidence because checksum reuse does not serialize different content.

2. Deploy

Result<WorkflowDefinition> deployed =
    await repository.DeployAsync(
        "matter-intake-approval",
        "Matter intake approval v2",
        json,
        currentUserId,
        currentTenantId,
        cancellationToken);

// Propagate the stable error before dereferencing Value.
if (deployed.IsFailure)
    return deployed.Error;

// Publication must use the immutable identifier returned by this deployment.
string definitionId = deployed.Value.DefinitionId!;

Deploy parses but does not run complete graph validation. A release service should enforce ValidateDefinitionAsync before Deploy.

3. Selection hierarchy

New-instance deployment resolution is three-tier:

1. exact tenant and office;
2. tenant and ALL;
3. PLATFORM and ALL.

There is no hidden fourth fallback to “the key’s latest definition.” If no tier matches, the start command fails — publication is therefore a real gate: a definition with no published binding can never serve a new instance.

4. Normal publish

First publish creates Active. Subsequent publish moves the old Active to Previous and sets the new DefinitionId. The corresponding deployment cache is invalidated.

// definitionId must refer to a validated immutable version.
Result published = await repository.PublishDeploymentAsync(
    "matter-intake-approval",
    currentTenantId,
    officeId: null, // normalized to ALL
    definitionId,
    currentUserId,
    cancellationToken);

// Success changes new starts only; it does not migrate running instances.

Publish validates that the definition exists and the key matches, but does not explicitly check definition tenant against binding tenant.

5. Actual grayscale semantics

StartGrayScale requires a binding, moves current Active to Previous, sets newDefinitionId as Active, and marks status GrayScale. Every subsequent instance selects the new Active. No percentage, stable bucket, allowlist, office subset, or time window exists.

StartGrayScale also lacks Publish’s explicit definition existence and key checks.

6. Complete and rollback

CompleteGrayScale only changes status to Active. Rollback restores Previous to Active and clears Previous. Existing v2 instances are unaffected.

There is only one Previous slot, not a rollback stack.

7. Publish races

Deployment writes have no expectedVersion or compare-and-swap. Concurrent administrators can both read the same Active and overwrite each other, producing an unexpected Previous. Version allocation is also read-max-plus-one. Add row versioning, uniqueness, transaction, and a stable conflict response.

8. Cache consistency

Repository invalidates the binding after writes. FusionCache propagates through Redis backplane only when Redis services are registered. With L1 only, other API instances can keep the old binding until expiry.

Test multi-node publish and rollback, backplane outage, maximum stale window, cache-error response semantics, and agreement on selected DefinitionId.

9. Release runbook

1. Export current Active, Previous, status, and checksums.
2. Validate and simulate the candidate under a variable matrix.
3. Deploy and read back DefinitionId/version/checksum.
4. Publish to an isolated tenant or office and create a probe instance.
5. Inspect tasks, notifications, timers, timeline, reports, and cache.
6. Expand scope; there is no percentage canary today.
7. Monitor instance count and failure by DefinitionId.
8. Roll back on threshold and decide how to handle new-version in-flight instances.

10. Compatibility and deletion

Old definitions must remain while instances reference them. Management.DeleteDefinitionAsync currently deletes an existing definition without checking active instances, Active/Previous bindings, or cache. Restrict this endpoint until referential protection is implemented.

Node IDs and variable types are versioned contracts. Explicit migration requires mapping and rollback.

11. Source checks

rg -n "DeployAsync|PublishDeploymentAsync|StartGrayScaleAsync|RollbackDeploymentAsync" src/Framework/BitzOrcas.Workflow -g '*.cs'

rg -n "DeleteDefinitionAsync|ActiveDefinitionId|PreviousDefinitionId|DefinitionId" src/Framework/BitzOrcas.Workflow src/Platform/Workflow -g '*.cs'

Next: running instances