Operations Extension

Owns incident records, operational metrics, alert/SLA read models, and guarded operations ports for Webhook/CAP recovery, configuration reload, and host-service restart.

Primary path at a glance

The coral node marks the module’s central decision or state boundary.

Capability boundary

• Incident severity/status records and JSON note snapshots without a guarded state machine
• Tenant Webhook dead-letter views and global CAP failure operations through owner ports
• Current-process metrics plus guarded configuration-reload and service-restart contracts

The module explicitly does not own the following: Incident, metric, alert, and SLA persistence is owner-local. Webhooks, CAP, configuration providers, and service orchestration stay with their owners or adapters; a fail-closed port alone does not prove an operation is reachable.

Code map

Source location: src/Platform/OpsExtension

Core use cases

• CreateIncident
• UpdateIncidentStatus
• RetryDeadLetter
• RetryCapFailedMessage
• ReloadConfig
• RestartServiceCommand
• GetRuntimeMetrics
• GetObservabilitySeriesQuery

When reading a use case, start with its request contract and permission declaration, follow how the handler coordinates an aggregate or port, then inspect post-commit events, cache changes, and audit.

Key models and contracts

Use-case boundary example

Pseudocode

var tenantId = currentUser.User.TenantId;
var actorKey = currentUser.User.ActorUserId;
var deliveryId = command.DeliveryId.Trim();
// ① Sensitive operations pass the approval gate first; a bad ApprovalTicket is denied.
if (!await approvalGate.ValidateAsync(command.ApprovalTicket, cancellationToken))
    return Result.Failure(OpsExtensionErrors.DeliveryFailureApprovalRequired);
// ② Record the intent audit before acting; an unwritable audit fails closed entirely.
var intentPersisted = await DeliveryFailureAudit.TryRecordAsync(activityAuditSink,
    new ActivityRecord("DeliveryFailure.Webhook.Retry.Intent", actorKey, tenantId,
        true, $"delivery={deliveryId}", command.ApprovalTicket, clock.UtcNow),
    logger, cancellationToken);
if (!intentPersisted) return Result.Failure(OpsExtensionErrors.DeliveryFailureAuditUnavailable);
// ③ Only after the intent persists does the real retry run through the Webhooks port.
var retry = await webhookRetry.RetryAsync(tenantId, deliveryId, cancellationToken);
var outcome = retry.IsSuccess ? retry.GetValueOrThrow().Status : retry.Error.Code;
// ④ Completion audit uses CancellationToken.None: the outcome must land even if aborted.
var completionPersisted = await DeliveryFailureAudit.TryRecordAsync(activityAuditSink,
    new ActivityRecord("DeliveryFailure.Webhook.Retry.Completed", actorKey, tenantId,
        retry.IsSuccess, $"delivery={deliveryId},outcome={outcome}",
        command.ApprovalTicket, clock.UtcNow),
    logger, CancellationToken.None);
// ⑤ A failed retry returns its own error first; missing completion audit surfaces as
// DeliveryFailureOutcomeUnknown instead of pretending success or failure.
if (retry.IsFailure) return Result.Failure(retry.Error);
if (!completionPersisted) return Result.Failure(OpsExtensionErrors.DeliveryFailureOutcomeUnknown);
return Result.Success(retry.GetValueOrThrow());

Integration relationships

Webhook and CAP have concrete adapters. Configuration reload is conditional on AgileConfig, and service restart delegates to an external orchestrator instead of killing the API process. Action ports fail closed when unavailable.

Cross-module collaboration follows these directions:

• Callers depend on this module’s public Contracts or a narrow port.
• State changes propagate through versioned integration events.
• Database, messaging, cache, file, and vendor SDK details stay in Infrastructure.
• The composition root registers fail-closed defaults before production adapters.

Security, tenancy, and privacy

The permission catalog now aligns Read/View/Manage suffixes with authorization actions and is collected by the Governance Generator. Two boundaries remain: the default-off operations.extension declaration has no runtime feature gate in these use cases, and Incident Resolved/Closed alone is insufficient approval evidence for sensitive Operations actions.

Every tenant-scoped write persists TenantId, and list/detail paths share one authorization boundary. Logs, events, and audit retain only the minimum fields needed for diagnosis.

Failure semantics, idempotency, and observability

Webhook retry re-enters owner delivery logic, CAP retry conditionally resets infrastructure rows, and audit failure can currently replace an already-completed action result.

Test matrix

Keep one universal red line under test: no identifier, cache key, event, or query condition from tenant A may let tenant B read or mutate data.

Source navigation and change checks

Run from the BitzOrcasVNext repository root:

# List public types and confirm that new contracts belong to the module boundary.
rg -n "^public (sealed |abstract |static |partial )*(record|class|interface|enum)" src/Platform/OpsExtension -g '*.cs'

# Inspect cross-layer references; callers must not depend on another module's Infrastructure.
rg -n "ProjectReference|PackageReference" src/Platform/OpsExtension -g '*.csproj'

# Find unfinished or elided implementation; expect no matches.
rg -n "TODO|FIXME|// \.\.\." src/Platform/OpsExtension -g '*.cs'

Extension and release checklist

1. Is the new capability expressed through explicit Contracts, permissions, and a feature key?
2. Can persistence, connectors, and providers be replaced through narrow adapters?
3. Does production still resolve a Null/Unavailable default, and could health checks report a false green?
4. Is there evidence for transaction, event, cache-invalidation, audit, and background-job failure/retry behavior?
5. Did bilingual docs, diagrams, contract tests, and runbooks change with the code?

Back to module catalog · Module dependency diagram · Add a module guide