Operations Control Plane

Operations is the platform’s operational application orchestration layer. It aggregates the module registry, tenant store, background-job catalog, configuration declarations, and Framework operations ports. It also defines schema, backup, restore, and archive commands. It is no longer the read-only console described by the early knowledge base.

1. Architectural position

The module itself has Contracts and Application projects only. Database migration, backup, and archive implementations live in Framework Infrastructure. Approval comes from the API Host and OpsExtension. Shared Framework and Host composition provide job execution.

2. Current capability surface

3. Code map

Having no module-local Infrastructure project does not mean there is no infrastructure behavior. Operations delegates it to shared adapters.

4. Read the HTTP surface in two groups

OperationsEndpointGroup hand-maps 9 routes for job management, archive, and cache governance. The group applies authentication, the userPolicy rate limiter, and exact permissions.

Application declares another 16 [GenerateEndpoint] requests, covering governance, tenants, adapters, config, external connector readiness, and schema/backup routes. EnumerateAllTypes in the generator does not recurse into nested types, so an attribute is not proof of a runtime endpoint.

# Compare handwritten mappings with endpoint declarations.
rg -n "Map(Get|Post|Put)|GenerateEndpoint" \
  src/Hosts/BitzOrcas.Api/Endpoints/OperationsEndpointGroup.cs \
  src/Platform/Operations -g '*.cs'

# The current generator has no nested GetTypeMembers traversal.
rg -n "EnumerateAllTypes|GetTypeMembers" \
  src/Framework/BitzOrcas.Endpoint.SourceGenerator/GenerateEndpointSourceGenerator.cs

5. Authorization model

OperationsPermissions declares 19 permissions, including the two cache-governance permissions (operations.cache.view / operations.cache.manage). Two website analytics dead-letter permissions are owned by Operations even though their routes live in the Website Host group.

Handwritten endpoints use RequirePermission. Requests also implement IAuthorizedRequest, sending a resource and action through the Mediator authorization pipeline. New routes must prove the two expressions stay aligned.

6. Approval model

The API Host’s ProductionSensitiveOperationApprovalPort sets IsApprovalRequired=true in Production and Staging. An OpsExtension incident in Resolved or Closed status counts as approved.

ValidateAsync immediately succeeds in other environments. Calling it unconditionally does not mean approval is mandatory everywhere. In a required environment without an IOpsExtensionStore, a non-empty ticket degrades to success.

7. Guarantees and non-guarantees

8. Reading paths

• Build an operations dashboard with governance, adapters, config, and tenants.
• Plan database changes with schema drift and migration safety.
• Manage schedulers with background-job runtime control.
• Plan disaster recovery and retention with backup, restore, and archive.
• Make a release decision with security, testing, operations, and commercial GA.
• Invalidate cache or inspect connector wiring: see cache and connector governance below.
• Incidents, CAP failures, webhook dead letters, and config reload belong to Ops Extension.

9. Cache governance and external connector readiness

Both capabilities revolve around read-only projection plus controlled invalidation, but they differ in source and guarantees. Do not conflate them.

9.1 Cache governance

Cache governance gives platform operators a narrow, Host-only surface: query area health and fingerprints, invalidate by area, and run fill-missing or full rebuild on warmup-capable areas. Internal tags / Redis keys never enter the HTTP contract. Cross-cutting rules: Caching guide · area catalog.

Capability port ICacheGovernancePort (Framework) declares the provider: ProviderMode (LocalMemory / Distributed), SupportsTagInvalidation, SupportsCrossInstanceInvalidation. FusionCacheStore supports tags and cross-instance invalidation; MemoryCacheStore sets SupportsTagInvalidation to false.

Runtime source of truth ICacheAreaCatalog (DefaultCacheAreaCatalog); Operations keeps CacheGovernanceCatalog aligned with public keys. Current 11 areas:

Three Host-only operations (CallerType: Host, TenantId == "0"):

Rebuild body fields: Area, Mode, TenantIds?, ApprovalTicket, Reason, Confirm, IdempotencyKey. Lazy-only areas return Operations.Cache.WarmupUnsupported.

Shared guard chain (invalidate and rebuild): Host identity → area catalog → provider capability (LocalMemory → ProviderUnsupported) → approval ticket → reason and confirm token → idempotency key → two-phase activity audit (audit write failure is fail-closed).

Optional startup Cache:Warmup (default EnabledOnStartup=false): production configuration and the caching guide.

9.2 External connector readiness

GET /api/operations/connectors returns the configuration and registration state of 8 external connectors (resource operations/adapters, action View). Report fields: Group, Connector, Provider, Status (Configured/NotConfigured/Unavailable), AdapterRegistered, AdapterImplementation, ConfigurationSection, ChangesRequireRestart (always true), DependentCapabilities.

This report is not the same as the adapter report in §6. The adapter report classifies about 35 internal ports by implementation-class-name prefix (Production/Default/Unknown), answering “what can the container resolve.” Connector readiness answers “does this external system’s config section exist and is its adapter registered,” and when a provider SDK is missing it surfaces as a fail-closed Unavailable proxy. The two should be presented separately.

The report reads only configuration and registration state; it does not issue a real connectivity probe. ChangesRequireRestart is always true, meaning connector config changes require a process restart to take effect.

10. Minimal query example

// ① The request declares operations/governance + View for the authorization pipeline.
var query = new GetGovernanceReport.Query();

// ② The handler builds a snapshot from the current AppModuleRegistry.
Result<ModuleGovernanceReport> result =
    await mediator.Send(query, cancellationToken);

// ③ MermaidGraph is presentation data; architecture tests remain the release gate.
if (result.IsSuccess && result.Value!.HasCircularDependencies)
    release.Block("The module dependency graph contains a cycle");

11. Source inspection

# Module-local projects and use cases.
find src/Platform/Operations -maxdepth 3 -type f -name '*.cs' \
  -not -path '*/bin/*' -not -path '*/obj/*' | sort

# Permissions, resource actions, and HTTP surface.
rg -n "PermissionDefinition|ResourceDescriptor|GenerateEndpoint|RequirePermission" \
  src/Platform/Operations src/Hosts/BitzOrcas.Api/Endpoints/OperationsEndpointGroup.cs \
  -g '*.cs' --glob '!**/bin/**' --glob '!**/obj/**'

# Cache governance catalog, rebuild, and Host identity rule.
rg -n "CacheGovernanceCatalog|RebuildCacheArea|ConfirmToken|CacheGovernanceRules|ICacheAreaCatalog" \
  src/Platform/Operations src/Framework -g '*.cs'

# External connector readiness report.
rg -n "GetExternalConnectorReadiness|ExternalConnectorReadinessEntry" \
  src/Platform/Operations -g '*.cs'

# Old documentation's plan and lease types should have no implementation hit.
rg -n "PlanId|PlanHash|DistributedLock|MigrationPlanStore" \
  src/Platform/Operations -g '*.cs'

Back to platform modules · Ops Extension