AI management and conversation orchestration

AIManage is the platform’s AI connection and conversation-orchestration layer. It persists tenant workspaces, provider credentials, model metadata, user conversations, and messages. Application code depends on Microsoft.Extensions.AI.IChatClient; Infrastructure supplies a Semantic Kernel adapter over an OpenAI-compatible transport. RAG and Agent ports exist, but the primary chat path does not invoke them.

1. Implemented capabilities

• tenant workspace creation, listing, update, optimistic concurrency control, and soft deletion (ManageWorkspace command family);
• provider creation, listing, update, deletion, connectivity probe (TestProviderConnectivityCommand), and Data Protection encryption;
• persisted AIModel aggregates, provider model catalog management, and model CRUD operations;
• user conversation creation, user-scoped listing, archive, deletion, history, message append, and counters;
• non-streaming chat and an NDJSON streaming endpoint with incremental delivery and a typed kind frame protocol;
• conversation-instance ownership checks on Send and Stream (tenant + user bound before any provider call, returning AI.Conversation.NotOwner);
• idempotent requests via IAIMessageRequestCoordinator and a required RequestId (one provider attempt per key, Completed replays without re-calling the provider);
• a cross-module conversation seam IAIConversationPort / AIConversationMediatorPort that routes every call through the Mediator pipeline (authorization, runtime license, logging all apply);
• a 50-message history window and a 32-entry approximate-LRU client cache;
• a tenant skill metadata registry and process-local Markdown skill scanner;
• a Semantic Kernel IChatClient adapter;
• fail-closed RAG and Agent ports conditionally replaced by host composition;
• ORM-neutral stores with SqlSugar/EF Core parity tests;
• Compile-time reflection-free endpoint generation via [GenerateEndpoint] (ADR 0103).

Current boundary and evolution items: Prompt/response sanitization, advanced prompt injection guardrails, token/cost ledger accounting, tool execution approval workflows, and automated RAG pipelines.

2. Runtime shape

Contracts owns aggregates, DTOs, store contracts, the RAG port, permissions, and feature metadata. Application owns use-case orchestration, context construction, client caching, and file scanning. Infrastructure owns persistence and connector bridges.

3. Core HTTP routes

The handwritten group applies authentication and userPolicy rate limiting. Standard routes have request timeouts; the stream disables request timeout. File-skill routes bypass Mediator and therefore bypass ai.skill.view/manage request authorization.

4. Actual non-streaming flow

// RequestId is the client idempotency key; stable across retries.
// ModelId is an unchecked caller override; it need not exist in AIModel.
var result = await mediator.Send(new SendMessage.Command(
    ConversationId: conversationId,
    RequestId: "turn-42",
    Content: "Summarize this week's risks in priority order.",
    ModelId: "gpt-4o-mini"), cancellationToken);

if (result.IsFailure)
    return result.Error;

// The persisted assistant DTO contains no token, provider, model, or cost fields.
return result.Value!;

The handler validates content, loads the conversation, and rejects it if TenantId or UserId does not match the caller (AI.Conversation.NotOwner). It resolves workspace/default provider from the conversation’s tenant, chooses a model, claims the RequestId via IAIMessageRequestCoordinator (rejecting duplicates as AI.Message.RequestInProgress/PreviouslyFailed/RequestConflict, or replaying a Completed turn), persists the user message, builds history, calls the provider, and persists the assistant message. SendMessage.Command implements INonTransactionalCommand, so the claim and message writes run as bounded short transactions around the network call rather than one long transaction.

5. Resource authorization and instance ownership

Send and Stream implement IAuthorizedRequest with ResourceDescriptor(AIManagePermissions.Module, AIManagePermissions.ConversationResource) and AuthorizationAction.Use, so the authorization pipeline evaluates before the handler. Both handlers also inject ICurrentUser and bind TenantId + UserId + ConversationId before any provider call or message write. A mismatch returns AI.Conversation.NotOwner (Forbidden). History retrieves the conversation and compares both TenantId and UserId.

6. Model metadata is not enforced policy

Model selection is request ModelId, workspace DefaultModelId, then hardcoded gpt-4o. Execution always uses MaxOutputTokens=4096 and Temperature=0.7. Persisted AIModel.MaxTokens, Temperature, and ModelType are not loaded, and arbitrary model strings pass through.

Treat AIModel as future governance metadata, not an enforced policy. GA resolution should bind tenant, workspace, provider, and model, reject unregistered values, and record the actual model, parameters, token usage, and policy version.

7. Chapter map

• Workspaces, providers, and models;
• Conversations, messages, and security;
• Streaming, client caching, and usage;
• Skills, RAG, and Agent adapters;
• Testing, operations, and GA gate.

8. GA red lines

Delivered: conversation ownership (tenant + user bound), idempotent turns (RequestId + IAIMessageRequestCoordinator), incremental streaming with typed kind frames, and stable non-leaking stream error contracts. Remaining red lines:

1. Validate an active, tenant-owned workspace before conversation creation.
2. Resolve only managed models and persist usage, cost, and policy evidence.
3. Fail closed on credential-decryption errors; define key-ring persistence, rotation, and migration.
4. Prove concurrent default-provider, model ownership, and message-append behavior.
5. Authorize file-skill operations and secure every future tool invocation independently.
6. Apply classification, redaction, content safety, retention, and injection defenses.
7. Advertise RAG, Agent, or skills only after real wiring and tenant-safe evidence exist.
8. Add provider health, limits, timeouts, circuit breaking, and auditable fallback.
9. Gate releases on HTTP, dual-ORM, concurrency, failure, capacity, recovery, and security tests.

# Inventory the handwritten and generated HTTP surfaces.
rg -n "Map(Get|Post)|GenerateEndpoint\(" \
  src/Hosts/BitzOrcas.Api/Endpoints/AIManageEndpoints.cs src/Platform/AIManage -g '*.cs'

# Keep the current ID-only lookup and fixed execution options visible.
rg -n "c => c.Id == conversationId|MaxOutputTokens = 4096|Temperature = 0.7" \
  src/Platform/AIManage -g '*.cs'

Module catalog · Authorization · Guidance