Guidance streaming, testing, operations, and commercial GA gate

Guidance has a working content store, assistant adapter, and dual-ORM foundation. Commercial GA must prove that Drafts do not leak, editing does not remove the live guide, stale prompts can be revoked, cross-module session creation recovers, and NDJSON behaves consistently under errors, cancellation, and timeout.

1. Normal NDJSON stream

The handwritten endpoint receives an IAsyncEnumerable<string> from the mediator and wraps it in NdjsonTextStreamResult, which serializes each chunk as a typed {"kind":"data","text":"..."} frame, flushes every line, and finishes with {"kind":"done"}. A failure produces exactly one terminal {"kind":"error",...} frame. It sets:

• application/x-ndjson; charset=utf-8;
• Cache-Control: no-cache;
• X-Accel-Buffering: no;
• userPolicy;
• DisableRequestTimeout.

POST /api/v1/guidance/sessions/guidance-100-hash/messages/stream
Content-Type: application/json
Accept: application/x-ndjson
Authorization: Bearer <token>

{
  "content": "How do I filter unissued invoices?",
  "modelId": "managed-model-id"
}

2. Inconsistent error frames

Blank messages, disabled assistant, and missing sessions become [ERROR: description] strings, then the host wraps them as ordinary text frames. An unhandled host exception instead writes {"error":"Guidance stream response failed."}. Only normal completion writes done.

The client must parse a text prefix to find application errors. There is no stable code, retryable flag, correlation, or guaranteed terminal frame. GA needs a versioned union protocol.

{"type":"meta","version":1,"correlationId":"corr-1"}
{"type":"delta","sequence":1,"text":"Open the filter panel first."}
{"type":"error","code":"Guidance.Assistant.SessionNotFound","retryable":false}
{"type":"done","finishReason":"error"}

This is a target protocol, not current output.

3. Cancellation and timeout

The host sends RequestAborted to the query and enumerates with WithCancellation. OperationCanceledException represents client disconnect. The adapter forwards the token to AIManage’s stream handler.

The endpoint disables the standard timeout without replacement maximum duration, idle timeout, token/byte/chunk limit, or tenant concurrency bound. A stalled provider can hold connection and billing resources indefinitely.

4. Current automated evidence

Missing: real HTTP authorization and platform Draft leakage, publish concurrency, successful/racing/compensated session creation, stale-prompt revocation, AIManage owner policy, real NDJSON failures, provider faults, capacity, and recovery.

5. Minimum regression set

# Reuse one expression so all three evidence layers target the same module.
GUIDANCE_FILTER="FullyQualifiedName~Guidance"

# Guidance application, store, and adapter tests.
dotnet test tests/BitzOrcas.Application.Tests/BitzOrcas.Application.Tests.csproj \
  --filter "$GUIDANCE_FILTER"

# Module boundary, route, and persistence shape.
dotnet test tests/BitzOrcas.Architecture.Tests/BitzOrcas.Architecture.Tests.csproj \
  --filter "$GUIDANCE_FILTER"

# Dual-ORM content and session parity.
dotnet test tests/BitzOrcas.Integration.Tests/BitzOrcas.Integration.Tests.csproj \
  --filter "$GUIDANCE_FILTER"

Release evidence also needs the real API host and each supported database, not only in-memory EntitySet and substitutes.

6. Content HTTP matrix

Cover anonymous, published reader, Draft administrator, platform operator, cross-tenant ID, role mismatch, platform Draft, invalid language, repeated publish, concurrent revise/publish, and delete races. Query the database after every denial to prove zero side effects.

var published = await api.GetContextualAsync(route, cancellationToken);
await admin.ReviseDraftAsync(
    published.ContentId, newBody, published.ETag, cancellationToken);

// End users retain the old PublishedRevision while the new Draft is edited.
var stillLive = await api.GetContextualAsync(route, cancellationToken);
stillLive.RevisionId.ShouldBe(published.RevisionId);
stillLive.BodyMarkdown.ShouldBe(published.BodyMarkdown);

Current DTOs have no ETag/RevisionId; this is a GA target test.

7. Session fault injection

Inject at FindSession, CreateConversation, Session Add, unique commit, initial Send, and before response. Assert no orphan—or recoverable orphan—stable replay for the same idempotency key, and no duplicate conversations under concurrency.

var request = NewStartRequest(idempotencyKey: "guide-session-2026-42");

// Cross AIManage and Guidance persistence boundaries concurrently.
var results = await Task.WhenAll(
    client.StartAsync(request, cancellationToken),
    client.StartAsync(request, cancellationToken));

results[0].SessionId.ShouldBe(results[1].SessionId);
await evidence.AssertSingleConversationAndSessionAsync(request, cancellationToken);

8. Prompt and authorization regression

Start with a restricted guide, then revoke the role, revise/delete the guide, disable the feature, or change workspace. The next Send/Stream must reject or refresh. Testing only a new Start misses the stale-session risk.

Verify that prompts omit raw TenantId/UserId and logs omit the body. A Secret/PII detector applies policy before provider transmission.

9. NDJSON contract tests

Use real HTTP streaming for meta/delta/done, validation error, session not found, provider error, host exception, cancellation, proxy buffering, slow consumer, disconnect, and multibyte boundaries. Incremental readers split by lines; one network chunk is not one JSON frame.

Each stream has one done terminal. Error is followed by done(error) or an explicitly documented close. Cancellation never emits false success.

10. Markdown security tests

Cover raw HTML, script, javascript/data URLs, event attributes, SVG, tracking pixels, oversized images, Unicode confusion, nested lists, and code blocks. Administration preview and end-user display use the same sanitizer and browser tests prove no XSS.

Test prompt injection, secrets, and PII separately when the body enters AI. Browser sanitization does not protect the model path.

11. Capacity baselines

Measure 100/10,000/1,000,000 guides per tenant, 1/10/100 duplicate candidates, 1KB/100KB/1MB bodies, 1/100/1,000 concurrent streams, 16K prompts, long responses, and slow consumers. Capture P50/P95/P99, database scans, first byte, tokens, bytes, connection duration, memory, and cost.

FindBestMatch currently loads candidates into memory and ranks them. Duplicate natural keys amplify the cost. Evaluate database rank selection or a precomputed Published projection.

12. Metrics and alerts

• content create/revise/publish/delete/conflict;
• contextual hit/miss by rank, language, audience;
• platform-Draft access denial;
• session start/reuse/refresh/expire/orphan;
• conversation/session compensation;
• prompt characters/policy denial, never body;
• stream first-byte/duration/chunks/bytes/cancel/error;
• token/cost/quota/provider failure;
• outbox/event lag and cleanup backlog.

Alert on Draft leakage, no live revision after edit, orphan conversation, stale prompt use, stream without terminal, and timeout/cost spikes.

13. Backup and recovery

Back up current/history/published pointers, session mappings, referenced AIManage conversations, and Outbox. Verify platform-null tenant semantics, audience, published pointer, soft delete, session owner, prompt revision, and orphan mappings after restore.

If AIManage and Guidance restore to different points, reconciliation closes sessions with missing conversations and archives Guidance conversations with no session according to policy.

14. Release blockers

1. Platform Drafts can leak through Detail/List.
2. Same-row revise removes live Published and loses history.
3. Natural key, publication uniqueness, and concurrency controls are weak.
4. Markdown rendering-security contract is missing.
5. Assistant lacks separate permission/feature/quota and user may become "0".
6. Prompt exports raw identity/roles without sensitive-data policy.
7. Reused sessions do not refresh guide/role/permission state.
8. Conversation/session creation lacks idempotency, atomicity, or compensation evidence.
9. Direct AIManage handler invocation may bypass owner-pipeline policy.
10. NDJSON errors are not stable typed frames.
11. DisableRequestTimeout has no resource-bound replacement.
12. HTTP, concurrency, fault, privacy, capacity, and recovery evidence is incomplete.

15. Final sweeps

# P0 should remove or register every remaining compatibility use.
rg -n "UserId\?\.ToString.*\?\? \"0\"|DisableRequestTimeout|\[ERROR:" \
  src/Platform/Guidance src/Hosts/BitzOrcas.Api/Endpoints/GuidanceEndpointGroup.cs -g '*.cs'

# Target publication and session-governance capabilities.
rg -n "PublishedRevision|IdempotencyKey|PromptHash|ExpiresAt|AssistantUse" \
  src/Platform/Guidance -g '*.cs'

Guidance overview · Content lifecycle · Assistant security