When a business module’s data must appear in the top-bar command-palette global search and support full reconciliation (initialize/rebuild) on the /host/search governance page, three things are required: incremental index-event wiring, an authoritative rebuild source, and host source-catalog registration. This tutorial uses the Tracker module (tracker.item work items) as the canonical reference implementation and walks you from zero to end-to-end verification.
End-to-end data flow
Latency promise of the incremental chain: command commit → Outbox dispatch → CAP consumption → engine deferred batched commit (1000-change threshold or 2-second loop) → SearcherManager refresh on commit completion. Measured on a real environment, new documents are searchable within ~4 seconds.
Step 1: Define the index key and stable business type
The IndexKey is the stable natural key shared by the source, the index partition, and the governance page. Register a constant in GlobalSearchIndexKeys under BitzOrcas.Platform.Search.Contracts:
public static class GlobalSearchIndexKeys{ public const string Tickets = "tickets"; public const string TrackerItems = "tracker.item"; public const string WorkflowTasks = "workflow-tasks"; public const string SupportReports = "support-reports"; public const string SupportIssues = "support-issues"; // New modules append their own stable key here}Also fix the module’s stable business type (BusinessType). It must appear verbatim in three places — the incremental event, rebuild documents, and host configuration — because per-source hit filtering relies on it:
// Constant in the module's projection helper (Tracker example)internal const string EntityType = "TrackerItem";Step 2: Incremental projection (publish index events in the same transaction as the write)
2.1 Write the projection helper
The projection helper is a static class mapping an aggregate snapshot into the field dictionary of a SearchIndexChangedIntegrationEvent. See src/Platform/Tracker/BitzOrcas.Platform.Tracker.Application/Support/TrackerItemSearchIndexProjection.cs:
// <Module>.Application/Support/XxxSearchIndexProjection.cspublic static class XxxSearchIndexProjection{ internal const string EntityType = "XxxItem"; // verbatim match with host BusinessType
/// <summary>Publish a Created/Updated snapshot.</summary> public static Task PublishUpsertAsync( IIntegrationEventPublisher<SearchIndexChangedIntegrationEvent> events, XxxItem item, SearchIndexAction action, DateTimeOffset occurredAt, CancellationToken cancellationToken) { var owners = new[] { item.ReporterId, item.AssigneeId } .Where(owner => !string.IsNullOrWhiteSpace(owner)) .Distinct(StringComparer.Ordinal) .ToArray();
var fields = new Dictionary<string, string?>(StringComparer.Ordinal) { ["Subject"] = item.Subject, // title field: shown on hits ["Description"] = item.Description, // summary/search field // Own DataScope owners: the index-side basis for reporter/assignee visibility [GlobalSearchStoredFields.OwnerUserId] = item.ReporterId, [GlobalSearchStoredFields.OwnerUserIds] = string.Join( GlobalSearchStoredFields.MultiValueSeparator, owners) };
return events.PublishAsync( new SearchIndexChangedIntegrationEvent( GlobalSearchIndexKeys.XxxItems, // source key item.Id, // business id (not the storage id) item.TenantId, // tenant fact comes only from the aggregate EntityType, action, occurredAt.UtcTicks, occurredAt, JsonSerializer.Serialize(fields)), cancellationToken); }}2.2 Wire it into write commands (same transaction is the point)
In every write command handler that changes searchable fields, publish immediately after a successful save. Inject IIntegrationEventPublisher<SearchIndexChangedIntegrationEvent> and call the helper:
// Create command (Tracker example: CreateTrackerItem.cs)var saved = await items.SaveAsync(item, cancellationToken).ConfigureAwait(false);if (saved.IsFailure){ return Result.Failure<TrackerItemSummaryDto>(saved.Error);}
collector.Track(item);// Publish the Search incremental event in the same transaction as the business write,// so the new document becomes globally searchable immediately after creation.await TrackerItemSearchIndexProjection.PublishUpsertAsync( searchIndexEvents, item, SearchIndexAction.Created, clock.UtcNow, cancellationToken);For lifecycle commands prefer a “single seam”: Tracker funnels edit/assign/transition commands into one private PersistAsync method and publishes there once, covering every write path:
// The PersistAsync seam in TrackerItemLifecycle.cs (simplified)private async Task<Result> PersistAsync(TrackerItem item, Result mutation, CancellationToken ct){ if (mutation.IsFailure) return mutation; var saved = await items.SaveAsync(item, ct).ConfigureAwait(false); if (saved.IsFailure) return saved;
collector.Track(item); if (item.IsDeleted) { // Soft delete publishes Deleted: the engine removes the document by storage id await searchIndexEvents.PublishAsync( new SearchIndexChangedIntegrationEvent( GlobalSearchIndexKeys.TrackerItems, item.Id, item.TenantId, TrackerItemSearchIndexProjection.EntityType, SearchIndexAction.Deleted, clock.UtcNow.UtcTicks, clock.UtcNow, "{}"), ct); } else { // Edits (subject/description), assignments (owner fields) all refresh the snapshot here await TrackerItemSearchIndexProjection.PublishUpsertAsync( searchIndexEvents, item, SearchIndexAction.Updated, clock.UtcNow, ct); } return Result.Success();}Step 3: Authoritative rebuild source (governance-page reconciliation)
The governance page matches ISearchIndexRebuildSource by IndexKey; a source without a rebuild source shows “authoritative source not registered” and stays read-only (historically support-reports / support-issues were stuck this way). An architecture test now locks both directions: every configured host source must have a rebuild source.
3.1 Add a dedicated batch read to the read-model port
Do not reuse list-query DTOs (usually missing fields, carrying unrelated projections). Add a method that projects only index fields:
// Contracts: port method (Tracker example: ITrackerItemReadModelStore)Task<Result<IReadOnlyList<TrackerItemSearchIndexRecord>>> ListForSearchIndexAsync( BatchReadRequest batch, CancellationToken cancellationToken);
// Contracts: batch row record (index fields only)public sealed record TrackerItemSearchIndexRecord( string ItemId, string Subject, string Description, string ReporterId, string? AssigneeId);The Infrastructure implementation pages by stable primary key ordering:
public async Task<Result<IReadOnlyList<TrackerItemSearchIndexRecord>>> ListForSearchIndexAsync( BatchReadRequest batch, CancellationToken cancellationToken){ var rows = await items.ListAsync(row => !row.IsDeleted, cancellationToken).ConfigureAwait(false); var page = rows .OrderBy(row => row.Id, StringComparer.Ordinal) .Skip(batch.Offset) .Take(batch.Take) .Select(row => new TrackerItemSearchIndexRecord( row.Id, row.Subject, row.Description, row.ReporterId, row.AssigneeId)) .ToArray(); return Result<IReadOnlyList<TrackerItemSearchIndexRecord>>.Success(page);}Do not forget the fail-closed default adapters (Unavailable*Stores) — add stubs returning StoreUnavailable.
3.2 Implement the rebuild source
// <Module>.Application/Support/XxxSearchIndexRebuildSource.cs[RegisterScopedEnumerable<ISearchIndexRebuildSource>]public sealed class XxxSearchIndexRebuildSource(IXxxReadModelStore readModels) : ISearchIndexRebuildSource{ private const string EntityType = "XxxItem";
public string IndexKey => GlobalSearchIndexKeys.XxxItems;
public async Task<IReadOnlyList<SearchIndexRebuildDocument>> GetBatchAsync( string tenantId, string? lastCursor, int pageSize, CancellationToken ct) { var offset = ParseCursor(lastCursor); var recordsResult = await readModels.ListForSearchIndexAsync( new BatchReadRequest(offset, pageSize), ct).ConfigureAwait(false); if (recordsResult.IsFailure) { // An unavailable read model must fail closed: silently returning an empty batch // would make the governance page believe "0 documents, done" throw new InvalidOperationException( $"Xxx search rebuild source is unavailable: {recordsResult.Error.Code}."); }
return recordsResult.GetValueOrThrow() .Select((record, index) => new SearchIndexRebuildDocument( record.ItemId, EntityType, BuildFields(record), // fields fully isomorphic with the projection! (offset + index + 1).ToString(CultureInfo.InvariantCulture))) .ToArray(); }
private static int ParseCursor(string? cursor) => int.TryParse(cursor, NumberStyles.None, CultureInfo.InvariantCulture, out var v) && v >= 0 ? v : throw new InvalidOperationException("Xxx search rebuild cursor is invalid.");}Step 4: Host source-catalog configuration
Register the source in Search:GlobalSources in src/Hosts/BitzOrcas.Api/appsettings.json (server-owned catalog; clients cannot override it):
{ "IndexKey": "tracker.item", "DisplayName": "工单", "Module": "tickets", "ResourceType": "ticket", "BusinessType": "TrackerItem", "NavigationTemplate": "/tickets/{id}", "TitleField": "Subject", "SubtitleField": "Description", "RequiresAuthoritativeAccessEvaluation": true, "SearchFields": ["Subject", "Description"]}| Field | Meaning | Pitfall |
|---|---|---|
IndexKey | Identical across the GlobalSearchIndexKeys constant and the rebuild source | A typo → governance page shows “authoritative source not registered” |
Module / ResourceType | Source-level authorization resource descriptor (who may search this source) | Use the actual permission-catalog module/resource, not a display name |
BusinessType | Hit-filter key; must match the event/rebuild EntityType verbatim | Mismatch → hits silently dropped by source filtering |
NavigationTemplate | Navigation path; {id} is URL-escaped | Point at a real frontend route |
TitleField / SubtitleField | Read title/subtitle from stored fields | Must exist in the projection field dictionary |
RequiresAuthoritativeAccessEvaluation | true requires an evaluator whose SourceKey matches, else the source fails closed | Missing evaluator → whole source unavailable (fail-closed) |
SearchFields | Field names included in full-text search | Must match projection field names |
Configuration drift is locked by architecture tests (SearchArchitectureTests): configured keys == rebuild-source keys; BusinessType == projection EntityType.
Step 5 (as needed): Authoritative hit evaluator
Index snapshots can go stale (ownership changes, reassignment). When a source needs real-time adjudication against authoritative business state, implement IGlobalSearchHitAccessEvaluator:
[RegisterScopedEnumerable<IGlobalSearchHitAccessEvaluator>]public sealed class XxxGlobalSearchAccessEvaluator( IXxxRepository items, XxxAuthorizationService authorization, ICurrentUser currentUser): IGlobalSearchHitAccessEvaluator{ // Must equal the host-configured IndexKey; the catalog routes re-checks by this key public string SourceKey => GlobalSearchIndexKeys.XxxItems;
public async ValueTask<bool> IsAllowedAsync( GlobalSearchHitAccessContext context, CancellationToken ct) { var user = currentUser.User; if (!string.Equals(context.SourceKey, SourceKey, StringComparison.Ordinal) || !string.Equals(context.TenantId, user.TenantId, StringComparison.Ordinal)) { return false; }
// Re-read authoritative data + unified authorization; never trust index fields var item = await items.FindAsync(context.BusinessId, ct).ConfigureAwait(false); return item.IsSuccess && (await authorization.EnsureCanViewAsync(user, item.GetValueOrThrow(), ct)).IsSuccess; }}Rules: a source with RequiresAuthoritativeAccessEvaluation=true must have an evaluator (missing → fail closed); Own DataScope pre-filters on index owner fields in the handler, and the evaluator is the final judge.
Step 6: Tests and governance locks
- Rebuild-source unit tests: fake the read model; assert key, field isomorphism, cursor advance, empty batch ends iteration, and store failure throws
InvalidOperationException(fail closed). Seetests/BitzOrcas.Unit.Tests/Tracker/SearchIndexRebuildSourceTests.cs. - Configuration-alignment architecture test: maintain bidirectional equality between configured sources and rebuild sources in
SearchArchitectureTests, preventing read-only sources or config drift. - Composition contract tests: if the port has a fail-closed fallback, assert the production registration replaces it (see
SearchPlatformCompositionTests).
Step 7: End-to-end verification
Governance page (/host/search): every source should show ready and be rebuildable; on a fresh environment initialize with ExpectedVersion=0 (requires an approval ticket + confirm token REBUILD INDEX + separation of duties).
Verify the incremental loop from the command line (real-environment procedure):
# 1. Obtain JWT token via /api/auth/cipher-key + RSA-OAEP login# 2. Create a business recordcurl -X POST http://localhost:6881/api/tracker/items \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{"projectId":"PRJ-2026-001","type":"Task","subject":"zephyr wiring check","description":"Verifying end-to-end incremental search index event loop"}'# 3. Wait ~4 seconds (Outbox→CAP→engine commit→refresh); search should hitcurl -X POST http://localhost:6881/api/search/global \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{"Keyword":"zephyr","Take":10}'Troubleshooting quick reference
| Symptom | Root cause | Fix |
|---|---|---|
Governance source unavailable + SearchIndex.Statistics.Unavailable | Composition root resolved the fail-closed engine stub (TryAdd did not override) | Ensure the host Replaces the real engine (see SearchDependencyInjection) |
| Source read-only / “authoritative source not registered” | No matching ISearchIndexRebuildSource | Add one; the architecture test blocks such configuration |
| Newly created records not searchable | Command not wired / BusinessType mismatch / commit not refreshed | Audit Steps 2 and 4 |
| Fresh source search reports unavailable | Index never initialized (no directory, no commits) | Initialize on the governance page; or wait for the first incremental events (uninitialized sources degrade to zero-hit sources instead of failing the whole search) |
Rebuild 403 SearchIndex.Rebuild.ApprovalRequired | Approval ticket missing/not approved | Move an OpsExtension incident to Resolved and rebuild with the ticket id; ticket creator ≠ operator (SoD) |
| Rebuild 409 version conflict | ExpectedVersion does not match the snapshot | Re-read the overview for the current version; use 0 for fresh initialization |
Integration checklist
- Stable key registered in
GlobalSearchIndexKeys(only[A-Za-z0-9_.-], ≤128) - Incremental projection helper + same-transaction wiring in every command that changes indexed fields (including soft-delete
Deleted) - Read-model
ListForSearchIndexAsyncbatch read + fail-closed fallback stub -
ISearchIndexRebuildSource(fields isomorphic with the projection, offset cursor semantics, fail closed) - Host
Search:GlobalSourcesregistration (verify all nine fields;BusinessTypeconsistent in three places) - Implement
IGlobalSearchHitAccessEvaluatoras needed (mandatory whenRequiresAuthoritativeAccessEvaluation=true) - Rebuild-source unit tests + configuration-alignment architecture test
- Governance-page four-state verification + create→4s→hit E2E
Further reading
- Module reference: Search module, index events & rebuild, HTTP & authorization surface
- Prerequisites: Writing a command slice, Adding a standalone module
- In-repo exemplars:
src/Platform/Tracker/**(incremental projections and rebuild sources),src/Platform/Workflow/**(engine-listener style wiring)