Skip to content
bitzorcas
中EN

Tutorial

Tutorial: Integrating a Module with Global Search and Index Governance

Complete walkthrough for wiring a business module into BitzOrcas.Modern global search: same-transaction incremental index events, an authoritative rebuild source, host source-catalog configuration, authoritative hit re-checks, architecture-test locks, and end-to-end verification.

Last updated

When a business module’s data must appear in the top-bar command-palette global search and support full reconciliation (initialize/rebuild) on the /host/search governance page, three things are required: incremental index-event wiring, an authoritative rebuild source, and host source-catalog registration. This tutorial uses the Tracker module (tracker.item work items) as the canonical reference implementation and walks you from zero to end-to-end verification.

End-to-end data flow

ConsumersSearch platform moduleSame-transaction infrastructureBusiness module (e.g. Tracker)same-transaction publishExpectedVersion=0initialize

Write command handlers
(create/edit/assign/delete)

Incremental projection helper
XxxSearchIndexProjection

Authoritative rebuild source
ISearchIndexRebuildSource

Dedicated batch read
ListForSearchIndexAsync

CAP Outbox
(same transaction as the business write)

RabbitMQ

SearchIndexEventHandler
(CAP consumer)

Lucene engine
Upsert / Delete / Rebuild

Immediate SearcherManager
refresh after commit

Global search palette
/api/search/global

Governance page /host/search
approval-gated reconciliation

Latency promise of the incremental chain: command commit → Outbox dispatch → CAP consumption → engine deferred batched commit (1000-change threshold or 2-second loop) → SearcherManager refresh on commit completion. Measured on a real environment, new documents are searchable within ~4 seconds.


Step 1: Define the index key and stable business type

The IndexKey is the stable natural key shared by the source, the index partition, and the governance page. Register a constant in GlobalSearchIndexKeys under BitzOrcas.Platform.Search.Contracts:

src/Platform/Search/BitzOrcas.Platform.Search.Contracts/Search/GlobalSearchContracts.cs
public static class GlobalSearchIndexKeys
{
public const string Tickets = "tickets";
public const string TrackerItems = "tracker.item";
public const string WorkflowTasks = "workflow-tasks";
public const string SupportReports = "support-reports";
public const string SupportIssues = "support-issues";
// New modules append their own stable key here
}

Also fix the module’s stable business type (BusinessType). It must appear verbatim in three places — the incremental event, rebuild documents, and host configuration — because per-source hit filtering relies on it:

// Constant in the module's projection helper (Tracker example)
internal const string EntityType = "TrackerItem";

Step 2: Incremental projection (publish index events in the same transaction as the write)

2.1 Write the projection helper

The projection helper is a static class mapping an aggregate snapshot into the field dictionary of a SearchIndexChangedIntegrationEvent. See src/Platform/Tracker/BitzOrcas.Platform.Tracker.Application/Support/TrackerItemSearchIndexProjection.cs:

// <Module>.Application/Support/XxxSearchIndexProjection.cs
public static class XxxSearchIndexProjection
{
internal const string EntityType = "XxxItem"; // verbatim match with host BusinessType
/// <summary>Publish a Created/Updated snapshot.</summary>
public static Task PublishUpsertAsync(
IIntegrationEventPublisher<SearchIndexChangedIntegrationEvent> events,
XxxItem item,
SearchIndexAction action,
DateTimeOffset occurredAt,
CancellationToken cancellationToken)
{
var owners = new[] { item.ReporterId, item.AssigneeId }
.Where(owner => !string.IsNullOrWhiteSpace(owner))
.Distinct(StringComparer.Ordinal)
.ToArray();
var fields = new Dictionary<string, string?>(StringComparer.Ordinal)
{
["Subject"] = item.Subject, // title field: shown on hits
["Description"] = item.Description, // summary/search field
// Own DataScope owners: the index-side basis for reporter/assignee visibility
[GlobalSearchStoredFields.OwnerUserId] = item.ReporterId,
[GlobalSearchStoredFields.OwnerUserIds] = string.Join(
GlobalSearchStoredFields.MultiValueSeparator, owners)
};
return events.PublishAsync(
new SearchIndexChangedIntegrationEvent(
GlobalSearchIndexKeys.XxxItems, // source key
item.Id, // business id (not the storage id)
item.TenantId, // tenant fact comes only from the aggregate
EntityType,
action,
occurredAt.UtcTicks,
occurredAt,
JsonSerializer.Serialize(fields)),
cancellationToken);
}
}

2.2 Wire it into write commands (same transaction is the point)

In every write command handler that changes searchable fields, publish immediately after a successful save. Inject IIntegrationEventPublisher<SearchIndexChangedIntegrationEvent> and call the helper:

// Create command (Tracker example: CreateTrackerItem.cs)
var saved = await items.SaveAsync(item, cancellationToken).ConfigureAwait(false);
if (saved.IsFailure)
{
return Result.Failure<TrackerItemSummaryDto>(saved.Error);
}
collector.Track(item);
// Publish the Search incremental event in the same transaction as the business write,
// so the new document becomes globally searchable immediately after creation.
await TrackerItemSearchIndexProjection.PublishUpsertAsync(
searchIndexEvents, item, SearchIndexAction.Created, clock.UtcNow, cancellationToken);

For lifecycle commands prefer a “single seam”: Tracker funnels edit/assign/transition commands into one private PersistAsync method and publishes there once, covering every write path:

// The PersistAsync seam in TrackerItemLifecycle.cs (simplified)
private async Task<Result> PersistAsync(TrackerItem item, Result mutation, CancellationToken ct)
{
if (mutation.IsFailure) return mutation;
var saved = await items.SaveAsync(item, ct).ConfigureAwait(false);
if (saved.IsFailure) return saved;
collector.Track(item);
if (item.IsDeleted)
{
// Soft delete publishes Deleted: the engine removes the document by storage id
await searchIndexEvents.PublishAsync(
new SearchIndexChangedIntegrationEvent(
GlobalSearchIndexKeys.TrackerItems, item.Id, item.TenantId,
TrackerItemSearchIndexProjection.EntityType,
SearchIndexAction.Deleted, clock.UtcNow.UtcTicks, clock.UtcNow, "{}"),
ct);
}
else
{
// Edits (subject/description), assignments (owner fields) all refresh the snapshot here
await TrackerItemSearchIndexProjection.PublishUpsertAsync(
searchIndexEvents, item, SearchIndexAction.Updated, clock.UtcNow, ct);
}
return Result.Success();
}

Step 3: Authoritative rebuild source (governance-page reconciliation)

The governance page matches ISearchIndexRebuildSource by IndexKey; a source without a rebuild source shows “authoritative source not registered” and stays read-only (historically support-reports / support-issues were stuck this way). An architecture test now locks both directions: every configured host source must have a rebuild source.

3.1 Add a dedicated batch read to the read-model port

Do not reuse list-query DTOs (usually missing fields, carrying unrelated projections). Add a method that projects only index fields:

// Contracts: port method (Tracker example: ITrackerItemReadModelStore)
Task<Result<IReadOnlyList<TrackerItemSearchIndexRecord>>> ListForSearchIndexAsync(
BatchReadRequest batch, CancellationToken cancellationToken);
// Contracts: batch row record (index fields only)
public sealed record TrackerItemSearchIndexRecord(
string ItemId, string Subject, string Description,
string ReporterId, string? AssigneeId);

The Infrastructure implementation pages by stable primary key ordering:

Infrastructure/ReadModels/XxxReadModelStore.cs
public async Task<Result<IReadOnlyList<TrackerItemSearchIndexRecord>>> ListForSearchIndexAsync(
BatchReadRequest batch, CancellationToken cancellationToken)
{
var rows = await items.ListAsync(row => !row.IsDeleted, cancellationToken).ConfigureAwait(false);
var page = rows
.OrderBy(row => row.Id, StringComparer.Ordinal)
.Skip(batch.Offset)
.Take(batch.Take)
.Select(row => new TrackerItemSearchIndexRecord(
row.Id, row.Subject, row.Description, row.ReporterId, row.AssigneeId))
.ToArray();
return Result<IReadOnlyList<TrackerItemSearchIndexRecord>>.Success(page);
}

Do not forget the fail-closed default adapters (Unavailable*Stores) — add stubs returning StoreUnavailable.

3.2 Implement the rebuild source

// <Module>.Application/Support/XxxSearchIndexRebuildSource.cs
[RegisterScopedEnumerable<ISearchIndexRebuildSource>]
public sealed class XxxSearchIndexRebuildSource(IXxxReadModelStore readModels)
: ISearchIndexRebuildSource
{
private const string EntityType = "XxxItem";
public string IndexKey => GlobalSearchIndexKeys.XxxItems;
public async Task<IReadOnlyList<SearchIndexRebuildDocument>> GetBatchAsync(
string tenantId, string? lastCursor, int pageSize, CancellationToken ct)
{
var offset = ParseCursor(lastCursor);
var recordsResult = await readModels.ListForSearchIndexAsync(
new BatchReadRequest(offset, pageSize), ct).ConfigureAwait(false);
if (recordsResult.IsFailure)
{
// An unavailable read model must fail closed: silently returning an empty batch
// would make the governance page believe "0 documents, done"
throw new InvalidOperationException(
$"Xxx search rebuild source is unavailable: {recordsResult.Error.Code}.");
}
return recordsResult.GetValueOrThrow()
.Select((record, index) => new SearchIndexRebuildDocument(
record.ItemId,
EntityType,
BuildFields(record), // fields fully isomorphic with the projection!
(offset + index + 1).ToString(CultureInfo.InvariantCulture)))
.ToArray();
}
private static int ParseCursor(string? cursor)
=> int.TryParse(cursor, NumberStyles.None, CultureInfo.InvariantCulture, out var v) && v >= 0
? v
: throw new InvalidOperationException("Xxx search rebuild cursor is invalid.");
}

Step 4: Host source-catalog configuration

Register the source in Search:GlobalSources in src/Hosts/BitzOrcas.Api/appsettings.json (server-owned catalog; clients cannot override it):

{
"IndexKey": "tracker.item",
"DisplayName": "工单",
"Module": "tickets",
"ResourceType": "ticket",
"BusinessType": "TrackerItem",
"NavigationTemplate": "/tickets/{id}",
"TitleField": "Subject",
"SubtitleField": "Description",
"RequiresAuthoritativeAccessEvaluation": true,
"SearchFields": ["Subject", "Description"]
}
FieldMeaningPitfall
IndexKeyIdentical across the GlobalSearchIndexKeys constant and the rebuild sourceA typo → governance page shows “authoritative source not registered”
Module / ResourceTypeSource-level authorization resource descriptor (who may search this source)Use the actual permission-catalog module/resource, not a display name
BusinessTypeHit-filter key; must match the event/rebuild EntityType verbatimMismatch → hits silently dropped by source filtering
NavigationTemplateNavigation path; {id} is URL-escapedPoint at a real frontend route
TitleField / SubtitleFieldRead title/subtitle from stored fieldsMust exist in the projection field dictionary
RequiresAuthoritativeAccessEvaluationtrue requires an evaluator whose SourceKey matches, else the source fails closedMissing evaluator → whole source unavailable (fail-closed)
SearchFieldsField names included in full-text searchMust match projection field names

Configuration drift is locked by architecture tests (SearchArchitectureTests): configured keys == rebuild-source keys; BusinessType == projection EntityType.


Step 5 (as needed): Authoritative hit evaluator

Index snapshots can go stale (ownership changes, reassignment). When a source needs real-time adjudication against authoritative business state, implement IGlobalSearchHitAccessEvaluator:

[RegisterScopedEnumerable<IGlobalSearchHitAccessEvaluator>]
public sealed class XxxGlobalSearchAccessEvaluator(
IXxxRepository items,
XxxAuthorizationService authorization,
ICurrentUser currentUser): IGlobalSearchHitAccessEvaluator
{
// Must equal the host-configured IndexKey; the catalog routes re-checks by this key
public string SourceKey => GlobalSearchIndexKeys.XxxItems;
public async ValueTask<bool> IsAllowedAsync(
GlobalSearchHitAccessContext context, CancellationToken ct)
{
var user = currentUser.User;
if (!string.Equals(context.SourceKey, SourceKey, StringComparison.Ordinal)
|| !string.Equals(context.TenantId, user.TenantId, StringComparison.Ordinal))
{
return false;
}
// Re-read authoritative data + unified authorization; never trust index fields
var item = await items.FindAsync(context.BusinessId, ct).ConfigureAwait(false);
return item.IsSuccess
&& (await authorization.EnsureCanViewAsync(user, item.GetValueOrThrow(), ct)).IsSuccess;
}
}

Rules: a source with RequiresAuthoritativeAccessEvaluation=true must have an evaluator (missing → fail closed); Own DataScope pre-filters on index owner fields in the handler, and the evaluator is the final judge.


Step 6: Tests and governance locks

  1. Rebuild-source unit tests: fake the read model; assert key, field isomorphism, cursor advance, empty batch ends iteration, and store failure throws InvalidOperationException (fail closed). See tests/BitzOrcas.Unit.Tests/Tracker/SearchIndexRebuildSourceTests.cs.
  2. Configuration-alignment architecture test: maintain bidirectional equality between configured sources and rebuild sources in SearchArchitectureTests, preventing read-only sources or config drift.
  3. Composition contract tests: if the port has a fail-closed fallback, assert the production registration replaces it (see SearchPlatformCompositionTests).

Step 7: End-to-end verification

Governance page (/host/search): every source should show ready and be rebuildable; on a fresh environment initialize with ExpectedVersion=0 (requires an approval ticket + confirm token REBUILD INDEX + separation of duties).

Verify the incremental loop from the command line (real-environment procedure):

Terminal window
# 1. Obtain JWT token via /api/auth/cipher-key + RSA-OAEP login
# 2. Create a business record
curl -X POST http://localhost:6881/api/tracker/items \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"projectId":"PRJ-2026-001","type":"Task","subject":"zephyr wiring check","description":"Verifying end-to-end incremental search index event loop"}'
# 3. Wait ~4 seconds (Outbox→CAP→engine commit→refresh); search should hit
curl -X POST http://localhost:6881/api/search/global \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"Keyword":"zephyr","Take":10}'

Troubleshooting quick reference

SymptomRoot causeFix
Governance source unavailable + SearchIndex.Statistics.UnavailableComposition root resolved the fail-closed engine stub (TryAdd did not override)Ensure the host Replaces the real engine (see SearchDependencyInjection)
Source read-only / “authoritative source not registered”No matching ISearchIndexRebuildSourceAdd one; the architecture test blocks such configuration
Newly created records not searchableCommand not wired / BusinessType mismatch / commit not refreshedAudit Steps 2 and 4
Fresh source search reports unavailableIndex never initialized (no directory, no commits)Initialize on the governance page; or wait for the first incremental events (uninitialized sources degrade to zero-hit sources instead of failing the whole search)
Rebuild 403 SearchIndex.Rebuild.ApprovalRequiredApproval ticket missing/not approvedMove an OpsExtension incident to Resolved and rebuild with the ticket id; ticket creator ≠ operator (SoD)
Rebuild 409 version conflictExpectedVersion does not match the snapshotRe-read the overview for the current version; use 0 for fresh initialization

Integration checklist

  • Stable key registered in GlobalSearchIndexKeys (only [A-Za-z0-9_.-], ≤128)
  • Incremental projection helper + same-transaction wiring in every command that changes indexed fields (including soft-delete Deleted)
  • Read-model ListForSearchIndexAsync batch read + fail-closed fallback stub
  • ISearchIndexRebuildSource (fields isomorphic with the projection, offset cursor semantics, fail closed)
  • Host Search:GlobalSources registration (verify all nine fields; BusinessType consistent in three places)
  • Implement IGlobalSearchHitAccessEvaluator as needed (mandatory when RequiresAuthoritativeAccessEvaluation=true)
  • Rebuild-source unit tests + configuration-alignment architecture test
  • Governance-page four-state verification + create→4s→hit E2E

Further reading

100%

Scroll or use controls to zoom · drag when enlarged · double-click for 100% / 200%