Search, Conflict Checking, and Indexing

Search composes the Bitzsoft.Integrations.Search ports and Lucene provider with platform authorization and two owner-local configuration tables. It exposes unified search, legal conflict checking, index catalog/statistics, segmentation-word administration, and full rebuild over registered data sources. It is not yet a complete ingestion platform: no business module publishes SearchIndexChangedIntegrationEvent, and the Created/Updated consumer branches are logging-only no-ops.

1. Implemented scope

Implemented:

• a Lucene provider configured through Search:Provider and Search:LuceneBasePath;
• POST /api/search/unified against the current TenantId and caller-selected IndexKey;
• POST /api/search/conflict/check against the fixed conflict index;
• generated catalog, statistics, rebuild, and conflict-record endpoints;
• manual add/delete/list segmentation-word endpoints;
• global SearchIndexCatalog and SysSegmentConfig tables;
• a Lucene/ElasticSearch/OpenSearch manifest where only Lucene is implemented;
• the Deleted branch of a search.index.changed consumer;
• fail-closed unavailable ports in API Shell and explicit production composition.

Not implemented: business document producers, Created/Updated Upsert, event inbox/version gating, conflict-record persistence, owner-resource authorization, StoredFields allowlist/masking, runtime Feature mapping, complete permission alignment, catalog seed, freshness/health metrics, replica consistency, disaster recovery, or end-to-end Search tests.

2. Runtime shape

API composes Search. JobHost references the Infrastructure project but does not call AddBitzOrcasSearchPlatform and has no rebuild executor. The default search-index path is process-relative. Multiple API replicas therefore have independent filesystem views; CAP load balancing does not replicate Lucene documents.

3. Nine HTTP endpoints

The governance catalog declares only search.search.view, search.conflict.search/view, and search.index.manage/view. Conflict check therefore derives search.conflict.search and aligns with the catalog; segmentation writes use search.index.create/delete, drifting from the catalog’s manage/view. See HTTP contracts, permissions, and Feature.

4. First unified query

The application call below searches the current tenant’s conflict index for “Hailan Technology.” The handler puts Keyword into Name, Subject, and Content. It does not validate IndexKey, Skip/Take, or access to each source Case/Document/Ticket. See the endpoint contract for the complete HTTP body.

var result = await mediator.Send(
    new UnifiedSearch.Query(
        IndexKey: "conflict",
        Keyword: "Hailan Technology",
        Skip: 0,
        Take: 20),
    cancellationToken);

// A provider, directory, or query failure is not a valid zero-hit result.
if (result.IsFailure)
    return result.Error;

// Hits are index projections, not source authorization or business truth.
return result.Value!;

Responses contain BusinessId, BusinessType, MatchKeywords, HitLevel, and every StoredFields entry. There is no output-field allowlist or masker.

5. Conflict checking is not ordinary search

The handler correctly ignores ConflictCheckRequest.TenantId and uses the current tenant. It groups Name, ForeignName, FormerName, and CreditCode terms and can exclude the current SourceBusinessId.

The result still means only “the index returned a match.” There is no persisted check, reviewer, rule version, manual confirmation, override, owner authorization, immutable audit evidence, or index watermark. GetConflictRecord always returns ConflictRecord.PendingCasesModule. A Boolean HasConflict is not a complete legal intake workflow.

See Unified search, conflict checking, and result security.

6. Incremental indexing is incomplete

The typed event declares EventId, IndexKey, DocumentId, TenantId, EntityType, Action, SourceVersion, and OccurredAt. It has no [IntegrationTopic], and there is no publisher construction in the repository.

Created/Updated log that IIndexDocumentProvider is missing and still call NotifyIndexChangedAsync; their exceptions are suppressed. Deleted calls ISearchEngine.DeleteAsync and rethrows failures for CAP retry. SourceVersion is forwarded only to the notifier and does not gate index writes.

See Index events, ordering, and rebuild.

7. Provider, catalog, and segmentation

Lucene is the only implemented provider. ElasticSearch/OpenSearch configurations fail startup rather than falling back. External package version 1.0.0-alpha.8 supplies the core and Lucene engine; the platform owns composition, persistence, and API.

The global catalog has no seed producer, GetAll has no stable order, and an unknown status is restored as Active. Segmentation configuration is also global. Writes compare exact strings, while in-memory buckets use ordinal-ignore-case. The store is scoped, so its cache is per instance/request, not cross-request or distributed.

See Lucene provider, catalog, and segmentation.

8. Security boundary

Correct today: current TenantId is used for query/statistics, conflict check rejects the request tenant as authority, and the manual route group requires authentication and user rate limiting.

Missing:

• IndexKey allowlist and per-scenario permission;
• owner-level ACL/DataScope checks for each hit;
• StoredFields schema, masking, and size bounds;
• dedicated audit/retention for conflict-check person data;
• stronger platform scope for global catalog/word changes;
• runtime search.index entitlement;
• producer-side protection against an incorrectly tenant-tagged document.

Tenant filtering cannot repair a TenantId that was wrong at ingestion time.

9. Freshness and recovery

The catalog has DocumentCount, LastRebuiltAt, and Version, but no current platform flow updates them after rebuild. Query DTOs omit index version, as-of, lag, and producer watermark. There is no search health check, corruption detector, disk threshold, commit-lag metric, CAP backlog metric, reconciliation, or shadow-rebuild runbook.

A Lucene directory is not automatically protected by the relational database backup process. Decide whether it is a rebuildable cache or separately backed up. Calling it rebuildable first requires a working source and a successful drill.

10. Current test evidence

Tests presently cover one ListSegmentWords delegation, two in-memory IEntitySet store paths, owner-local metadata/ORM isolation, host composition, provider manifest, and unavailable defaults.

Missing: real Lucene query/Upsert/Delete, event bytes, all action branches, permission suffixes, nine endpoints, request rules, StoredFields security, conflict rules, database-provider parity, corruption/replica/rebuild/recovery, and performance at production scale.

11. GA blockers

1. Define an owner-to-versioned-document production contract.
2. Implement Created/Updated and version/inbox/gap semantics for every action.
3. Add data sources, shadow rebuild, checkpoints, reconciliation, cutover, and rollback.
4. Align .check/.search and .manage/.create/.update/.delete.
5. Evaluate search.index at runtime.
6. Enforce IndexKey allowlists, owner authorization, and StoredFields schemas.
7. Add evidence, review, override, and persistence to conflict checking.
8. Validate global segmentation scope, normalization, concurrency, and invalidation.
9. Define replica topology, persistent volume, writer ownership, and disaster recovery.
10. Add contract, endpoint, provider, recovery, security, and performance suites.

12. Source map

Back to modules · Documents · Tickets · Authorization