The repository has no dedicated Tool Connectors tests. Buildable Contracts and Infrastructure do not establish commercial readiness. GA evidence must cover composition, input security, provider contracts, owner workflows, and operational recovery.
1. Test pyramid
Unit tests cannot prove provider behavior; sandbox tests are expensive and unstable. Each layer locks a different risk.
2. Composition contracts
For every connector cover Disabled, invalid value, Enabled without SDK, Enabled with SDK, pre-registered override, and registration order. Assert the selected implementation and behavior, not merely non-null resolution.
Host tests must prove that the API references Infrastructure, registers SDK and adapters, and enforces tenant feature and permissions. None of that evidence exists today.
3. Unavailable contracts
Every asynchronous method should return its stable ProviderUnavailable without network or disk access. OCR CAPTCHA currently must be asserted to throw InvalidOperationException.
Keep error code, HTTP mapping, retryability, and log level stable. Disabled is expected configuration and should not generate an incident storm.
4. Crawler security gate
Automate SSRF address families, redirects, rebinding, schemes/ports, arbitrary-script rejection, output budgets, tenant session isolation, cookie cleanup, timed-out driver retirement, and resource quotas.
Use controlled DNS/HTTP fixtures; CI must not probe a real private network. Pin browser/driver compatibility and test deterministic local pages rather than public sites.
5. OCR security gate
Verify path/link escape, URL SSRF, huge/malformed/Base64 inputs, image bombs, region overflow, cancellation, memory budgets, low confidence, and sensitive-text redaction.
Maintain licensed quality canaries per language and document type. Engine and language-pack upgrades need a diff report.
6. iManage contract gate
In a provider sandbox or trusted emulator, verify upload metadata, stream ownership, size limits, checkout conflict, true null/empty checkin behavior, history timezone, custom fields, and provider error mapping.
Inject “provider completed but client timed out.” Tests must prove reconciliation convergence instead of checking immediate failure only.
7. Independent consumer contracts
Each owner has its own contract suite because the connector has no endpoints:
- tenant/actor comes from trusted context;
- ownership, state, approval, and scan are enforced;
- feature and permission actually reject;
- input policy runs before the port;
- mutation has idempotency and an operation record;
- output redaction, audit, and retention work;
- provider failures map to stable business errors.
This is the essential consumer-contract gate, not one adapter happy-path test.
8. Fault drills
Exercise DNS failure, TLS error, 401/403, 429, slow response, reset connection, partially completed upload, browser crash, OCR memory pressure, secret rotation, and provider-region outage.
Record trigger, expected SLO, alert, user behavior, recovery, and reconciliation. Unknown outcomes require automated and human handling.
// ① The sandbox commits, then the transport drops the response.provider.CompleteThenDropResponse(operationId);
// ② The consumer records UnknownOutcome instead of retrying blindly.Result result = await useCase.ExecuteAsync(command, ct);Assert.Equal(OperationStatus.UnknownOutcome, await operations.GetStatus(operationId));
// ③ Reconciliation reads provider truth and converges the same operation.await reconciler.RunOnceAsync(ct);Assert.Equal(OperationStatus.Succeeded, await operations.GetStatus(operationId));9. Observability contract
Standard trace attributes should include connector, operation, outcome, retryability, provider status category, and correlation ID. Use a controlled low-cardinality partition for tenant; never label with raw URL, filename, or OCR text.
Core SLI: availability, p50/p95/p99 latency, timeout/rate-limit ratio, input rejection, queue wait, active sessions, bytes, unknown outcomes, and reconciliation age.
10. Runbooks
Each connector needs a runbook answering:
- Is failure composition, configuration, or provider-side?
- How is it disabled safely and consumers informed?
- How are credentials and certificates rotated?
- How are queues drained, browsers recycled, or locks released?
- How are unknown outcomes found and reconciled?
- How is SDK/language pack/driver rolled back?
- Which diagnostics are safe to share?
- Who owns upgrades and incidents?
11. Data and compliance
Maintain a data-flow inventory for source, purpose, classification, processing location, third parties, encryption, retention, deletion, subject requests, and cross-border constraints.
Crawler pages, OCR text, and legal documents are potentially high sensitivity. Do not place bodies in application logs, metrics, or traces. Test fixtures also require license and redaction.
12. Commercial GA blockers
| Priority | Blocker |
|---|---|
| P0 | Host not composed; no consumer/endpoint; permission/feature not enforced |
| P0 | Crawler SSRF, script, and cross-tenant session isolation absent |
| P0 | OCR path/URL/size/image safety absent |
| P0 | iManage mutation lacks idempotency, unknown-outcome reconciliation, and audit |
| P1 | Full-memory download, dynamic errors, incomplete exception boundary |
| P1 | Duplicate governance, ordering, health, and metrics gaps |
| P1 | No module, consumer, sandbox, or fault-drill tests |
13. GA evidence pack
Attach source/package versions, SBOM, configuration matrix, authorization/feature results, threat model, data flow, test report, capacity results, fault drill, provider compatibility, runbooks, dashboards, rollback, and known limitations.
Without evidence, label the capability Preview or Not delivered. An unavailable adapter is a safe default, not provider readiness.
14. Current executable checks
# ① Compile the vendor-free contract boundary.dotnet build src/Platform/ToolConnectors/BitzOrcas.Platform.ToolConnectors.Contracts/\BitzOrcas.Platform.ToolConnectors.Contracts.csproj --no-restore
# ② Compile all three vendor adapters.dotnet build src/Platform/ToolConnectors/BitzOrcas.Platform.ToolConnectors.Infrastructure/\BitzOrcas.Platform.ToolConnectors.Infrastructure.csproj --no-restore
# ③ Current source should produce no dedicated test matches.rg -n "CrawlingAdapter|OcrAdapter|IManageAdapter|UnavailableOcrPort" tests -g '*.cs'15. Definition of done
A connector reaches GA only when host composition, owner use case, input policy, authorization/feature, provider contract, idempotency/reconciliation, observability, runbook, and independent tests pass together. Rate Crawler, OCR, and iManage separately.