Auditing capture, normalization, and redaction

Audit evidence is trustworthy only when its capture point, tenant/actor source, and collected data are explicit. BitzOrcas first creates a typed record for each producer and then normalizes it into the storage-neutral AuditLogEntry; it does not pretend that every producer owns the same fields.

1. Producers and capture timing

One business operation can legitimately produce HTTP, Mediator, entity-change, and authorization evidence. Those records answer different questions and should not be collapsed into one row merely because they share a correlation ID.

2. HTTP request audit

RequestAuditMiddleware runs after tenant resolution and before authorization. Before calling downstream code it enriches the OTel Activity with tenant, caller type, correlation ID, and enduser.id. In finally it records:

• HTTP method and the path classified by RequestEndpointClassifier;
• final status and elapsed time;
• ActorKey, effective tenant, and correlation ID;
• a 500 outcome for thrown exceptions while preserving the original exception.

It does not read request/response bodies or persist query values and headers. It still attempts enqueue with CancellationToken.None after a client disconnect. An audit-write failure becomes a diagnostic warning and does not rewrite the business response.

3. Mediator activity

The activity behavior evaluates IResult only after a normal handler return: success records success; business failure records the stable error code. The DI Source Generator projects [AuditIgnore] into MessageAuditPolicyRegistry, so runtime code does not reflect over attributes. An unregistered message is audited by default.

Three boundaries matter:

• a thrown handler has no activity row and relies on HTTP/exception evidence;
• sink failure is contained and cannot roll back completed business work;
• TenantId remains the authenticated owner’s tenant. Delegation records the effective subject separately but does not transfer management ownership.

C# type names change during refactoring. Compliance reports that require a stable action should emit an explicit business activity rather than treating a Command class name as a permanent contract.

4. EF Core entity changes

The EF interceptor snapshots mapped scalar properties before SaveChanges, then holds records in a transaction buffer. It publishes only after the outer transaction commits. Failed, cancelled, or rolled-back saves discard evidence and restore any pre-incremented concurrency version.

The snapshot deliberately minimizes exposure:

• Insert captures mapped new values, Update captures before/after values only for modified properties, and Delete captures original values;
• encrypted columns, names containing password/token/phone/email/address and related fragments, byte arrays, and complex values become ***;
• strings are redacted by default; only structural values such as identifiers, tenant, status, type, version, currency, and locale remain clear;
• tenant comes from the entity when available, and the actor must resolve to a stable ActorKey or the path fails closed.

The CAP-aware unit of work places the snapshot batch in the transaction’s Outbox. The non-CAP EF unit of work delivers to the in-process sink after database commit: it no longer records rolled-back changes, but a post-commit/pre-enqueue loss window remains.

5. SqlSugar entity changes

SqlSugar AOP cannot provide a reliable property-level before/after snapshot. The current implementation therefore parses only INSERT, UPDATE, DELETE, and the target table from the beginning of executed SQL:

• SQL text, parameters, OldValues, and NewValues are not persisted;
• EntityId is unavailable; the UI must not present this as a field diff;
• audit, CAP, migration, and Quartz infrastructure tables are excluded;
• an explicit transaction uses an AsyncLocal buffer that is discarded on rollback;
• the CAP SqlSugar unit of work publishes the buffered batch to its transactional Outbox before commit.

Outside an explicit transaction, the synchronous AOP callback requires ISynchronousEntityChangeAuditSink. A full or closed queue fails explicitly rather than dropping the record silently.

AuditIgnoreEntityChangeAttribute is declared but neither ORM capture path consumes it. Schema and seed paths use the explicit AuditEmissionScope; application code must not assume the attribute currently suppresses evidence.

6. Authorization, exceptions, and alerts

Authorization evidence includes resource, action, caller type, client ID, reason, trace, and decision. SqlSugar stores Security and Exception in SysSpecialLog, but writes distinct Level values and preserves that distinction in query and retention; exceptions are no longer projected as Security.

Exception messages, reasons, and payload-like text still pass through the common redaction and length bounds. Fatal/Error can trigger an audit alert. The alert is an independent notification, not a recoverable copy of the audit record, and an alert failure cannot replace the original exception.

7. Outbound HttpClient

ConfigureHttpClientDefaults adds ExternalRequestLoggingHandler to clients built by IHttpClientFactory. The handler deliberately does not read request or response bodies and never rebuilds streams for logging. It records only:

• method, status, duration, and correlation ID;
• a URL without user info, query, or fragment, bounded to 2,048 characters;
• effective tenant and current actor;
• transport failure as status 0 with a stable error summary.

The outbound call is already irreversible when audit completion begins. Audit enqueue therefore has an independent five-second bound and contains its failure, preventing prolonged backpressure from occupying the HTTP connection or turning a successful third-party call into a false business failure.

8. CAP consumers and background jobs

The CAP observability filter uses MessageName as topic and ExecutionInstanceId or MessageId to measure duration. Both success and failure emit CapConsumerRecord; failures retain only the exception type, not arbitrary exception text. Tracing and audit are attempted independently, and neither observability failure changes the CAP outcome already decided.

Jobs use a separate BackgroundJobRecord. SqlSugar co-locates both categories in SysCommunicationLog with a category marker in RunPars; Mongo uses separate collections. Query and retention honor the same distinction.

9. Common normalization boundary

AuditBatchIntegrity.NormalizeForPersistence is the final shared gate for SqlSugar and Mongo:

• TenantId, AuditId, actor, correlation, and other identifiers must be canonical and bounded;
• event time must fit the shared SQL Server range and duration cannot be negative;
• path, module, action, and error text have separate limits;
• payload-like text is capped at 16,384 characters, redacted within a bounded scan, and marked with …[truncated] when shortened;
• JSON password/secret/token/api-key/cookie values, Bearer tokens, form/connection secrets, URI passwords, and payment card numbers are masked;
• conflicting evidence under the same AuditId fails closed rather than selecting one version.

Redaction is not permission to collect. The safe order remains: avoid unnecessary sensitive input at the producer, normalize as defense in depth, project minimum fields on read, and apply purpose restrictions again to export and alerting.

10. Capture acceptance matrix

First trace each fact to its current producer, paying particular attention to accidental request-body or SQL-parameter capture:

# HTTP, Mediator, and outbound-request capture points.
rg -n "RequestAuditMiddleware|ActivityAuditPipelineBehavior|ExternalRequestLoggingHandler" \
  src/Framework src/Hosts -g '*.cs'

Then inspect post-commit entity-change delivery separately from final normalization. A shared DTO does not prove that both transaction paths are safe:

# Dual-ORM transaction buffers, CAP outbox delivery, and persistence normalization.
rg -n "EntityChangeAudit|CapEntityChangeAuditPublisher|NormalizeForPersistence" \
  src/Framework -g '*.cs'

• HTTP 200/400/401/403/429/500, exception propagation, and client disconnect;
• activity success, business failure, handler exception, and generated [AuditIgnore] policy;
• EF no-transaction, explicit-transaction, and CAP-transaction commit/rollback/unknown-outcome paths plus sensitive-field redaction;
• SqlSugar INSERT/UPDATE/DELETE, unrecognized dialect/CTE, infrastructure exclusion, and transaction buffering;
• outbound success, 4xx/5xx, DNS/timeout/cancellation, query-secret removal, and audit timeout;
• CAP success/failure, duplicate execution key, duration, correlation, and observability isolation;
• invalid TenantId/AuditId, oversized text, conflicting replay, and nested-secret redaction.

Previous: Auditing overview · Next: Queue and delivery