Skip to content
bitzorcas
中EN

Concept

Auditing Module Architecture: Append-Only Trails & Tiered Storage

Explore the BitzOrcas.Modern Auditing Subsystem. Learn immutable SysAuditRecord entities, automated pipeline interception, SQL/Mongo tiered storage, and GDPR retention.

Last updated

In enterprise software delivery and compliance audits (SOC2, ISO 27001, GDPR), the Auditing Subsystem is the primary line of forensic accountability:

  • Missing Audit Records: Privileged operators modifying contracts without logging, leaving no forensic trace;
  • Audit Log Tampering: Malicious actors acquiring standard database credentials and executing UPDATE / DELETE statements to destroy evidence;
  • Storage Bloat Exhausting Databases: Millions of daily mutations bloating tables, degrading general application performance.

BitzOrcas.Modern Auditing Module implements “10-Stage Pipeline Auto-Capture + Append-Only Storage + SQL/Mongo Tiered Retention”:

Auditing Lifecycle and Tiered Storage Topology

Older than 90 Days

1. Mutating Command Slice

2. ActivityAuditPipelineBehavior (Auto-Captures Operator, Tenant, Duration & Diff)

3. In-Memory Buffer (Channel Batch Aggregator)

4. AuditPersistenceBackgroundWorker (Async Batch Flusher)

5. Hot Storage (SqlSugar Split Tables / PostgreSQL - 90 Days Retention)

6. Cold Archive (MongoDB / Compressed S3 - 3 Years Retention)


Step 1: Immutable Audit Entity SysAuditRecord

Audit records follow an append-only invariant model where physical updates are strictly forbidden:

SysAuditRecord.cs: Append-Only Audit Entity
using BitzOrcas.Domain.Entities;
using BitzOrcas.Persistence.Metadata;
namespace BitzOrcas.Auditing.Domain;
// Split monthly (e.g. SysAuditLog_202608) to maintain bounded table sizes
[BitzTable("SysAuditLog", IsTenant = true, IsSoftDelete = false, Description = "Immutable Audit Records")]
[BitzIndex("IX_SysAuditLog_TraceId", nameof(TraceId))]
[BitzIndex("IX_SysAuditLog_Tenant_Time", nameof(TenantId), nameof(OccurredAt))]
public sealed class SysAuditRecord : Entity<string>
{
[BitzColumn(Length = 64, IsRequired = true)]
public string TraceId { get; set; } = string.Empty;
[BitzColumn(Length = 64, IsRequired = true)]
public string ActionName { get; set; } = string.Empty;
[BitzColumn(Length = 64, IsRequired = true)]
public string OperatorId { get; set; } = string.Empty;
[BitzColumn(Length = 120)]
public string OperatorName { get; set; } = string.Empty;
[BitzColumn(Length = 50)]
public string ClientIp { get; set; } = string.Empty;
[BitzColumn(IsRequired = true)]
public DateTimeOffset OccurredAt { get; set; }
[BitzColumn(IsRequired = true)]
public long DurationMs { get; set; }
// JSON mutation diff snapshot
[BitzColumn(IsJson = true)]
public string? DiffSnapshot { get; set; }
}

Step 2: High-Performance Cursor-Based Querying

To prevent OFFSET 1000000 memory spikes on large audit datasets, the query slice enforces cursor pagination:

QueryAuditLogsQuery.cs: Cursor-Based Audit Query
using BitzOrcas.Application.Abstractions.Queries;
using BitzOrcas.Domain.Results;
using BitzOrcas.Endpoint.Attributes;
using Mediator;
namespace BitzOrcas.Auditing.Application.Queries;
// 1. Declarative query endpoint: compile-time Minimal API mapping
[GenerateEndpoint(HttpRoute.Query, "/api/auditing/logs", Tag = "Auditing")]
// 2. Cursor pagination contract: eliminates deep pagination memory spikes via CursorTime and CursorId
public sealed record QueryAuditLogsQuery(
DateTimeOffset? StartTime,
DateTimeOffset? EndTime,
string? CursorTime,
string? CursorId,
int PageSize = 50) : IQuery<Result<PagedList<AuditLogDto>>>;

Summary

The Auditing Module meets strict regulatory compliance standards:

  • Zero Omission: 10-stage pipeline automatically captures mutations with zero handler boilerplate;
  • Append-Only Immutability: Enforces insert-only storage guarantees;
  • Tiered High-Throughput: Table sharding + cursor pagination delivers microsecond query responses.

100%

Scroll or use controls to zoom · drag when enlarged · double-click for 100% / 200%