Skip to content
bitzorcas
中EN

Guide

Website analytics ingestion, anonymous identity, and privacy

Visitor cookie, journey, telemetry minimization, in-memory queue, persistence, DLQ, mart, consent gap, and privacy endpoint risks.

Last updated

Website Analytics correlates anonymous behavior with a server-generated VisitorId, writes through a bounded Channel, and rebuilds tenant marts. It minimizes data and exposes read/erase APIs, but currently has neither consent state nor proof that an anonymous visitor owns a VisitorId.

1. Ingestion path

Public request

Visitor middleware

Redis journey

Capture context

Schema + PII checks

in-process Channel

Batch writer

AnalyticsEvent facts

Mart aggregation

in-process DLQ

The cookie is 256-bit random Base64Url, fixed length 43, HttpOnly, SameSite=Lax, Path=/, MaxAge 365 days, Secure only on HTTPS requests, and IsEssential=true.

Middleware validates length/charset only; the cookie is not signed. A client can supply another well-formed value. Source explicitly says VisitorId is not an identity credential.

3. Journey

Redis Journey uses a global VisitorId key and distributed lock to preserve first touch. Thirty minutes of inactivity creates a new Session; TTL is 400 days. UTM values are bounded and reject obvious emails/phone numbers.

On cache failure middleware logs and continues with a fallback SessionId="missing". Degraded traffic then shares the same session value and can distort reports.

4. Automatic page views

Public, telemetry, and short-link surfaces pass through middleware. Successful non-telemetry GETs with a Journey automatically capture pageview, including content, privacy GET, assignment, and redirect responses.

No consent cookie/store, Global Privacy Control, or Do Not Track check exists. IsEssential is cookie configuration, not legal basis. Documentation must not claim “consent before capture.”

5. Custom events and Web Vitals

Telemetry batches contain 1..50, Web Vitals 1..20; LCP/INP are 0..120,000ms and CLS 0..10. Client time must be within five minutes of receipt.

Clients cannot submit experiment-assigned. Conversion with experiment dimensions calls the experiment service to verify assignment token. Web Vitals adds lcp/cls/inp/thresholdViolation dimensions.

6. Minimization

Path drops query/fragment; Referrer retains scheme/host/path; UA becomes coarse device/browser; IP is truncated to IPv4 /24 or IPv6 /64 before GeoIP; only region is stored.

At most 20 dimensions; key ≤48, value ≤256; only built-in/server-registered keys pass. Heuristics reject names such as email/phone/contact/name/ip/useragent and obvious identifier values.

Heuristics are not DLP and may both miss identifiers and reject legitimate numbers. Every new dimension needs privacy and cardinality review.

7. Acceptance semantics

Current queue acceptance
// ① Collector builds the complete batch and assigns stable persistence IDs.
Result created = collector.Capture(context, events);
// ② TryWrite returns AnalyticsQueueFull immediately when full.
if (created.IsFailure)
return Problem(created.Error);
// ③ Success means in-process acceptance; persistence may not have happened.
return Accepted();

The Channel uses FullMode=Wait but Collector calls TryWrite, so it does not wait. Graceful stop completes/drains; crash, kill -9, or node loss drops in-flight batches.

8. Write and idempotency

The writer creates a scope per batch, retries with exponential backoff, then calls the store. The store queries existing stable IDs, inserts pending rows, and commits one explicit unit of work.

Stable IDs make replay of the same batch idempotent, but query-then-insert can still race on uniqueness. Dual-ORM contracts must classify that collision. This is not end-to-end exactly once.

9. Dead letters

Exhausted batches enter a per-process ConcurrentDictionary with full events and exception summary. Health becomes Degraded and declares durability=in-process.

Operations sees only the current instance; restart loses entries; load balancing may route operators elsewhere. GA needs durable cluster-visible DLQ, lease-based replay, audit, and retention.

10. Privacy APIs

Authenticated APIs use current tenant + current user to query/soft-delete SubjectId. Anonymous APIs use PublicTenantId + Visitor cookie and also remove Journey.

A replaceable cookie is not ownership proof. Anyone who obtains or guesses another valid VisitorId can query/erase it. Use a signed ownership token, one-time recovery credential, or verified-account binding.

Erase soft-deletes facts, but existing marts are not synchronously rebuilt; queued/DLQ events may persist afterward; middleware can touch Journey again during the erase request. This is not a complete replay-safe DSR workflow.

11. Mart

The job rebuilds day/week/month rows for configured TenantIds and LookbackDays. Aligned queries prefer marts; others scan facts; last-five-minute realtime reads facts.

Tenant discovery is manual. Missing a tenant silently omits its materialization. Replace-window transactions and concurrent-reader semantics need dual-ORM evidence.

12. Tests and operations

Cover cookie substitution/theft, capture without consent, Journey fallback, PII/cardinality, queue full, graceful drain, crash loss, retry/DLQ/restart, multi-instance operations, duplicate IDs, erase/in-flight race, late events, and both ORMs.

Terminal window
# ① Consent should have no Website implementation; preserve the non-credential warning.
rg -n "Consent|VisitorId.*不构成身份凭证|IsEssential" \
src/Platform/Website src/Hosts/BitzOrcas.Api -g '*.cs'
# ② Inspect in-process queue and DLQ.
rg -n "Channel\.CreateBounded|ConcurrentDictionary|durability.*in-process" src/Platform/Website -g '*.cs'

GA design must explicitly define:

  • purpose, version, source, time, and consent evidence;
  • strict essential-versus-analytics cookie split;
  • whether no-consent skips Journey and GeoIP completely;
  • immediate capture stop after withdrawal;
  • GPC/DNT and regional decision rules;
  • Visitor-to-Subject link and unlink;
  • anonymous ownership-token issue, rotation, and revoke;
  • CSRF and cookie-fixation protection for read/erase;
  • one operation ID across queue, facts, DLQ, and mart;
  • isolation of new events while erase runs;
  • soft delete, physical purge, and backup-expiry SLAs;
  • how a subject contribution leaves materialized marts;
  • whether replay checks current consent/erasure tombstone;
  • legal/product basis for 400-day Journey TTL;
  • retention of network prefix and region;
  • privacy approval for custom dimensions;
  • accepted/persisted/loss-estimate SLOs;
  • cluster aggregation of queue depth and DLQ;
  • capacity/backpressure/cost of a crash-safe inbox;
  • a human-readable export format.
  • approval and reason audit for operational replay;
  • processors and cross-border locations in the evidence pack;
  • client-visible versioning for consent, capture, and erasure.

Back to Website

100%

Scroll or use controls to zoom · drag when enlarged · double-click for 100% / 200%