Webhook signatures, secrets, and rotation

A Webhook signature lets a receiver confirm that the body was not modified and the sender possesses the shared secret. It does not establish an end-user identity, replace TLS, or maintain replay state. The receiver must implement both a timestamp window and EventId deduplication.

1. Exact wire contract

The sender hashes the raw UTF-8 bytes of payloadJson:

payloadHash = lowercase_hex(SHA256(UTF8(payloadJson)))
canonical   = eventId + "\n" + unixMilliseconds + "\n" + payloadHash
signature   = lowercase_hex(HMAC_SHA256(UTF8(secret), UTF8(canonical)))

It then sends:

POST /v1/bitzorcas/events HTTP/1.1
Content-Type: application/json; charset=utf-8
X-BitzOrcas-Event-Id: 019c48cb7ba47000953d5af574ddc531
X-BitzOrcas-Event-Type: files.finalized
X-BitzOrcas-Timestamp: 1784102400123
X-BitzOrcas-Payload-Hash: 3e478f21bc9e1a84f395d824d55b018b1081395fc02d847936a282924a2ef932
X-BitzOrcas-Signature: 813a6bc7d921e42f9b89154a32014cd7ea310f852b719463b2160d5e128174f9

{"fileId":"file-42","status":"Finalized"}

EventType, TargetUrl, HTTP method, Content-Type, and subscription id are not in the canonical input. A future change needs an explicit signature-version header; silently changing the string breaks every receiver.

2. ASP.NET Core receiver verification

Verify before JSON model binding changes or discards the original body. Never deserialize and reserialize before hashing: property order, whitespace, escaping, or number formatting can change.

app.MapPost("/v1/bitzorcas/events", async (
    HttpRequest request,
    IWebhookReplayStore replayStore,
    ISecretStore secrets,
    CancellationToken cancellationToken) =>
{
    request.EnableBuffering();
    using var reader = new StreamReader(
        request.Body, Encoding.UTF8, detectEncodingFromByteOrderMarks: false,
        leaveOpen: true);
    var payload = await reader.ReadToEndAsync(cancellationToken);
    request.Body.Position = 0;

    var eventId = request.Headers["X-BitzOrcas-Event-Id"].ToString();
    var timestampText = request.Headers["X-BitzOrcas-Timestamp"].ToString();
    var claimedHash = request.Headers["X-BitzOrcas-Payload-Hash"].ToString();
    var claimedSignature = request.Headers["X-BitzOrcas-Signature"].ToString();

    if (!long.TryParse(timestampText, CultureInfo.InvariantCulture, out var timestamp))
        return Results.Unauthorized();

    // Five minutes is an example; review it with queue delay and clock discipline.
    var sentAt = DateTimeOffset.FromUnixTimeMilliseconds(timestamp);
    if ((DateTimeOffset.UtcNow - sentAt).Duration() > TimeSpan.FromMinutes(5))
        return Results.Unauthorized();

    var computedHash = WebhookSignature.ComputePayloadHash(payload);
    if (!CryptographicOperations.FixedTimeEquals(
            Encoding.ASCII.GetBytes(computedHash),
            Encoding.ASCII.GetBytes(claimedHash)))
        return Results.Unauthorized();

    var secret = await secrets.GetCurrentAsync(cancellationToken);
    if (!WebhookSignature.Verify(
            secret, eventId, timestamp, computedHash, claimedSignature))
        return Results.Unauthorized();

    // Atomically claim eventId after authentication; duplicates return 2xx without effects.
    if (!await replayStore.TryBeginAsync(eventId, cancellationToken))
        return Results.Ok(new { duplicate = true });

    await ProcessEventAsync(payload, cancellationToken);
    await replayStore.MarkCompletedAsync(eventId, cancellationToken);
    return Results.Ok();
});

Production also needs body limits, Content-Type checks, key/signature version, trusted-proxy TLS, and low-cardinality failure metrics. Never log the secret or full signature.

3. Receiver deduplication must be durable

The sender deduplicates (EventId, SubscriptionId). A receiver usually handles one subscription and can key on EventId; if several subscriptions converge on one service, use (SubscriptionId, EventId) or add a signed SubscriptionId header.

A replay record needs Processing/Completed, first-seen time, and expiry. Business-first then inbox-write duplicates after a crash; Completed-first can lose the event. Use the same transaction or a recoverable Processing lease.

4. One-time secret lifecycle

WebhookSubscription.Create uses RandomNumberGenerator.GetBytes(32) and returns Base64. The aggregate stores a SHA-256 SecretHash and signing SecretMaterial. Create returns plaintext once; summaries exclude all secret fields.

Correct operation:

1. show once and require a secure-storage acknowledgement;
2. write the receiver copy to Secret Manager/KMS/Vault, not appsettings, CI logs, or tickets;
3. persist and share the BitzOrcas DataProtection key ring across instances;
4. make the receiver accept current and previous before sender activation;
5. on exposure, rotate urgently, suspend, audit, and inspect replay.

SecretHash currently participates in no management verification, index, or audit. Do not claim the framework uses it to authenticate callback clients.

5. At-rest encryption path

Infrastructure registers WebhookSecretDataProtectionCipher with purpose Webhooks.SecretMaterial. Repository SaveAsync Protects before Add/Update and replaces the aggregate field with ciphertext. Delivery Unprotects once before HMAC.

DataProtection is not a substitute for external KMS auditing. A lost key ring makes historical material undecryptable. Current Unprotect catches every exception and returns the input for legacy-plaintext compatibility; key-ring damage therefore does not fail closed—it uses ciphertext as the HMAC secret and receivers see unexplained signature failure.

6. Repeated Save double-encrypts today

Repository does not test whether material is already protected:

// First creation: plaintext -> Protect -> ciphertext-1, correct.
await repository.SaveAsync(created, cancellationToken);

// A later load obtains ciphertext-1; suspend/update does not change the secret.
var loaded = await repository.FindAsync(tenantId, id, cancellationToken);
loaded.Value!.Suspend();

// Save protects ciphertext-1 again -> ciphertext-2.
await repository.SaveAsync(loaded.Value, cancellationToken);

// Delivery unprotects once, obtains ciphertext-1, and uses it as the HMAC secret.

Dual-ORM parity tests save/read delivery facts but do not prove “Update then Deliver still signs with the original plaintext.” This is a GA blocker. Repeated Unprotect loops are not a safe fix because arbitrary legacy plaintext and encryption depth are not reliably distinguishable.

Separate domain secret state from persistence ciphertext, or add an explicit ciphertext version/prefix and Protect only plaintext state. Add Create→Update→Deliver, Suspend→Resume→Deliver, repeated Save, and rotated-previous-key regressions on both ORMs.

7. Current rotation-overlap meaning

RotateSecret:

• rejects Deleted but permits Suspended;
• generates a new secret;
• moves current material to PreviousSecretMaterial;
• sets PreviousSecretExpiresAt to now + overlap, default 300 seconds;
• updates hash/current/RotatedAt;
• returns new plaintext once.

The sender always signs with current material. It never retries with previous, and no inbound verifier uses previous. The field has no runtime reader and no expiry cleanup. Keeping a previous key in the sender database does not help an external receiver validate already in-flight requests signed with the old key.

8. Recommended versioned rotation

Add X-BitzOrcas-Signature-Version and X-BitzOrcas-Key-Id. Prefer prepare/activate/retire to immediate switch. Retirement must cover maximum queue age, automatic retry age, and clock skew, with an audit event.

9. Migration adapter boundary

WebhookSecretMigrationAdapter scans every tenant subscription. If current DataProtection can decrypt a value it skips; otherwise it assumes legacy plaintext, Protects, and updates. Ciphertext encrypted by an unavailable old key ring is misclassified as plaintext and wrapped again.

Migration has no paging, checkpoint, distributed lock, or failure report. It also skips the whole row when current is protected without independently checking previous. Restore historical key rings first and validate decrypt/signature in staging.

10. Required security tests

1. A published fixed vector for secret, body, timestamp, hash, and signature.
2. Whitespace/order/Unicode/newline/large-body raw-byte behavior.
3. Missing/duplicate/oversized headers, malformed hex, fixed-time comparison.
4. Old/future timestamp and durable EventId claim/crash recovery.
5. Signature remains valid after Update/Suspend/Resume.
6. Rotation activates new key and retires old at the defined boundary.
7. Cross-instance DataProtection, key rotation, loss, and restore drills.
8. Migration checkpoint and wrong-key/current-previous mixed rows.
9. No secret/body leakage in logs, traces, Problem Details, or audit.
10. On-call emergency rotation and suspension runbook.

11. Review commands

rg -n "WebhookSignature|WebhookHeaderNames|HMACSHA256|PayloadHash" \
  src/Platform/Webhooks -g '*.cs'

rg -n "Protect\(|Unprotect\(|ApplyEncryptedSecrets|catch" \
  src/Platform/Webhooks -g '*.cs'

# Currently expected to find writes only; correction should add real verify/retire paths.
rg -n "PreviousSecretMaterial|PreviousSecretExpiresAt" \
  src/Platform/Webhooks -g '*.cs'

Back to Webhooks · subscriptions and scopes · delivery and retry