Seed data exporter

BitzOrcas.SeedData.Exporter is a one-time/controlled migration tool. It reads fourteen platform baseline-data categories from legacy SQL Server staging, repairs old table names and relationship keys, and distributes CSV to Framework, MasterData, Authorization, and Menu owners. It is not production backup, general ETL, or a judge of whether data may enter source control.

Data flow and ownership

The exporter owns technical conversion only. Data owners approve source environment, target tenant, minimization, CSV contents, and final commit.

Read current limitations first

The implementation has no --dry-run, preview manifest, temporary staging, atomic directory replacement, or --no-overwrite. On startup it creates output directories and uses File.Create(path), truncating/replacing same-name CSV. A mid-run failure can leave early files updated and later files stale.

Configuration and safe sources

Configuration providers load in override order: appsettings.json, appsettings.local.json, SEED_-prefixed environment variables, then command line. Properties live under Exporter, so an environment key is SEED_Exporter__SourceConnectionString.

# Keep connection material in this shell, not tracked JSON, history, or CI output.
export SEED_Exporter__SourceConnectionString="$LEGACY_STAGING_CONNECTION"
export SEED_Exporter__TargetTenantId='1000001'
export SEED_Exporter__PlatformTenantId='0'

# Redirect every owner to a separate temporary directory.
export SEED_Exporter__OutputDirectory="$TMPDIR/bitz-seed/framework"
export SEED_Exporter__MasterDataOutputDirectory="$TMPDIR/bitz-seed/master-data"
export SEED_Exporter__AuthorizationOutputDirectory="$TMPDIR/bitz-seed/authorization"
export SEED_Exporter__MenuOutputDirectory="$TMPDIR/bitz-seed/menu"

The program masks Password/Pwd in its connection-string log, but that does not prove stack traces, driver logs, and shell history are clean. Treat run logs as sensitive artifacts.

Pre-run checks

1. Source is an approved, sanitized, versioned staging snapshot—not the production primary.
2. SQL identity is least-privilege read-only over listed legacy tables.
3. TargetTenantId is deliberate; sample 1000001 is not a safe business default.
4. PlatformTenantId matches the legacy platform convention, commonly 0.
5. Four temporary output directories are empty or their old contents are checksummed.
6. Record source commit, snapshot ID, operator, approval, and run time.

# Build Release from repository root so compilation exposes tool drift first.
dotnet build src/Tooling/BitzOrcas.SeedData.Exporter \
  --configuration Release

# Confirm output variables do not point at tracked owner Assets.
env | rg '^SEED_Exporter__(Output|MasterDataOutput|AuthorizationOutput|MenuOutput)Directory='

Run the export

Relative configuration paths resolve from the current directory. README defaults assume execution inside the project directory; from repository root, use absolute output paths to avoid unexpected destinations.

# Run inside the project so local settings and default relative paths agree.
cd src/Tooling/BitzOrcas.SeedData.Exporter
dotnet run --configuration Release

Startup validates options, creates four directories, opens SqlConnection, and performs fixed queries/writes in order. Logs show a masked connection, tenant, and absolute owner directories. Cancel immediately and discard temporary output if a destination is wrong.

Fourteen exports and normalization

The source README is authoritative for these mappings. Adding/removing a category requires synchronized exporter, owner seed step, asset inventory, validation tests, and source-fact documentation.

Fail-closed relationship handling

Module, permission, and role relationships cannot be “best effort.” AddStableKey registers both legacy ID and business code: blank keys fail, and one key resolving to different values is ambiguous and fails. Missing permission code, unknown module, or a user-only relation with no role ownership also stops the run.

A target-tenant role overrides a same-name platform role template and all output projects to the target TenantId. That is a migration rule, not universal runtime merge behavior; review override counts and permission differences.

Actual CSV format

Output is caret-delimited (^), not comma-delimited, matching current CsvSeedReader. CsvHelper writes headers; DateTime formats are ISO-like without timezone. Current .NET Encoding.UTF8 writes the stream. Filename prefixes align with seed ordering conventions, but SeedRunner ultimately schedules registered Order and DependsOn.

# Inspect file inventory, delimiter, and header without publishing sensitive rows.
find "$TMPDIR/bitz-seed" -type f -name '*.csv' -print | sort
head -n 1 "$TMPDIR/bitz-seed/authorization/230-sys_permission.csv"

# Empty files and comma-style headers require investigation.
find "$TMPDIR/bitz-seed" -type f -name '*.csv' -size 0 -print
rg -n '^[^^]+,[^^]+' "$TMPDIR/bitz-seed" --glob '*.csv'

When target SysTenant is absent, current code prints WARNING and continues instead of failing the process. Compensate in the post-run inventory: ensure 100-sys_platform_tenant.csv exists and belongs to this run.

Compare with owner Assets

Never copy the whole tree. Compare relative names, headers, row counts, stable business keys, and sensitive fields per owner, then update selectively.

# Inventory and count before looking at row-level diffs.
find "$TMPDIR/bitz-seed" -type f -name '*.csv' -exec wc -l {} + | sort

# Example: compare one permission asset using the current owner path.
git diff --no-index \
  src/Platform/Authorization/BitzOrcas.Platform.Authorization.Infrastructure/Seeders/Assets/230-sys_permission.csv \
  "$TMPDIR/bitz-seed/authorization/230-sys_permission.csv" || true

Review added/deleted permissions, role overrides, module-code changes, tenant IDs, audit identities, connection strings, email/phone, fixed tokens, and demo credentials. Unexplained bulk changes go back to the source snapshot and query rule.

Seed validation and idempotency

# Validate value lengths/format in final candidate Assets; no DB connection needed.
dotnet run --project src/Tooling/BitzOrcas.SchemaMaintenance -- \
  --validate-seed

# Verify seed architecture and real integration contracts.
dotnet test tests/BitzOrcas.Architecture.Tests \
  --configuration Release \
  --filter 'FullyQualifiedName~Seed'
dotnet test tests/BitzOrcas.Integration.Tests \
  --configuration Release \
  --filter 'Category=Docker&FullyQualifiedName~Seed'

Use an empty database for first insert and an existing database for replay: count stays stable, owner-managed values converge, runtime fields survive, and dependencies order correctly. “Second run did not throw” is insufficient.

Recover from failure

Writes are not atomic. After any exception mark the entire temporary root failed and delete it; correct source/configuration and rerun into an empty directory. Never combine files from separate runs to fill the missing tail.

Cancellation or source-network failure is also a failed run. Retain masked log, first exception, written-file inventory, and snapshot ID for diagnosis without sharing attachments containing real secret/PII.

Delivery checklist

• Source staging, snapshot, read-only identity, and data approval are traceable.
• All four outputs are temporary and never overwrite owner Assets directly.
• Fourteen expected files, headers, counts, and owners were checked.
• Unknown module, blank permission, and ambiguous role/key fail closed.
• Missing target-tenant seed was not hidden behind WARNING.
• PII, secret, connection-string, and demo-credential scans pass.
• Schema validation and empty/upgraded DB idempotency contracts pass.
• Only explainable diffs are selected, with generation provenance recorded.

See also

• Fixtures and seed data
• Seed data guide
• Schema maintenance
• Secrets and key management