Master Data Seeding and Asset Integrity

MasterData uses EntitySetCsvSeedStepBase<T> for row-by-row upsert. It is ORM-neutral; it is deterministic only when natural keys, headers, and content are correct.

1. Execution path

This is not a bulk merge, and the base class does not open one transaction for the whole step.

2. Eight steps

All inherit ProductionSafe, Version=1, and no explicit dependencies. Order sorts execution but is not a dependency graph.

3. Lenient parsing

The reader uses ^, ignores header case/BOM, permits missing fields, and installs lenient value converters. Header and missing-field validation are disabled.

A zero exit code is not proof that the asset contract passed.

4. Exchange-rate header drift

The asset says UpdateDate; the record and seed key say ModifyTimee. With no data rows, even the key warning does not fire. Appending rows can leave the date at its default and only warn.

# These currently expose different names.
head -1 src/Platform/MasterData/BitzOrcas.Platform.MasterData.Infrastructure/Seeders/Assets/150-sys_exchange_rate.csv
rg -n "ModifyTimee|UpdateDate" src/Platform/MasterData -g '*.cs' -g '*.csv'

GA should migrate to correct domain naming with a compatibility plan.

5. Idempotent-upsert boundary

// ① Match must agree with database uniqueness and the CSV business key.
var existing = await entities.FirstOrDefaultAsync(
    row => row.Code == source.Code && row.Class == source.Class,
    cancellationToken);

// ② The asset ID is kept; a blank ID receives the string "0".
if (existing is null)
    await entities.AddAsync(source, cancellationToken);
else
{
    // ③ Copy updates asset-managed fields, preserving persistence identity.
    CopyManagedFields(source, existing);
    await entities.UpdateAsync(existing, cancellationToken);
}

An incomplete matcher can overwrite the wrong row; a stricter database key can surface duplicates/conflicts.

6. CodeText collision

The database index includes Tenant+Code+Class+Language, but the seed matcher uses Code+Language. The same code in two classes can overwrite. The current empty asset hides rather than resolves this risk.

7. Empty assets

Country, industry, code-text, and exchange-rate files contain headers only. The reader returns zero rows, the step logs upserted 0, and execution succeeds.

Operationally this means the schema/step exists but the package supplies no data. A dependent environment needs minimum-row and authority-version gates.

8. Asset release gate

// ① Parse the same embedded asset and validate schema, key, and cardinality.
var report = await validator.ValidateAsync(new SeedAssetContract
{
    SeedId = "sys_general_code",
    RequiredHeaders = ["Code", "Class", "DisplayName"],
    UniqueKey = ["TenantId", "Code", "Class"],
    MinimumRows = 40_000
}, cancellationToken);

// ② Promote warnings to CI/production gate failures.
if (report.Errors.Count > 0 || report.Warnings.Count > 0)
    throw new SeedAssetValidationException(report);

This validator is a proposed design, not current source.

9. Update and deletion semantics

Copy updates selected fields, but removing a CSV row does not delete the database row. This is append/update, not desired-state reconciliation.

Keep a row and mark it inactive/deleted, or implement a reviewed deletion plan. Scan business references and provide a compatibility window before retiring a code.

10. Stable IDs and codes

Large assets carry historical string IDs. Business data should reference stable codes unless an established FK requires IDs. Changing a natural key creates a new fact while leaving the old row; publish an auditable add/update/deactivate/key-change diff.

11. Performance and transactions

43,603 codes currently perform a per-row lookup plus write. Benchmark empty/existing databases on both ORMs and realistic latency.

Optimization must preserve dual-ORM equivalence, key idempotency, bounded transactions/recovery, cancellation, actionable progress, and compile-time metadata/AOT constraints.

12. Production checklist

1. record source, license, revision, and owner;
2. require exact headers;
3. align unique asset key, index, and Match;
4. validate counts, nulls, parents, and ranges;
5. review semantic diff;
6. test empty and repeat runs on both ORMs;
7. meet the performance window;
8. rehearse failure recovery and rollback;
9. verify minimum counts and sentinel rows after execution;
10. update docs and source facts.

13. Test commands

dotnet test tests/BitzOrcas.Architecture.Tests/BitzOrcas.Architecture.Tests.csproj --filter MasterData
dotnet test tests/BitzOrcas.Integration.Tests/BitzOrcas.Integration.Tests.csproj --filter Seed

# Identify header-only assets.
for f in src/Platform/MasterData/*/Seeders/Assets/*.csv; do wc -l "$f"; done

Module overview · Dictionary and cache