Skip to content
bitzorcas
中EN

Guide

Consumer Contract Tests

Verify package closure, template boundaries, Generator assets, and Release publish with an isolated feed and an external project.

Last updated

Consumer Contract Tests answer a simple question: if a customer cannot see the product repository and receives only a template plus private-feed access, can the project still build and publish?

Why integration tests are not enough

Product Hosts can reference projects directly and may see local NuGet caches. That can hide missing package dependencies, Generators that are only accidentally visible, and templates that reference or copy product source.

The Consumer Contract removes those conveniences and verifies again.

Local contract flow

Read commercial catalog
→ pack candidates into a temporary feed
→ create an external Consumer project
→ write NuGet.Config with <clear /> and Package Source Mapping
→ isolate NuGet, HTTP, plugin, and CLI caches
→ restore / build / test / Release publish
→ inspect package assets and source exposure

Run it with:

Terminal window
dotnet test tests/BitzOrcas.Framework.ConsumerContract.Tests \
--configuration Release \
--blame-hang-timeout 12m

Required assertions

  • no product-repository ProjectReference;
  • no copied Framework, Platform, or Licensing implementation source;
  • every BitzOrcas.* package comes from the mapped feed;
  • Profile closure is complete and version-consistent;
  • Generators arrive as analyzer assets in the correct Host;
  • build, test, and Release publish pass;
  • customer modules reference only allowed Contracts/SDK seams.

Typical failures

SymptomLikely causeCheck first
NU1101Package not packed, catalog omission, or source mapping mismatchTemporary feed, catalog, NuGet.Config
Generated extension missingGenerator not projected into the HostProfile generator/hostRoles mapping
Local pass, CI failGlobal cache leaked into the testIsolation variables and cache directories
Publish failureRuntime assets, RID, or incorrect dependenciesFirst publish error and dependency graph
Product source foundTemplate copied a prohibited directoryTemplate content and exposure assertions

Difference from formal GA

The local contract repacks current source for PR and development feedback. Formal GA must use immutable release artifacts, an authenticated HTTPS feed, signatures, provenance, SBOM, and policy evidence.

Current coverage matrix

DimensionLocal Consumer ContractFormal Commercial GA
Package sourcePack current source into a temporary folder feedImmutable nupkg files from release workflow
Consumer locationSystem temporary directoryRepository-external Solution in release artifact
FeedIsolated local feed + source mappingAuthenticated HTTPS feed + short-lived read credential
CacheIsolated NuGet/HTTP/plugin/CLI stateSame empty-cache discipline
SignatureNot a local-contract conclusiondotnet nuget verify --all + fingerprint set
Provenance/SBOMDoes not validate formal evidenceSchema, hash, policy, and exemption-expiry checks
Publishmini-api: PublishTrimmed=false; default-business: adds a Mediator trim publishRelease publish; project settings determine trimming

Why isolation covers four caches

Clearing only NUGET_PACKAGES is insufficient. The harness isolates:

  • NUGET_PACKAGES, the global package cache;
  • NUGET_HTTP_CACHE_PATH, HTTP metadata and responses;
  • NUGET_PLUGINS_CACHE_PATH, Credential Provider/plugin state;
  • DOTNET_CLI_HOME, first-run, template, and user-level CLI state.

NuGet.Config starts with <clear /> and maps BitzOrcas.* to the isolated feed, preventing accidental restore from nuget.org or an undeclared machine source.

Terminal window
# Run the full contract set; the fixture packs once and tests copy its read-only baseline.
dotnet test tests/BitzOrcas.Framework.ConsumerContract.Tests \
--configuration Release \
--blame-hang-timeout 12m \
--logger 'console;verbosity=normal'
# Isolate the Mini API external-consumption happy path.
dotnet test tests/BitzOrcas.Framework.ConsumerContract.Tests \
--configuration Release \
--filter 'FullyQualifiedName~Profile_Mini_Api_Should_Restore_Build_And_Publish'

What deletion tests prove

A positive restore can be a false positive if the Profile meta-package is empty and the Consumer never truly depends on it. The contract copies the feed, deletes a Profile package or transitive closure dependency, and requires empty-cache restore to fail.

Deletion proves the package carries dependency responsibility. It does not prove every public API has a customer; API compatibility still needs API diff, compilation fixtures, or customer-scenario tests.

Package-content safety checks

Contracts extract nupkg files and reject product-core .cs, non-template .csproj, PDB, Source Link configuration, local absolute paths, private-key markers, and common credential markers. Template packages may contain Consumer shell source but not Framework, Platform, or Licensing core namespaces.

This is a delivery isolation line, not the only source-protection control. Package signing, feed authorization, minimal public API, legal terms, and release audit remain necessary.

Extending contracts for a new Profile

  1. register the packable project in the machine-readable commercial catalog;
  2. declare full package closure, host roles, generators, and License Features;
  3. create an external Consumer referencing the meta-package, never product projects;
  4. verify empty-cache restore, Release build, test, and publish;
  5. delete the Profile and key transitive packages and require restore failure;
  6. inspect template output and nupkg files for product implementation source;
  7. assert Generator/analyzer assets for every target host;
  8. add a separate trimmed-publish contract when trim/AOT support is claimed.

Contract-batch acceptance evidence

  • logs identify commit, SDK, and configuration;
  • temporary feed package set matches catalog projection;
  • every Consumer uses PackageReference only;
  • four cache classes are isolated and source mapping is active;
  • positive flow and deletion tests behave as expected;
  • package scan finds no source, path, or secret leak;
  • whether each Profile’s publish is trimmed is recorded per profile in the conclusion;
  • formal release still hands immutable artifacts to the GA workflow.

See also

100%

Scroll or use controls to zoom · drag when enlarged · double-click for 100% / 200%