Integration tests

BitzOrcas.Integration.Tests contains both container-free Host/API contracts and Testcontainers contracts. Their cost, failure modes, and proof scope differ, so Category filters must separate them. Sharing a project does not mean they should share fixtures or run in the same CI job.

Evidence layers

Container-free contracts verify ASP.NET Core composition and cross-layer control flow but do not claim that a database provider works. Docker contracts prove real adapter semantics, but they still do not replace production capacity, disaster recovery, or security tests.

Container-free contracts

These tests use TestApiFactory or build application services directly. They cover API Shell startup, authentication/tenant middleware, Problem Details, health checks, missing-configuration fail-closed behavior, and cross-layer paths that need no external broker or database.

# Fast local feedback and normal PR gate; explicitly exclude Testcontainers.
dotnet test tests/BitzOrcas.Integration.Tests \
  --configuration Release \
  --filter 'Category!=Docker'

# Reproduce only the API Shell build smoke contract.
dotnet test tests/BitzOrcas.Integration.Tests \
  --configuration Release \
  --filter 'FullyQualifiedName~ApiShellHostBuildSmokeTests'

Good assertions here include endpoint mapping, unauthenticated response protocol, tenant-context projection, and failure when production configuration is absent. SQL unique indexes, rollback, locking, and message delivery belong in Docker contracts.

Docker contracts and traits

Real-infrastructure tests use [Trait(TestCategories.CategoryKey, TestCategories.Docker)]. If a new container test omits the trait, it leaks into the container-free job; DockerContractTraitTests guards that classification contract.

[Trait(TestCategories.CategoryKey, TestCategories.Docker)]
public sealed class WebsitePersistenceProviderContractTests : IAsyncLifetime
{
    // Share containers for this contract class; each test still owns its data.
    public Task InitializeAsync() => StartSqlServerAndCreateSchemaAsync();

    [Fact]
    public async Task DuplicateSlug_ShouldMapToSameConflict_ForBothProviders()
    {
        // Drive EF Core and SqlSugar with one scenario; do not compare SQL text.
        var efResult = await ExecuteDuplicateSlugAsync(PersistenceProvider.EfCore);
        var sqlSugarResult = await ExecuteDuplicateSlugAsync(PersistenceProvider.SqlSugar);

        efResult.ErrorCode.ShouldBe("Website.SlugConflict");
        sqlSugarResult.ErrorCode.ShouldBe(efResult.ErrorCode);
    }

    public Task DisposeAsync() => StopContainersAsync();
}

This illustrates the contract shape. The repository’s Website contracts also verify production metadata, unique indexes, schema reset, and real IEntitySet behavior. Provider parity is more than “neither ORM threw.”

What ORM parity compares

Parity concerns observable semantics, not identical generated SQL:

Provider-specific capabilities may differ only when the adapter contract and documentation state the difference explicitly.

Messaging and Golden Use Case

GoldenUseCaseApiFactory composes SQL Server, RabbitMQ, and the real API Host. Golden Use Case and CAP/Outbox contracts ask whether a request commits and the correct handler eventually consumes the message—not whether an in-memory event list has one item.

// Arrange: create a tenant and business key unique to this test.
var client = factory.CreateTenantClient(tenantId);

// Act: hit the real Golden Use Case route /api/notes — one request writes the database and emits an integration event.
var noteTitle = "matter-kickoff-note";
var response = await client.PostAsJsonAsync("/api/notes", new { title = noteTitle });
response.StatusCode.ShouldBe(HttpStatusCode.Created);

// Assert: poll a message probe until CAP consumes from the Outbox; bound the wait instead of Thread.Sleep.
await EventuallyAsync(
    () => messageProbe.ContainsAsync(noteTitle),
    timeout: TimeSpan.FromSeconds(20));

Messaging tests also cover duplicate delivery, handler retry, cancellation, and timeout. A happy-path assertion alone does not prove idempotency or recovery.

The thirteen CI shards

The docker-integration-contracts.yml matrix splits Docker contracts into thirteen shards: shared-port-a/b/c/d, shared-repository, shared-readmodel-queryshape, shared-generic, website, webhooks, messaging, authorization-capacity, core, and workflow. The workflow triggers via workflow_call/workflow_dispatch (invoked by CI or release flows rather than on every push/PR) with fail-fast: false; the local equivalent entry is scripts/build/test-docker.sh (list/smoke/shard/full modes).

# Reproduce the website shard by copying its exact CI filter.
dotnet test tests/BitzOrcas.Integration.Tests \
  --configuration Release \
  --filter 'Category=Docker&FullyQualifiedName~BitzOrcas.Integration.Tests.Website.' \
  --blame-hang-timeout 10m

A new Docker test must belong to exactly one shard, keeping the Architecture.Tests shard-ownership contract in sync. Because core is exclusion-based, namespace changes can overlap or omit tests; prefer explicit namespaces when adding tests.

Isolation and lifecycle

Assembly-level CollectionBehavior(DisableTestParallelization = true) currently disables parallel execution to avoid Docker resource contention. Tests must still use unique database names, tenants, business keys, and queue identifiers rather than relying on that switch to hide data pollution. Put cleanup in DisposeAsync so assertion failures still release containers.

Fixed ports collide with development services; prefer Testcontainers dynamic binding. Readiness must wait for an actually usable service, not merely a Running container. No test should require another test to seed its data first.

Failure diagnosis order

1. Confirm Docker daemon, disk, memory, and image pull health.
2. Inspect container logs and readiness to separate startup from business failure.
3. Confirm schema initialization, migrations, and the seed report.
4. Verify that the current shard filter selected the intended tests.
5. Then compare provider input, tenant context, transaction boundary, and assertions.
6. For hangs retain blame evidence and inspect waits, cancellation, and cleanup.

Review checklist

• Docker tests have the standard trait and exactly one CI shard.
• Data, tenant, database, and queue identity are isolated per test.
• Provider parity compares business semantics, not SQL strings.
• Eventually-consistent assertions use bounded polling instead of fixed sleep.
• Migrations cover first run, upgrade, replay, and partial failure.
• Failure output carries provider, tenant, business key, and container clues.
• Container-free tests are not overstated as real-infrastructure evidence.

See also

• Fixtures and seed data
• Running tests
• Persistence
• Consumer contracts