Deployment Method Selection

The BitzOrcas production Hosts are three independent processes: API, JobHost, and Gateway. The repository does not declare any single topology as the only official path. Instead it ships scripts and manifests for three common routes — single-host systemd, single-host docker-compose, and Kubernetes. All three routes must satisfy the same invariants: migrations complete before resident services start, the three services start and stop in dependency order, and a failing /health/ready must block traffic.

Comparing the three routes

Shared invariants

Regardless of route, the deploy flow must preserve this ordering, or migrations and startup will race and health checks will misjudge readiness:

1. Database, Redis, and RabbitMQ are ready first.
2. The API migrator runs --init-schema (auto CREATE DATABASE + business tables + audit partitions + idempotent seed).
3. The JobHost migrator runs --init-quartz-schema (Quartz AdoJobStore persistence tables).
4. Start JobHost, then API, then Gateway (Gateway reverse-proxies API and depends on API health).
5. Each service’s /health/ready returns 200 before traffic is routed.

Stop order is the reverse: Gateway → API → JobHost.

systemd single-host deployment

Preparation

One-time server setup:

# ① Install .NET 10 Runtime (migrators and resident processes need it; SDK not required)
#    Ubuntu example; use dnf on CentOS/RHEL
sudo apt-get install -y dotnet-runtime-10.0
dotnet --version    # should print 10.0.x

# ② Create the directory layout
sudo mkdir -p /www/{apps,releases,backups,scripts} /etc/bitzorcas

# ③ Place the deploy script (from the code repo scripts/deploy/)
sudo cp scripts/deploy/deploy.sh /www/scripts/deploy.sh
sudo chmod +x /www/scripts/deploy.sh

# ④ Generate deployment-id (community.small seat-counting anchor; once set, never change it)
sudo mkdir -p /www/apps/<env>/.bitzorcas/license
cat /proc/sys/kernel/random/uuid | tr -d '-' \
  | sudo tee /www/apps/<env>/.bitzorcas/license/deployment-id

Configure the environment file

/etc/bitzorcas/<env>.env is used by deploy.sh (load_env_file for migrators—do not source keys that are illegal shell identifiers) and by the three systemd units (EnvironmentFile=), so all three services share one file:

sudo cp scripts/deploy/env.example /etc/bitzorcas/<env>.env
sudo chmod 600 /etc/bitzorcas/<env>.env   # contains DB passwords and JWT keys; root-only
sudo nano /etc/bitzorcas/<env>.env

Required keys are documented in scripts/deploy/env.example; the critical ones:

• ConnectionStrings__Default: database connection string
• PiiEncryption__SearchHashKey: openssl rand -base64 32; API and JobHost must share the same stable value
• Jwt__Secret: openssl rand -base64 48, ≥32 chars
• Licensing__Runtime__PolicyId: community.small.v1 (≤30-person phase) or a commercial policy
• Frontend__BaseUrl / Cors__AllowedOrigins__0: exact browser origin (scheme + host + port)
• Gateway__KnownProxies__0: production ingress CIDR (loopback-only rejected in Production); preview may use 127.0.0.1/8
• ReverseProxy__Clusters__<api|signalr|files>__Destinations: cluster ids must not contain hyphens (systemd drops api-cluster)
• OpenApi__Enabled / OpenApi__RequireAuthentication / OpenApi__PersistAuthentication: normally disable documentation in Production; an enabled shared preview requires the product documentation session and keeps Scalar authentication persistence false
• OpenApi__Servers__0__Url: public origin when documentation is enabled; an IP entry must include its non-default port

Seed-related variables:

Packaging

Pack the three Hosts into a single zip (api/ jobhost/ gateway/ + manifest.txt):

# First deploy: force full package
./scripts/deploy/pack.sh <env> <version> --full

# Later incremental: download server baseline first, then pack without --full
scp <user>@<server>:/www/apps/<env>/manifest.txt \
  ../publish/<branch>/manifest-server-<env>-$(date +%m%d%H%M).txt
./scripts/deploy/pack.sh <env> <version>

# Windows PowerShell (UTF-8 scripts; console forced to UTF-8 at startup)
.\scripts\deploy\pack.ps1 -Env <env> -Version <version>
.\scripts\deploy\pack-frontend.ps1 -Environment <env> -Version <version>

Incremental mode trusts only manifest-server-<env>-*.txt. Local manifest-baseline-* is audit-only; without a server baseline the packer falls back to a full package. Output: <repo-parent>/publish/<branch>/bitzorcas-<env>-<version>.zip.

Deploy

# Upload to the server
scp bitzorcas-<env>-<version>.zip <user>@<server>:/www/releases/

# Default platform-seed: platform seeds (dictionaries, …), demo users skipped — not “tables only”
ssh <user>@<server> "sudo /www/scripts/deploy.sh /www/releases/bitzorcas-<env>-<version>.zip <env>"

# Day-to-day app deploys: tables/columns only, no seeds at all (including dictionaries)
ssh <user>@<server> "sudo BITZORCAS_DEPLOY_DB_MODE=schema-only /www/scripts/deploy.sh /www/releases/bitzorcas-<env>-<version>.zip <env>"
# Compat:
# ssh <user>@<server> "sudo BITZORCAS_INIT_NO_SEED=1 /www/scripts/deploy.sh /www/releases/bitzorcas-<env>-<version>.zip <env>"

# First demo environment (requires USER__*__PASSWORD):
# ssh <user>@<server> "sudo BITZORCAS_DEPLOY_DB_MODE=full-seed /www/scripts/deploy.sh /www/releases/bitzorcas-<env>-<version>.zip <env>"

deploy.sh flow: precheck → stop (Gateway→API→JobHost) → ports → backup → extract → write Gateway appsettings.Deploy.json → API schema (seeds depend on DEPLOY_DB_MODE) → JobHost --init-quartz-schema → systemd units → start (JobHost→API→Gateway) → health (journal dump on failure) → auto-rollback on failure → install server manifest.txt on success.

Schema: SQL Server catalog probe skips CodeFirst when complete; when only some tables/columns/indexes are missing, only affected models run InitTables; BITZORCAS_SCHEMA_FULL_INIT=1 or probe failure forces full CodeFirst. The probe checks object existence only, not column type/length.

Judging success

Logs: journalctl -u 'bitzorcas-*-<env>' -f

On a 1Panel host that needs a zero-downtime Gateway cutover, do not keep overwriting in place with deploy.sh. Wire locations to the named upstream as in 1Panel OpenResty blue-green, then:

# ① Wire the named upstream first. Rollback does not reverse schema.
sudo /www/scripts/setup-1panel-openresty.sh <env>
sudo BITZORCAS_DEPLOY_DB_MODE=schema-only \
  /www/scripts/deploy-blue-green.sh /www/releases/bitzorcas-<env>-<version>.zip <env>
sudo /www/scripts/deploy-rollback.sh <env>

Rollback

deploy.sh takes an incremental backup to /www/backups/<env>/ before each deploy and auto-rolls back on health-check failure (restore backup → re-run migrations → restart). Manual rollback:

# Restore the previous release, reconcile both schemas, and only then restart traffic.
sudo systemctl stop bitzorcas-gateway-<env> bitzorcas-api-<env> bitzorcas-jobhost-<env>
sudo tar -xzf /www/backups/<env>/backup_<timestamp>.tar.gz -C /www/apps/<env>
sudo dotnet /www/apps/<env>/api/BitzOrcas.Api.dll --init-schema
sudo dotnet /www/apps/<env>/jobhost/BitzOrcas.JobHost.dll --init-quartz-schema
sudo systemctl start bitzorcas-jobhost-<env> bitzorcas-api-<env> bitzorcas-gateway-<env>

docker-compose single-host deployment

For single-host scenarios that prefer reproducible environments over bare process management. The repo-root docker-compose.yml orchestrates only infrastructure (SQL Server / RabbitMQ / Redis / AgileConfig / Qdrant); the application stack is added by the deploy/docker-compose.app.yml overlay.

# ① Prepare config
cp .env.example .env                    # infrastructure credentials
cp deploy/app.env.example deploy/app.env # app config (License/DataProtection/CORS)

# ② Build three images
docker build -f deploy/container/Dockerfile        -t bitzorcas-api:local .
docker build -f deploy/container/JobHost.Dockerfile -t bitzorcas-jobhost:local .
docker build -f deploy/container/Gateway.Dockerfile -t bitzorcas-gateway:local .

# ③ Start (migrator services run first; api/jobhost/gateway start after they complete)
docker compose -f docker-compose.yml -f deploy/docker-compose.app.yml up -d

depends_on + condition: service_completed_successfully guarantees migrators finish before resident services. The first phase (community.small) uses ASPNETCORE_ENVIRONMENT=Development because Production is gated by RuntimeConfigurationGuard requiring the full production dependency set (see production-configuration).

Kubernetes deployment

For production scenarios needing multi-replica, autoscaling, and fault-domain isolation. Manifests live in deploy/kubernetes/:

# Fill every replace-with-* placeholder in configmap.yaml
# Create bitzorcas-api-runtime Secret + a separate license envelope Secret
kubectl apply -f deploy/kubernetes/namespace.yaml
kubectl apply -f deploy/kubernetes/configmap.yaml

# The migration Job must complete successfully before applying Deployments
kubectl apply -f deploy/kubernetes/migrate-job.yaml
kubectl wait --for=condition=complete job/bitzorcas-schema-migrator -n bitzorcas

kubectl apply -f deploy/kubernetes/api.yaml
kubectl apply -f deploy/kubernetes/jobhost.yaml
kubectl apply -f deploy/kubernetes/gateway.yaml

The full K8s contract is in deploy/kubernetes/README.md: runtime replicas never apply schema at startup, readOnlyRootFilesystem, deployment-id must be identical across replicas, and PII HMAC rotation does not support rolling deploys.

License transition

All three routes use the same procedure to switch from community.small (≤30 active natural-person users, no signed License) to a commercial License (full Platform capabilities):

1. Deploy BitzOrcas.LicenseSigner (a standalone intranet service holding the KMS/HSM private key, Bearer-authenticated only).
2. Add the trusted public key Licensing__Runtime__TrustedPublicKeys__<key-id> to the environment config.
3. Approve issuance via the LicenseSigner control plane with four-eyes approval; this produces runtime-license.json bound to the current deployment-id.
4. Switch Licensing__Runtime__PolicyId to the commercial policy and restart API and JobHost.

After the switch /health/license moves from LIC.COMMUNITY_SMALL to Valid and the 30-seat cap is lifted. See runtime license issuance.

Common failures

See also

• Local Aspire orchestration
• 1Panel OpenResty blue-green
• Database initialization and migrations
• Production configuration
• Monitoring and alerting
• Runtime license issuance
• OpenAPI and Scalar documentation surface