Container Orchestration

import { Aside, Tabs, TabItem } from ‘@astrojs/starlight/components’;

Containerised sinks are the recommended approach for cloud and Kubernetes environments. The official Helm chart includes sink deployments alongside the rest of the Nexomatic platform.

---

Deploying with Helm

The Nexomatic Helm chart (helm/nexomatic) includes sub-charts for the built-in sink types. Each sub-chart deploys a Kubernetes Deployment that auto-scales based on CPU.

Prerequisites

• Kubernetes 1.24+ cluster
• Helm 3
• The Nexomatic Helm chart and its dependencies

Configure values

The sink sub-charts share a common configuration pattern. In your values.yaml (or a -f overrides.yaml file), set the environment variables for each sink:

# Python sink (LOCAL:BASE)
internal-python-sink:
  replicaCount: 2
  image:
    repository: your-registry/nexomatic-sink-local
    tag: latest
  env:
    - name: HUB_HOST
      value: "nexomatic-hub"        # Kubernetes service name
    - name: HUB_PORT
      value: "50051"
    - name: CLIENT_ID
      value: "LOCAL:BASE"
    - name: CLIENT_SECRET
      valueFrom:
        secretKeyRef:
          name: nexomatic-sink-secrets
          key: client-secret
    - name: TENANT_UUID
      value: ""                      # Leave empty for multi-tenant
    - name: TENANT_SECRET
      value: ""

# PowerShell sink (LOCAL:PWSH)
internal-pwsh-sink:
  replicaCount: 1
  image:
    repository: your-registry/nexomatic-sink-powershell-local
    tag: latest
  env:
    - name: HUB_HOST
      value: "nexomatic-hub"
    - name: HUB_PORT
      value: "50051"
    - name: CLIENT_ID
      value: "LOCAL:PWSH"
    - name: CLIENT_SECRET
      valueFrom:
        secretKeyRef:
          name: nexomatic-sink-secrets
          key: pwsh-client-secret

Create the Kubernetes Secret

kubectl create secret generic nexomatic-sink-secrets \
  --namespace nexomatic \
  --from-literal=client-secret='<python-sink-secret>' \
  --from-literal=pwsh-client-secret='<powershell-sink-secret>'

Deploy

helm upgrade --install nexomatic ./helm/nexomatic \
  --namespace nexomatic \
  --create-namespace \
  -f values.yaml

---

Azure Workload Identity (AKS)

If your sinks need to access Azure resources (Key Vault, Storage, etc.) and you are running in AKS, you can use Workload Identity instead of storing credentials as secrets.

internal-pwsh-sink:
  serviceAccount:
    azureWorkloadIdentity: "<managed-identity-client-id>"
  env:
    - name: AUTH_METHOD
      value: "entra_id"
    - name: AZURE_TENANT_ID
      value: "<azure-tenant-id>"
    - name: AZURE_CLIENT_ID
      value: "<managed-identity-client-id>"
    - name: AZURE_FEDERATED_TOKEN_FILE
      value: "/var/run/secrets/azure/tokens/azure-identity-token"

---

Deploying with Docker Compose

For smaller deployments or local development, Docker Compose is a convenient alternative.

# docker-compose.yml (sink only — assumes Hub is already running)
services:
  sink-python:
    image: your-registry/nexomatic-sink-local:latest
    restart: unless-stopped
    environment:
      HUB_HOST: hub.example.com
      HUB_PORT: "50051"
      CLIENT_ID: LOCAL:BASE
      CLIENT_SECRET: "${SINK_CLIENT_SECRET}"
      TENANT_UUID: "${TENANT_UUID}"
      TENANT_SECRET: "${TENANT_SECRET}"

  sink-powershell:
    image: your-registry/nexomatic-sink-powershell-local:latest
    restart: unless-stopped
    environment:
      HUB_HOST: hub.example.com
      HUB_PORT: "50051"
      CLIENT_ID: LOCAL:PWSH
      CLIENT_SECRET: "${PWSH_CLIENT_SECRET}"

Store secrets in a .env file (not committed to source control):

.env

SINK_CLIENT_SECRET=<secret>
PWSH_CLIENT_SECRET=<secret>
TENANT_UUID=<uuid>
TENANT_SECRET=<secret>

Start the sinks:

docker compose up -d

---

Scaling

Sinks are stateless and horizontally scalable. The Hub distributes tasks across all connected sinks of the matching type.

• Kubernetes: Adjust replicaCount or enable HPA (Horizontal Pod Autoscaler) in the Helm values.
• Docker Compose: Use docker compose up --scale sink-python=4.

Each replica registers independently with the Hub. More replicas means more concurrent task capacity. The Hub automatically balances work across all registered sinks of a given type.

---

Health Checks

The sink containers expose no HTTP endpoint. Kubernetes readiness and liveness are determined by the process being alive and the gRPC stream being established.

If a sink pod crashes, Kubernetes restarts it automatically. The Hub detects the disconnection and re-queues any in-flight tasks.

---

Environment Variables Reference

Variable	Required	Description
HUB_HOST	Yes	Hub hostname (use the Kubernetes service name inside the cluster)
HUB_PORT	Yes	Hub port (default 50051)
CLIENT_ID	Yes	Sink Type name
CLIENT_SECRET	Yes	Client secret
TENANT_UUID	No	Pin to a single tenant
TENANT_SECRET	No	Tenant registration secret
MAX_LOG_BYTES	No	Maximum raw log size per task (default 5 MB)
TASK_TIMEOUT	No	Seconds before a task is aborted (default 3600)
BACKPRESSURE_THRESHOLD	No	Queue depth that triggers back-pressure signalling (default 10)