Container Orchestration

import { Aside, Tabs, TabItem } from ‘@astrojs/starlight/components’;

Containerised sinks are the recommended approach for cloud and Kubernetes environments. The official Helm chart includes sink deployments alongside the rest of the Nexomatic platform.

Deploying with Helm

The Nexomatic Helm chart (helm/nexomatic) includes sub-charts for the built-in sink types. Each sub-chart deploys a Kubernetes Deployment that auto-scales based on CPU.

Prerequisites

Kubernetes 1.24+ cluster
Helm 3
The Nexomatic Helm chart and its dependencies
A configured Sink Type with its Client ID and Client Secret for each sink deployment (see Sink Types & Configuration)

Configure values

The sink sub-charts share a common configuration pattern. In your values.yaml (or a -f overrides.yaml file), set the environment variables for each sink:

# Python sink (LOCAL:BASE)
internal-python-sink:
  replicaCount: 2
  image:
    repository: your-registry/nexomatic-sink-local
    tag: latest
  env:
    - name: SERVER_URL
      value: "https://controller.example.com"
    - name: HUB_HOST
      value: "nexomatic-hub"        # Kubernetes service name
    - name: HUB_PORT
      value: "50051"
    - name: HUB_TLS
      value: ""                      # Empty for in-cluster plaintext
    - name: CLIENT_ID
      value: "LOCAL:BASE"
    - name: CLIENT_SECRET
      valueFrom:
        secretKeyRef:
          name: nexomatic-sink-secrets
          key: client-secret
    - name: AUTH_METHOD
      value: "client_credentials"
    - name: TENANT_UUID
      value: ""                      # Leave empty for multi-tenant
    - name: TENANT_SECRET
      value: ""

# PowerShell sink (LOCAL:PWSH)
internal-pwsh-sink:
  replicaCount: 1
  image:
    repository: your-registry/nexomatic-sink-powershell-local
    tag: latest
  env:
    - name: SERVER_URL
      value: "https://controller.example.com"
    - name: HUB_HOST
      value: "nexomatic-hub"
    - name: HUB_PORT
      value: "50051"
    - name: HUB_TLS
      value: ""
    - name: CLIENT_ID
      value: "LOCAL:PWSH"
    - name: CLIENT_SECRET
      valueFrom:
        secretKeyRef:
          name: nexomatic-sink-secrets
          key: pwsh-client-secret
    - name: AUTH_METHOD
      value: "client_credentials"

Create the Kubernetes Secret

kubectl create secret generic nexomatic-sink-secrets \
  --namespace nexomatic \
  --from-literal=client-secret='<python-sink-secret>' \
  --from-literal=pwsh-client-secret='<powershell-sink-secret>'

Deploy

helm upgrade --install nexomatic ./helm/nexomatic \
  --namespace nexomatic \
  --create-namespace \
  -f values.yaml

Azure Workload Identity (AKS)

If your sinks need to access Azure resources (Key Vault, Storage, etc.) and you are running in AKS, you can use Workload Identity instead of storing credentials as secrets.

internal-pwsh-sink:
  serviceAccount:
    azureWorkloadIdentity: "<managed-identity-client-id>"
  env:
    - name: AUTH_METHOD
      value: "entra_id"
    - name: AZURE_TENANT_ID
      value: "<azure-tenant-id>"
    - name: AZURE_CLIENT_ID
      value: "<managed-identity-client-id>"
    - name: AZURE_FEDERATED_TOKEN_FILE
      value: "/var/run/secrets/azure/tokens/azure-identity-token"

Deploying with Docker Compose

For smaller deployments or local development, Docker Compose is a convenient alternative.

# docker-compose.yml (sink only — assumes Hub is already running)
services:
  sink-python:
    image: your-registry/nexomatic-sink-local:latest
    restart: unless-stopped
    environment:
      SERVER_URL: https://controller.example.com
      HUB_HOST: hub.example.com
      HUB_PORT: "50051"
      HUB_TLS: "true"
      CLIENT_ID: LOCAL:BASE
      CLIENT_SECRET: "${SINK_CLIENT_SECRET}"
      AUTH_METHOD: client_credentials
      TENANT_UUID: "${TENANT_UUID}"
      TENANT_SECRET: "${TENANT_SECRET}"

  sink-powershell:
    image: your-registry/nexomatic-sink-powershell-local:latest
    restart: unless-stopped
    environment:
      SERVER_URL: https://controller.example.com
      HUB_HOST: hub.example.com
      HUB_PORT: "50051"
      HUB_TLS: "true"
      CLIENT_ID: LOCAL:PWSH
      CLIENT_SECRET: "${PWSH_CLIENT_SECRET}"
      AUTH_METHOD: client_credentials

Store secrets in a .env file (not committed to source control):

SINK_CLIENT_SECRET=<secret>
PWSH_CLIENT_SECRET=<secret>
TENANT_UUID=<uuid>
TENANT_SECRET=<secret>

Start the sinks:

docker compose up -d

Scaling

Sinks are stateless and horizontally scalable. The Hub distributes tasks across all connected sinks of the matching type.

Kubernetes: Adjust replicaCount or enable HPA (Horizontal Pod Autoscaler) in the Helm values.
Docker Compose: Use docker compose up --scale sink-python=4.

Health Checks

The sink containers expose no HTTP endpoint. Kubernetes readiness and liveness are determined by the process being alive and the gRPC stream being established.

If a sink pod crashes, Kubernetes restarts it automatically. The Hub detects the disconnection and re-queues any in-flight tasks.

Environment Variables Reference

Variable	Required	Description
`SERVER_URL`	Yes	Controller API URL
`HUB_HOST`	Yes	Hub hostname (use the Kubernetes service name inside the cluster)
`HUB_PORT`	Yes	Hub port (default `50051`)
`HUB_TLS`	Yes	`true` for TLS, empty for in-cluster plaintext
`CLIENT_ID`	Yes	Sink Type identifier
`CLIENT_SECRET`	Yes	Client secret for the Sink Type
`AUTH_METHOD`	Yes	`client_credentials`, `entra_id`, `google`, `github`, or `none`
`TENANT_UUID`	No	Pin to a single tenant
`TENANT_SECRET`	No	Tenant registration secret
`MAX_LOG_BYTES`	No	Maximum raw log size per task (default 5 MB)
`TASK_TIMEOUT`	No	Seconds before a task is aborted (default 3600)
`BACKPRESSURE_THRESHOLD`	No	Queue depth that triggers back-pressure signalling (default 10)