Container Orchestration
Containerised sinks are the recommended approach for cloud and Kubernetes environments. The official Helm chart includes sink deployments alongside the rest of the Nexomatic platform.
Deploying with Helm
Section titled “Deploying with Helm”The Nexomatic Helm chart (helm/nexomatic) includes sub-charts for the built-in sink node kinds. Each sub-chart deploys a Kubernetes Deployment that auto-scales based on CPU.
Prerequisites
Section titled “Prerequisites”- Kubernetes 1.24+ cluster
- Helm 3
- The Nexomatic Helm chart and its dependencies
- A node created on the Nodes screen for each sink deployment, with its Client ID and Client Secret (see Node Credentials)
Configure values
Section titled “Configure values”The sink sub-charts share a common configuration pattern. Environment variables come in two shapes: a plain env list (name/value), and the secret-backed envFromSecrets block — CLIENT_ID and CLIENT_SECRET are pulled from a Kubernetes Secret via secretKeyRef:
# Python sink (LOCAL:BASE)internal-python-sink: replicaCount: 2 image: repository: localhost:32000/nexomatic-sink-local tag: latest env: - name: HUB_HOST value: "nexomatic-hub" # Kubernetes service name - name: HUB_PORT value: "50051" - name: HUB_TLS value: "" # Empty for in-cluster plaintext - name: AUTH_METHOD value: "client_credentials" envFromSecrets: clientId: secret: nexomatic-sink-secrets key: client-id clientSecret: secret: nexomatic-sink-secrets key: client-secret
# PowerShell sink (LOCAL:POWERSHELL)internal-pwsh-sink: replicaCount: 1 image: repository: localhost:32000/nexomatic-sink-powershell-local tag: latest env: - name: HUB_HOST value: "nexomatic-hub" - name: HUB_PORT value: "50051" - name: HUB_TLS value: "" - name: AUTH_METHOD value: "client_credentials" envFromSecrets: clientId: secret: nexomatic-sink-secrets key: pwsh-client-id clientSecret: secret: nexomatic-sink-secrets key: pwsh-client-secretThe chart renders SERVER_URL itself (pointing at the release’s controller service) — any SERVER_URL entry you add to env is ignored.
Create the Kubernetes Secret
Section titled “Create the Kubernetes Secret”kubectl create secret generic nexomatic-sink-secrets \ --namespace nexomatic \ --from-literal=client-id='<python-sink-client-id>' \ --from-literal=client-secret='<python-sink-client-secret>' \ --from-literal=pwsh-client-id='<powershell-sink-client-id>' \ --from-literal=pwsh-client-secret='<powershell-sink-client-secret>'Deploy
Section titled “Deploy”helm upgrade --install nexomatic ./helm/nexomatic \ --namespace nexomatic \ --create-namespace \ -f values.yamlAzure Workload Identity (AKS)
Section titled “Azure Workload Identity (AKS)”If your sinks need to access Azure resources (Key Vault, Storage, etc.) and you are running in AKS, you can use Workload Identity instead of storing credentials as secrets.
internal-pwsh-sink: serviceAccount: azureWorkloadIdentity: "<managed-identity-client-id>" env: - name: AUTH_METHOD value: "entra_id" - name: AZURE_TENANT_ID value: "<azure-tenant-id>" - name: AZURE_CLIENT_ID value: "<managed-identity-client-id>" - name: AZURE_FEDERATED_TOKEN_FILE value: "/var/run/secrets/azure/tokens/azure-identity-token"Deploying with Docker Compose
Section titled “Deploying with Docker Compose”For smaller deployments or local development, Docker Compose is a convenient alternative.
# docker-compose.yml (sink only — assumes Hub is already running)services: sink-python: image: your-registry/nexomatic-sink-local:latest restart: unless-stopped environment: SERVER_URL: https://controller.example.com HUB_HOST: hub.example.com HUB_PORT: "50051" HUB_TLS: "true" CLIENT_ID: LOCAL:BASE CLIENT_SECRET: "${SINK_CLIENT_SECRET}" AUTH_METHOD: client_credentials
sink-powershell: image: your-registry/nexomatic-sink-powershell-local:latest restart: unless-stopped environment: SERVER_URL: https://controller.example.com HUB_HOST: hub.example.com HUB_PORT: "50051" HUB_TLS: "true" CLIENT_ID: LOCAL:POWERSHELL CLIENT_SECRET: "${PWSH_CLIENT_SECRET}" AUTH_METHOD: client_credentialsStore secrets in a .env file (not committed to source control):
SINK_CLIENT_SECRET=<secret>PWSH_CLIENT_SECRET=<secret>Start the sinks:
docker compose up -dScaling
Section titled “Scaling”Sinks are stateless and horizontally scalable. The Hub distributes tasks across all connected sinks that match the step’s placement and constraints.
- Kubernetes: Adjust
replicaCountor enable HPA (Horizontal Pod Autoscaler) in the Helm values. - Docker Compose: Use
docker compose up --scale sink-python=4.
Health Checks
Section titled “Health Checks”The sink containers serve HTTP health endpoints: the chart’s defaults are a liveness probe on /healthz and a readiness probe on /readyz, both on port 8006.
If a sink pod crashes, Kubernetes restarts it automatically. The Hub detects the disconnection and re-queues any in-flight tasks.
Environment Variables Reference
Section titled “Environment Variables Reference”| Variable | Required | Description |
|---|---|---|
SERVER_URL | (chart-rendered) | Controller API URL — the chart sets it to the release’s controller service |
HUB_HOST | Yes | Hub hostname (use the Kubernetes service name inside the cluster) |
HUB_PORT | Yes | Hub port (default 50051) |
HUB_TLS | Yes | true for TLS, empty for in-cluster plaintext |
CLIENT_ID | Yes | Node client id — via envFromSecrets.clientId |
CLIENT_SECRET | Yes | Node client secret — via envFromSecrets.clientSecret |
AUTH_METHOD | Yes | client_credentials, entra_id, google, github, or none |
MAX_LOG_BYTES | No | Maximum raw log size per task (default 5 MB) |
TASK_TIMEOUT | No | Seconds before a task is aborted (default 3600) |
BACKPRESSURE_THRESHOLD | No | Queue depth that triggers back-pressure signalling (default 10) |