Multi-Cluster Guide¶

Multi-cluster blueprints place pods in separate Kubernetes clusters. The WoSP tunnel spans the cluster boundary using LoadBalancer IPs for pod-to-pod communication.

What makes multi-cluster different¶

In a single-cluster blueprint, pods communicate using Kubernetes DNS (service.namespace.svc.cluster.local). In a multi-cluster blueprint, there is no shared Kubernetes control plane — pods in different clusters cannot resolve each other's DNS names. Instead, they communicate via external LoadBalancer IPs that are known at deploy time.

This means:

You must deploy the processor/sink cluster before the gateway cluster
LoadBalancer IPs must be assigned before the gateway manifest is applied
For cloud deployments, pre-provision static IPs to avoid a circular dependency

Supported blueprints¶

Blueprint	Clusters	Protocol
Fast International Ferry	2	WebSocket
International Ferry	2	gRPC

Deploy order: processor first, gateway second¶

The gateway cluster's Envoy configuration references the sink/processor's LoadBalancer IP. If you deploy the gateway first, its Envoy configuration points to an IP that doesn't exist yet and the pods will fail to connect.

Always deploy in this order: 1. Deploy processor/sink cluster → wait for LoadBalancer IP to be assigned 2. Record the LoadBalancer IP 3. Deploy gateway cluster (using the IP from step 2)

Local k3d setup¶

Why k3d clusters can't talk by default¶

Two separate k3d clusters run in separate Docker bridge networks. Pods in cluster A cannot reach pods in cluster B via their cluster-internal IPs.

Solution: host.k3d.internal + port mappings¶

k3d >= 5.0 resolves host.k3d.internal to the Docker host IP from inside any container. By mapping specific host ports to each cluster's LoadBalancer at creation time, pods in cluster A can reach pods in cluster B via host.k3d.internal:<port>.

# Create clusters with port mappings (must be set at creation — cannot be added later)
k3d cluster create cluster-a \
  --port "30100:30100@loadbalancer" \
  --agents 1

k3d cluster create cluster-b \
  --port "30200:30200@loadbalancer" \
  --agents 1

Deploy processor cluster first¶

# Deploy cluster-b (processor/sink)
kubectl config use-context k3d-cluster-b
bash deploy-cluster-b.sh

# Verify LoadBalancer is listening on port 30200
curl -v http://host.k3d.internal:30200/health

# Deploy cluster-a (gateway) — now that cluster-b's address is known
kubectl config use-context k3d-cluster-a
bash deploy-cluster-a.sh

Verify cross-cluster connectivity¶

Check the gateway auto-trigger:

kubectl logs -n <blueprint>-initiator-ns deployment/initiator -c web-app -f

Expected: 🔁 Auto-trigger complete — 5/5 messages sent.

Confirm the result trail contains pod names from both clusters:

curl http://localhost:8000/output

The "trail" in each result should list pods from both cluster A and cluster B.

Cloud deployment¶

Pre-provision static IPs¶

Cloud LoadBalancer IPs are assigned dynamically, which creates a circular dependency: you need the IP to configure the gateway manifest, but the IP isn't assigned until after you deploy.

Resolve this by pre-provisioning static IPs before running any kubectl apply:

GKE: Reserve a static regional IP in each cluster's region
EKS: Pre-create an NLB with a fixed hostname
AKS: Reserve a static public IP in the resource group

Update the LoadBalancer annotations in 02-secrets.yaml or the service manifest to request the pre-provisioned IP, then deploy the processor cluster first, confirm the static IP is attached, and proceed with the gateway cluster.

Deploy order on cloud¶

# 1. Deploy processor cluster
kubectl --context cluster-b apply -f sink/
# Wait for LoadBalancer IP: kubectl --context cluster-b get svc -A --watch

# 2. Deploy gateway cluster
kubectl --context cluster-a apply -f gateway/

Both deploy.sh scripts accept a --cluster argument to target the correct context. Refer to the blueprint README for the exact commands.