# Self-host (K8s/Openshift)

import { Callout } from "zudoku/ui/Callout";
import { Stepper } from "zudoku/ui/Stepper";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "zudoku/ui/Tabs";

<Callout type="caution" title="Enterprise only!">
  Self-deployment of the Vulavula inference stack is intended for
  **enterprise** customers running the platform inside their own
  infrastructure. It is **not** the right path for most users — the managed
  [Vulavula API](https://api.lelapa.ai) is faster to integrate, fully
  maintained, and requires no cluster or GPU operations on your side.
</Callout>

<Callout type="caution" title="Please reach out first!">
  Before you attempt a self-deploy, contact us at
  [support@lelapa.ai](mailto:support@lelapa.ai) so we can assist with
  sizing, licensing, access to the private GHCR registry, and ongoing
  support. Deploying without that conversation is unsupported and will run
  into private-image auth, chart-version pinning, and feature-gate
  prerequisites you'll otherwise discover the hard way.
</Callout>

## What this is

The `your-cloud` Helm chart deploys the full Lelapa inference stack — the
same one powering the managed [Vulavula API](https://api.lelapa.ai) — into
your own Kubernetes cluster. You bring the cluster and GPU capacity; the
chart ships everything else needed to run speech-to-text and translation
workloads.

It front-loads a small FastAPI service that accepts your requests and
routes them to two GPU-backed predictor pools behind an internal
load-balancer. The two pools cover the two distinct workloads the API
supports: low-latency streaming and higher-throughput batch processing.
Models travel as OCI images pulled with the same credentials as the app
images — no separate S3 or model-registry plumbing on your side.

This chart does **not** expose anything outside the cluster or attach
authentication — bring your own Ingress / Gateway / LoadBalancer and auth
layer in front. See the chart's own `README.md`, `RUNBOOK.md`, and
`DESIGN.md` (shipped inside the chart tarball at `charts/your-cloud/`)
for the full, maintained reference. The steps below are a quickstart;
reach out to us first (see the callout above) to get access.

## Prerequisites

Before you start, confirm with us which `your-cloud` chart version and
which `their-cloud-mvp` image release tag you should pin — both are
private (GHCR `internal` visibility) and version advice is per-customer.

- **Kubernetes `ImageVolume` feature gate** (alpha since 1.31, off by
  default) — the predictor pods mount model weights straight out of an
  OCI image; there's no fallback path. Confirm the gate is actually on
  with whoever manages the cluster.
- **At least one GPU** in the cluster for inference. Set
  `inferenceServer.{live,sync}.nodePool.{selector,tolerations}` in your
  values file to match your actual GPU node's labels/taints.
- **A `lelapa-ghcr` image pull secret** in the target namespace — the
  app, runtime, and model weights images are all private (GHCR
  `internal`). Create it with a GitHub PAT that has `read:packages`
  scope (Step 6 below); the chart never provisions it itself.
- **Your own Ingress / Gateway + auth** in front of the chart's internal
  `:9000` service — the chart does not expose anything outside the
  cluster, terminate TLS, or attach authentication. Its only built-in
  auth is a Basic Auth pair on the service itself, not a substitute for
  real edge auth.

<Callout type="note" title="cert-manager and KServe">
  `cert-manager` and `KServe` are also required, but they're installed as
  part of the steps below — skip those steps if your cluster already has
  them.
</Callout>

## Install

<Stepper>

1. **Get a chart version and image tag from us**

   Email [support@lelapa.ai](mailto:support@lelapa.ai) for the
   `your-cloud` chart version (`X.Y.Z`) and the `their-cloud-mvp` image
   release tag (`YY.MM.<counter>`, e.g. `26.7.2`) you should pin. Both
   artifacts live under private GHCR packages.

2. **Install cert-manager** *(skip if your cluster already has it)*

   ```bash
   kubectl get ns cert-manager || {
     kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml
     kubectl wait --for=condition=Available --timeout=180s -n cert-manager \
       deployment/cert-manager deployment/cert-manager-cainjector deployment/cert-manager-webhook
   }
   ```

3. **Install KServe (CRDs + controller)** *(skip if already installed)*

   Two separate `helm install` calls — the CRD chart must land and become
   discoverable before the controller chart runs:

   ```bash
   helm upgrade --install kserve-crd oci://ghcr.io/kserve/charts/kserve-crd \
     --version v0.18.0 --namespace kserve --create-namespace

   helm upgrade --install kserve-resources oci://ghcr.io/kserve/charts/kserve-resources \
     --version v0.18.0 --namespace kserve \
     --set kserve.service.serviceClusterIPNone=true

   kubectl wait --for=condition=Available --timeout=180s -n kserve deployment/kserve-controller-manager
   ```

4. **Log in to the Lelapa Helm OCI registry**

   ```bash
   helm registry login ghcr.io -u <github-user> -p <github-token-with-read:packages>
   ```

5. **Configure your `my-values.yaml`**

   Minimum every install needs to touch:

   ```yaml
   namespace: <namespace>          # where you're deploying

   image:
     tag: "<X.Y.Z>"                # shared release tag for gateway/triton/models — pin an immutable release tag, not a rolling one

   inferenceServer:
      live:
        enabled: true
        nodePool:
          selector: {}              # match your actual GPU node labels (Step 1 prereq)
          tolerations: []
      sync:
        enabled: true
        nodePool:
          selector: {}
          tolerations: []

   secrets:
     mode: directValues             # existingSecret | directValues — directValues lets the chart render the Secret itself
     directValues: {}              # leave CLIENT_USERNAME/CLIENT_PASSWORD unset to auto-generate random Basic Auth on first install

   persistence:
     meteringDb:
       enabled: true
       storageClass: ""            # leave empty for the cluster default StorageClass; must be RWX-capable if adapters.replicas > 1
   ```

   To see the full set of available keys (or check what any of the above
   defaults to), render the default values straight from the published
   chart:

   ```bash
   helm show values oci://ghcr.io/lelapa-ai/charts/your-cloud --version <X.Y.Z>
   ```

6. **Create the `lelapa-ghcr` image pull secret**

   ```bash
   kubectl create secret docker-registry lelapa-ghcr \
     --docker-server=ghcr.io \
     --docker-username=<github-user> \
     --docker-password=<github-token-with-read:packages> \
     --namespace <namespace>
   ```

7. **Install `your-cloud`**

   Dry-run against the real cluster first — this catches CRD/webhook
   mismatches that `helm template` alone won't, since it actually talks to
   the API server:

   ```bash
   helm upgrade --install your-cloud oci://ghcr.io/lelapa-ai/charts/your-cloud \
     --version <X.Y.Z> \
     -f my-values.yaml \
     --namespace <namespace> --create-namespace \
     --dry-run=server
   ```

   Once that looks right, drop `--dry-run=server`:

   ```bash
   helm upgrade --install your-cloud oci://ghcr.io/lelapa-ai/charts/your-cloud \
     --version <X.Y.Z> \
     -f my-values.yaml \
     --namespace <namespace> --create-namespace
   ```

   **Match `--namespace` to whatever `namespace:` you set in
   `my-values.yaml`** — Helm tracks the release itself in whatever
   `--namespace` you pass here, independently of `.Values.namespace`
   governing where the chart's resources land. A mismatch doesn't break
   the install, but it does mean `helm list` / `helm uninstall` won't find
   the release where you expect it.

   The chart's preflight check (on by default; set
   `--set preflight.enabled=false` to bypass) fails the install immediately
   with a clear message if the KServe CRD isn't registered or — on
   upgrade — if the predictor Services aren't actually headless.

8. **Verify**

   ```bash
   kubectl get isvc,deploy,pods -n <namespace>
   kubectl get pods -n <namespace> -w   # watch until Running/Ready
   kubectl port-forward -n <namespace> svc/adapters 9000:9000
   curl localhost:9000/health
   ```

   Check both predictor pools are ready (KServe sets `.status.conditions`):

   ```bash
   kubectl get isvc -n <namespace> -o wide
   kubectl describe isvc inference-server-live -n <namespace>
   kubectl describe isvc inference-server-sync -n <namespace>
   ```

</Stepper>

<Callout type="info" title="External exposure and auth are your responsibility">
  This chart does **not** create an Ingress / Gateway / LoadBalancer or
  attach authentication in front of `adapters` (`:9000`, `ClusterIP`).
  Front it with your own, and terminate TLS there — the chart-rendered
  `adapters-secrets` Basic Auth pair travels in the clear otherwise.
</Callout>

## Update

```bash
helm diff upgrade your-cloud oci://ghcr.io/lelapa-ai/charts/your-cloud \
  --version <X.Y.Z> \
  -f my-values.yaml -n <namespace>     # requires the helm-diff plugin

helm upgrade --install your-cloud oci://ghcr.io/lelapa-ai/charts/your-cloud \
  --version <X.Y.Z> \
  -f my-values.yaml -n <namespace>
```

Same command for first install and every subsequent upgrade — `helm
upgrade --install` is idempotent. Bump only the chart `--version` to move
chart templates / schema / defaults; bump only `image.tag` in your values
file to move what code / model weights run. Tell us when you intend to do
either so we can confirm version compatibility.

## Rollback

```bash
helm history your-cloud -n <namespace>
helm rollback your-cloud <REVISION> -n <namespace>
```

Rollback reverts the chart's rendered manifests to a previous release, not
any data — the `metering-db` PVC (if enabled) isn't touched either way.

## Uninstall

```bash
helm uninstall your-cloud -n <namespace>
```

The `metering-db-pvc` PVC is not deleted automatically by Helm — remove
it manually if you actually want the data gone:

```bash
kubectl delete pvc metering-db-pvc -n <namespace>
```

## Troubleshooting

- **Predictor pod stuck `Pending`, no obvious reason** — almost always
  the `ImageVolume` feature gate, or a GPU node selector/toleration
  mismatch. Re-check your actual node labels against
  `inferenceServer.{live,sync}.nodePool.{selector,tolerations}`.
- **Predictor pod stuck `Init:0/1`** — the `models-image-tracker`
  initContainer is a trivial `command: ["true"]` that exists only so
  ArgoCD's image scanner sees the models image; if it's stuck, the actual
  model image pull is stuck. Check `image.models.tag` exists and
  `image.pullSecret` can read it.
- **Predictor pod `Running` but not serving** — check the
  `pod-liveness-guard` sidecar logs first. It force-deletes the pod after
  3 consecutive `/v2/health/live` failures; if it's cycling, the
  underlying Triton process is what to debug, not Kubernetes scheduling.
- **`adapters-secrets` missing / pods stuck `CreateContainerConfigError`**
  — with `secrets.mode: existingSecret`, the chart never creates this
  Secret; confirm whatever provisions it (ExternalSecret, SealedSecret,
  etc.) has actually run. With `mode: directValues` and
  `CLIENT_USERNAME`/`CLIENT_PASSWORD` left unset, the chart generates
  random ones on first install — retrieve them with:
  ```bash
  kubectl get secret adapters-secrets -n <namespace> -o jsonpath='{.data.CLIENT_USERNAME}' | base64 -d
  kubectl get secret adapters-secrets -n <namespace> -o jsonpath='{.data.CLIENT_PASSWORD}' | base64 -d
  ```
- **`LEAST_REQUEST` LB doesn't seem to be doing anything (all traffic
  hits one pod)** — almost always means the predictor Service isn't
  headless. Confirm
  `kubectl get svc <name>-predictor -n <namespace> -o jsonpath='{.spec.clusterIP}'`
  returns `None`; otherwise `kserve.service.serviceClusterIPNone` wasn't
  set on the KServe installation (see Step 3).
- **HPA not scaling `sync`** — expected. `inferenceServer.sync.
  autoscaling.maxReplicas` is pinned to `1` by design.

## Need help?

Self-deploying is something we want to support you through, not something
to figure out alone. Reach out at [support@lelapa.ai](mailto:support@lelapa.ai)
or join our [Discord community](https://discord.gg/QC6QJHasMP) and we'll
pair you with someone who knows the chart.