Skip to main content

Ingress Setup

After first installing the LlamaCloud helm chart into your kubernetes environment, you will be able to test the deployment immediately by port-forwarding the frontend server to your local machine using the following command:

kubectl --namespace <your-namespace> port-forward svc/llamacloud-frontend 3000:3000

Once that command is running, you will be able to visit the LlamaCloud UI at http://localhost:3000.

While this may be sufficient for initial testing of your deployment, you will eventually need to setup the an ingress to allow for external connections into the frontend and backend LlamaCloud services.

The LlamaCloud helm-chart explicitly leaves ingress setup out of scope of the helm chart itself. This is to accommodate the variety of ingress setups that any individual self-hosted LlamaCloud deployment may require.

However, for most deployments we recommend a simple ingress setup using ingress-nginx with three routing rules:

  • /api: route all incoming requests with a path prefix of /api to the backend service
  • /api/deployment: routes a specific endpoint hosted by frontend server for exposing runtime configurations to frontend UI.
  • /: route all other requests to the frontend service

The following template can be used as a reference for setting up such an ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: llamacloud-nginx-ingress
  namespace: default
spec:
  ingressClassName: nginx-internal
  rules:
  - http:
      paths:
      - backend:
          service:
            name: llamacloud-frontend
            port:
              number: 3000
        path: /api/deployment
        pathType: Exact
      - backend:
          service:
            name: llamacloud-backend
            port:
              number: 8000
        path: /api
        pathType: Prefix
      - backend:
          service:
            name: llamacloud-frontend
            port:
              number: 3000
        path: /
        pathType: Prefix

Once your ingress endpoint is setup, you can either connect to it directly or using the DNS name you setup for it, instead of using the port-forwarding approach mentioned earlier for initial testing.