Skip to main content

Ingress Setup

After first installing the LlamaCloud helm chart into your kubernetes environment, you will be able to test the deployment immediately by port-forwarding the frontend server to your local machine using the following command:

kubectl --namespace <your-namespace> port-forward svc/llamacloud-frontend 3000:3000

Once that command is running, you will be able to visit the LlamaCloud UI at http://localhost:3000.

While this may be sufficient for initial testing of your deployment, you will eventually need to setup the an ingress when taking your deployment to production and leveraging LlamaCloud as an API based service.

Requirements​

  • A super-cool.domain
  • An ingress controller deployed in your kubernetes cluster

Helm Chart Configuration​

As of version 0.1.47, the LlamaCloud helm chart supports the ability to configure and create an ingress resource for your deployment. The chart supports the following configuration:

# values.yaml
ingress:
  enabled: true
  create: true
  labels: {}
  annotations: {}
  host: "" # your desired hostname
  scheme: https # or http
  tlsSecretName: "" # important if you are serving LlamaCloud over HTTPS
  ingressClassName: "" # different ingress controllers may require a different class name. if unset, the default ingress class will be used.

We recommend setting up the ingress resource using the helm chart configuration above.

Once your ingress endpoint is setup, you can visit the LlamaCloud UI at https://<your-domain-name>. And, you can check the status of your ingress resource:

kubectl -n <your-llamacloud-namespace> get ingress

# Example output
NAME                 CLASS    HOSTS                ADDRESS                              PORTS    AGE
llamacloud-ingress   nginx    llamacloud.example   <ip-or-load-balancer-address>        80, 443  10m

Ingress Route Details​

  • /api: route all incoming requests with a path prefix of /api to the Backend service
  • /: route all other requests to the Frontend service

TLS Configuration​

Depending on your ingress controller, you may need to add a TLS secret to your ingress resource. You can specify that in the .Values.ingress.tlsSecretName field. Currently, we only support a single host and a single TLS secret. For more information on TLS Secrets, see the Kubernetes Ingress documentation.

Self-Managed Ingress​

If you prefer to manage the ingress resource yourself, you can disable the ingress resource creation by setting ingress.create to false and then apply the following ingress resource to your cluster:

For information about configuring an ingress resource, see the Kubernetes Ingress documentation.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: llamacloud-nginx-ingress
  namespace: <your-llamacloud-namespace>
spec:
  ingressClassName: <your-ingress-class-name>
  rules:
  - http:
      paths:
      - backend:
          service:
            name: llamacloud-backend # or whatever your backend service is named
            port:
              number: 8000
        path: /api
        pathType: Prefix
      - backend:
          service:
            name: llamacloud-frontend # or whatever your frontend service is named
            port:
              number: 3000
        path: /
        pathType: Prefix

BACKEND_URL Configuration​

The BACKEND_URL environment variable is used by the Frontend service to know where to send /api requests to. When you self-manage your ingress resource, it is recommended to explicitly set this environment variable so the Frontend service doesn't have to rely on the internal NextJS proxy to route requests to the Backend service.

# (recommended) values.yaml
# Enabling ingress via the charts, but not creating the ingress resource yourself. The helm charts will automatically configure the BACKEND_URL environment variable for the Frontend service to point to the ingress host.
ingress:
  enabled: true
  create: false
  host: <your-domain-name>

# alternative values.yaml
frontend:
  extraEnvVariables:
    - name: BACKEND_URL
      value: "https://<your-domain-name>"

Common Issues​

  • When an ingress resource is created, sometimes the frontend service may not be able to resolve the certificate. Users may experience slowness or failure during the login flow if this happens and may see an UNABLE_TO_VERIFY_LEAF_SIGNATURE error in the frontend logs.
    • There are easy and hard ways to resolve this issue.
    • The easy way is to set NODE_TLS_REJECT_UNAUTHORIZED=0 in the Frontend deployment with .Values.frontend.extraEnvVariables. This will tell the Frontend to ignore the certificate error. One drawback of this approach is that it may not be accepted by your organization's security policies.
    • The harder way is enable the Frontend pod to resolve the certificate you used to sign the ingress host. You can do this by mounting the certificate into the Frontend pod.
  • If you do not have an automated DNS setup, you will need to provide the IP address of the ingress resource to your DNS provider.

If you require assistance with setting up an ingress resource, please reach out to us on support at llamaindex.ai and we will be happy to help you!