Installation
Before you begin
Make sure the following conditions are met:
- A Kubernetes cluster with version 1.21 or newer is running. Learn how to install the Kubernetes tools.
- The
SuspendJob
feature gate is enabled. In Kubernetes 1.22 or newer, the feature gate is enabled by default. - (Optional) The
JobMutableNodeSchedulingDirectives
feature gate (available in Kubernetes 1.22 or newer) is enabled. In Kubernetes 1.23 or newer, the feature gate is enabled by default. - The kubectl command-line tool has communication with your cluster.
Kueue publishes metrics to monitor its operators. You can scrape these metrics with Prometheus. Use kube-prometheus if you don’t have your own monitoring system.
The webhook server in kueue uses an internal cert management for provisioning certificates. If you want to use a third-party one, e.g. cert-manager, follow these steps:
- Set
internalCertManagement.enable
tofalse
in config file. - Comment out the
internalcert
folder inconfig/default/kustomization.yaml
. - Enable
cert-manager
inconfig/default/kustomization.yaml
and uncomment all sections with ‘CERTMANAGER’.
Install a released version
To install a released version of Kueue in your cluster, run the following command:
VERSION=v0.3.1
kubectl apply -f https://github.com/kubernetes-sigs/kueue/releases/download/$VERSION/manifests.yaml
Add metrics scraping for prometheus-operator
Available in Kueue v0.2.1 and later
To allow prometheus-operator to scrape metrics from kueue components, run the following command:
kubectl apply -f https://github.com/kubernetes-sigs/kueue/releases/download/$VERSION/prometheus.yaml
Uninstall
To uninstall a released version of Kueue from your cluster, run the following command:
VERSION=v0.3.1
kubectl delete -f https://github.com/kubernetes-sigs/kueue/releases/download/$VERSION/manifests.yaml
Upgrading from 0.2 to 0.3
Upgrading from 0.2.x
to 0.3.y
is not supported because of breaking API
changes.
To install Kueue 0.3.y
, uninstall the older version first.
Install a custom-configured released version
To install a custom-configured released version of Kueue in your cluster, execute the following steps:
-
Download the release’s
manifests.yaml
file:VERSION=v0.3.1 wget https://github.com/kubernetes-sigs/kueue/releases/download/$VERSION/manifests.yaml
-
With an editor of your preference, open
manifests.yaml
. -
In the
kueue-manager-config
ConfigMap manifest, edit thecontroller_manager_config.yaml
data entry. The entry represents the default Kueue Configuration struct (v1beta1@main). The contents of the ConfigMap are similar to the following:
The
namespace
andinternalCertManagement
fields are available in Kueue v0.3.0 and later
apiVersion: v1
kind: ConfigMap
metadata:
name: kueue-manager-config
namespace: kueue-system
data:
controller_manager_config.yaml: |
apiVersion: config.kueue.x-k8s.io/v1beta1
kind: Configuration
namespace: kueue-system
health:
healthProbeBindAddress: :8081
metrics:
bindAddress: :8080
webhook:
port: 9443
manageJobsWithoutQueueName: true
internalCertManagement:
enable: true
webhookServiceName: kueue-webhook-service
webhookSecretName: kueue-webhook-server-cert
waitForPodsReady:
enable: true
timeout: 10m
The namespace
, waitForPodsReady
, and internalCertManagement
fields are available in Kueue v0.3.0 and later
Note See Sequential Admission with Ready Pods to learn more about using
waitForPodsReady
for Kueue.
- Apply the customized manifests to the cluster:
kubectl apply -f manifests.yaml
Install the latest development version
To install the latest development version of Kueue in your cluster, run the following command:
kubectl apply -k "github.com/kubernetes-sigs/kueue/config/default?ref=main"
The controller runs in the kueue-system
namespace.
Uninstall
To uninstall Kueue, run the following command:
kubectl delete -k "github.com/kubernetes-sigs/kueue/config/default?ref=main"
Build and install from source
To build Kueue from source and install Kueue in your cluster, run the following commands:
git clone https://github.com/kubernetes-sigs/kueue.git
cd kueue
IMAGE_REGISTRY=registry.example.com/my-user make image-local-push deploy
Add metrics scraping for prometheus-operator
Available in Kueue v0.2.0 and later
To allow prometheus-operator to scrape metrics from kueue components, run the following command:
make prometheus
Uninstall
To uninstall Kueue, run the following command:
make undeploy
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.