Run A Tekton Pipeline

This page shows how to leverage Kueue’s scheduling and resource management capabilities when running Tekton pipelines.

This guide is for batch users that have a basic understanding of Kueue. For more information, see Kueue’s overview.

We demonstrate how to support scheduling Tekton Pipelines Tasks in Kueue based on the Plain Pod integration, where every Pod from a Pipeline is represented as a single independent Plain Pod.

Before you begin

1. Learn how to install Kueue with a custom manager configuration.

2. Follow the steps in Run Plain Pods to learn how to enable and configure the pod integration.

3. Check Administrator cluster quotas for details on the initial Kueue step.

4. Your cluster has tekton pipelines installed.

Tekton Background

Tekton has the concept of Pipelines, Tasks and PipelineRun.

A pipeline consists of tasks. Tasks and pipelines must be created before running a pipeline.

A PipelineRun runs the pipeline.

A TaskRun runs a single task. PipelineRuns will reuse TaskRuns to run each task in a pipeline.

Tekton Defintions

As a simple example, we will define two tasks named sleep and hello:

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: sleep
spec:
  steps:
    - name: echo
      image: alpine
      script: |
        #!/bin/sh
        sleep 100        

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: hello
spec:
  params:
  - name: username
    type: string
  steps:
    - name: hello
      image: ubuntu
      script: |
        #!/bin/bash
        echo "Hello $(params.username)!"        

A pipeline composes these tasks.

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: kueue-test
spec:
  params:
  - name: username
    type: string
  tasks:
    - name: sleep
      taskRef:
        name: sleep
    - name: hello
      runAfter:
        - sleep
      taskRef:
        name: hello
      params:
      - name: username
        value: $(params.username)

a. Targeting a single LocalQueue

If you want every task to target a single local queue, it should be specified in the metadata.label section of the PipelineRun configuration.

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  generateName: kueue-test
  labels: 
    kueue.x-k8s.io/queue-name: my-local-queue
spec:
  pipelineRef:
    name: kueue-test
  params:
  - name: username
    value: "Tekton"

This will inject the kueue label on every pod of the pipeline. Kueue will gate the pods once you are over the quota limits.

Limitations

• Kueue will only manage pods created by Tekton.
• Each pod in a Workflow will create a new Workload resource and must wait for admission by Kueue.
• There is no way to ensure that a Workflow will complete before it is started. If one step of a multi-step Workflow does not have available quota, Tekton pipelines will run all previous steps and then wait for quota to become available.