Kueue v1alpha1 API

Generated API reference documentation for kueue.x-k8s.io/v1alpha1.

Resource Types

CapacityProvider

Appears in:

CapacityProvider reports capacity for ResourceFlavors. DQO uses this capacity only when CapacitySynchronized=True.

FieldDescription
apiVersion
string
kueue.x-k8s.io/v1alpha1
kind
string
CapacityProvider
spec
CapacityProviderSpec

spec defines the desired state of the CapacityProvider.

status
CapacityProviderStatus

status defines the capacity observed and published by the provider.

DynamicQuotaOrchestrator

Appears in:

DynamicQuotaOrchestrator is the Schema for the dynamicquotaorchestrators API

FieldDescription
apiVersion
string
kueue.x-k8s.io/v1alpha1
kind
string
DynamicQuotaOrchestrator
spec
DynamicQuotaOrchestratorSpec

spec defines the desired state of the DynamicQuotaOrchestrator.

status
DynamicQuotaOrchestratorStatus

status defines the observed state of the DynamicQuotaOrchestrator.

PreemptionConfig

Appears in:

PreemptionConfig is the Schema for the preemptionconfigs API

FieldDescription
apiVersion
string
kueue.x-k8s.io/v1alpha1
kind
string
PreemptionConfig
spec
PreemptionConfigSpec

spec defines the preemption rules of the PreemptionConfig.

CapacityDiscovery

Appears in:

FieldDescription
providers [Required]
[]CapacityDiscoveryProviderContribution

providers lists CapacityProvider objects consumed by this DQO.

CapacityDiscoveryProviderContribution

Appears in:

FieldDescription
name [Required]
CapacityProviderName

name identifies a CapacityProvider.

effectiveCapacityMultiplier
k8s.io/apimachinery/pkg/api/resource.Quantity

effectiveCapacityMultiplier specifies the multiplier applied to the discovered capacity from this provider. It defaults to 1.

CapacityDistribution

Appears in:

FieldDescription
subtreeRootQuotaRef,omitzero [Required]
CapacityDistributionSubtreeRootRef

subtreeRootQuotaRef identifies the root of the quota subtree.

CapacityDistributionSubtreeRootRef

Appears in:

FieldDescription
kind [Required]
SubtreeRootRefKind

kind indicates the kind of the quota node, i.e. ClusterQueue or Cohort.

name [Required]
string

name indicates the name of the quota node, i.e. ClusterQueue or Cohort.

CapacityProviderControllerName

(Alias of string)

Appears in:

CapacityProviderName

(Alias of string)

Appears in:

CapacityProviderNormalizedCapacity

Appears in:

FieldDescription
flavors [Required]
[]CapacityProviderNormalizedCapacityFlavor

flavors reports capacity for each flavor and resource. The list must contain at least one flavor. It should contain the same flavors as spec.orchestratedFlavors. An empty resources map means zero capacity for all resources of the flavor.

CapacityProviderNormalizedCapacityFlavor

Appears in:

FieldDescription
name [Required]
ResourceFlavorReference

name identifies the ResourceFlavor whose capacity is reported.

resources [Required]
k8s.io/api/core/v1.ResourceList

resources contains total capacity by resource name. The provider orchestrates all resources of the flavor: a resource that is not listed, including when the map is empty, has zero capacity.

CapacityProviderOrchestratedFlavor

Appears in:

CapacityProviderOrchestratedFlavor identifies a flavor managed by a provider. The container allows the mapping to be extended in a future API version.

FieldDescription
name [Required]
ResourceFlavorReference

name identifies the ResourceFlavor managed by this provider.

CapacityProviderParametersReference

Appears in:

FieldDescription
apiGroup [Required]
string

apiGroup is the group for the resource being referenced.

kind [Required]
string

kind is the type of the resource being referenced.

name [Required]
string

name is the name of the resource being referenced.

CapacityProviderSpec

Appears in:

FieldDescription
orchestratedFlavors [Required]
[]CapacityProviderOrchestratedFlavor

orchestratedFlavors identifies the ResourceFlavors for which this provider may publish capacity. DQO ignores entries in status.capacity.flavors whose names are not listed here. The provider orchestrates all resources of the listed flavors: capacity that is not reported for a listed flavor is zero.

controllerName [Required]
CapacityProviderControllerName

controllerName identifies the controller publishing capacity. This field is immutable.

parameters
CapacityProviderParametersReference

parameters optionally references implementation-specific configuration. DQO does not read or validate the referenced object.

CapacityProviderStatus

Appears in:

FieldDescription
conditions
[]k8s.io/apimachinery/pkg/apis/meta/v1.Condition

conditions represents the current state of this provider.

capacity
CapacityProviderNormalizedCapacity

capacity is the normalized capacity published by the provider.

DynamicQuotaOrchestratorSpec

Appears in:

FieldDescription
capacityDiscovery,omitzero [Required]
CapacityDiscovery

capacityDiscovery specifies capacity aggregation.

capacityDistribution
CapacityDistribution

capacityDistribution specifies how aggregated capacity is distributed. When omitted, the DQO is discovery-only: it reports aggregated capacity but does not write effectiveQuotas status.

DynamicQuotaOrchestratorStatus

Appears in:

FieldDescription
conditions
[]k8s.io/apimachinery/pkg/apis/meta/v1.Condition

conditions represents the current state of the DQO.

effectiveCapacity
EffectiveCapacity

effectiveCapacity is the combined capacity from the referenced providers. Each provider's capacity is multiplied by its effectiveCapacityMultiplier. DQO clears this field if any referenced provider is missing or does not have CapacitySynchronized=True.

EffectiveCapacity

Appears in:

FieldDescription
flavors [Required]
[]EffectiveCapacityFlavor

flavors contains the total capacity for each flavor and resource. It includes all flavors listed in spec.orchestratedFlavors of the referenced providers, up to 128 distinct flavors. If the referenced providers together orchestrate more distinct flavors, the DQO sets EffectiveCapacityComputed=False with reason AggregationFailed and clears status.effectiveCapacity. An empty resources map means zero capacity for all resources of the flavor.

EffectiveCapacityFlavor

Appears in:

FieldDescription
name [Required]
ResourceFlavorReference

name identifies the ResourceFlavor whose capacity is reported.

resources [Required]
k8s.io/api/core/v1.ResourceList

resources contains total capacity by resource name. A resource that is not listed, including when the map is empty, has zero capacity for the flavor.

NumericComparison

(Alias of string)

Appears in:

NumericComparison defines how a specified numeric property (e.g., priority or custom numeric label value) of the candidate compares to the same property of the preemptor. Possible values are:

  • "LessThan": permits preemption if candidate field value < preemptor field value
  • "GreaterThan": permits preemption if candidate field value > preemptor field value
  • "LessThanOrEqual": permits preemption if candidate field value <= preemptor field value
  • "GreaterThanOrEqual": permits preemption if candidate field value >= preemptor field value

PreemptionConfigActivationPolicy

Appears in:

PreemptionConfigActivationPolicy defines when a preemption rule contributes candidates.

FieldDescription
trigger [Required]
PreemptionConfigActivationTrigger

trigger specifies the prerequisite for contributing candidates.

Possible values are:

  • Always: contributes matching candidates unconditionally.
  • InsufficientQuota: contributes matching candidates only if preempting baseline candidates does not yield sufficient quota to admit the preemptor workload.
  • QuotaFeasibleAndInsufficientTopology: contributes matching candidates only if quota is feasible for the entire preemptor under at least one eligible flavor assignment (after preempting baseline candidates and any candidates from InsufficientQuota rules), but the workload cannot be admitted because no eligible flavor assignment satisfies its topology requirements.

Baseline candidates are the deduplicated union of:

  • candidates selected by the preemptor's ClusterQueue.spec.preemption policy;
  • candidates selected by applicable rules in the referenced PreemptionConfig whose activationPolicy.trigger is Always.

PreemptionConfigActivationTrigger

(Alias of string)

Appears in:

PreemptionConfigActivationTrigger specifies when preemption rule should be treated as active.

PreemptionConfigNumericLabelConstraint

Appears in:

PreemptionConfigNumericLabelConstraint describes the rule for filtering a custom numerical label. For example, this can be used to filter candidates based on the label describing the required topology domain size, such as the "number of TPUs". If a user has a label "number-of-tpus" that describes the number of TPUs required in a single cube, it can be used to create a rule that selects only workloads requiring smaller cube slices by defining comparison: "LessThan". Such a configuration would allow preemption of "smaller" workloads, to achieve better cluster utilization and decrease fragmentation. Please note that those labels are not copied out of the box from job-like objects. You should remember to append the designated labels to the list of labels copied to the workload via the Kueue main configuration if you wish to use a custom label. As Kubernetes label values cannot start with '-', integer labels are always non-negative. A negative fallbackValue can thus ensure workloads without the label compare smaller than any labeled workload if this is desired. If neither comparison, minValue, nor maxValue are specified, the constraint checks only that candidate workloads possess the designated label key with a valid integer.

FieldDescription
key [Required]
string

key is the label key that stores the integer value in the workload that will be used for candidate selection.

fallbackValue
int32

fallbackValue is used when a workload does not have the label key or the value under the key cannot be parsed as an integer. If not specified, workloads without the label or with a label value not parsable as int are treated as incomparable, and therefore excluded from preemption candidates.

comparison
NumericComparison

comparison defines how the candidate's label value compares to the preemptor's.

minValue
int32

minValue specifies the lowest label value a candidate workload can have to be considered for preemption. If not specified, no lower bound is enforced.

maxValue
int32

maxValue specifies the highest label value a candidate workload can have to be considered for preemption. If not specified, no upper bound is enforced.

PreemptionConfigPreemptionCandidateSelector

Appears in:

PreemptionConfigPreemptionCandidateSelector defines the selection criteria for workloads that are candidates for preemption.

FieldDescription
scope [Required]
PreemptionConfigPreemptionQueueScope

scope specifies the queue or cohort relation boundary of candidates to the preemptor workload.

clusterQueueSelector
k8s.io/apimachinery/pkg/apis/meta/v1.LabelSelector

clusterQueueSelector defines label selector constraints on candidate ClusterQueues. Accepts all if not set.

labelSelector
k8s.io/apimachinery/pkg/apis/meta/v1.LabelSelector

labelSelector defines label selector constraints on candidate Workloads. Accepts all if not set.

numericLabels
[]PreemptionConfigNumericLabelConstraint

numericLabels defines rules for filtering candidates using custom numeric labels on the Workload resource. Multiple numeric labels are joined using AND-rule (all have to be satisfied). Accepts all if not set.

priority
PreemptionConfigPriorityConstraint

priority defines the requirements for the priority of candidates. Workloads not matching those requirements will not be considered as preemption candidates. If nil, no priority requirements are enforced.

PreemptionConfigPreemptionQueueScope

(Alias of string)

Appears in:

PreemptionConfigPreemptionQueueScope specifies the relational boundary between the preempting workload's queue and candidate workloads' queues. Possible values are:

  • "WithinLocalQueue": restricts preemption candidates to workloads submitted to the exact same LocalQueue (matching name and namespace).
  • "WithinClusterQueue": restricts preemption candidates to workloads submitted to the same ClusterQueue as the preemptor.
  • "WithinParentCohort": restricts preemption candidates to workloads in ClusterQueues that share the exact same immediate direct Cohort, as well as workloads in the preemptor's own ClusterQueue (even if standalone).
  • "WithinCohortTree": restricts preemption candidates to workloads in ClusterQueues that belong to the same Cohort Tree (sharing the same root ancestor Cohort), as well as workloads in the preemptor's own ClusterQueue (even if standalone).
  • "AnyClusterQueue": places no relationship restrictions on preemption candidates.

PreemptionConfigPreemptionRule

Appears in:

PreemptionConfigPreemptionRule defines a single rule under which preemptions can be triggered and the candidate workloads eligible for preemption.

FieldDescription
name [Required]
string

name is the identifier of the preemption rule.

preemptorSelector
k8s.io/apimachinery/pkg/apis/meta/v1.LabelSelector

preemptorSelector is a label selector indicating which workloads can trigger preemptions using this rule. Accepts all workloads if not set.

activationPolicy,omitzero [Required]
PreemptionConfigActivationPolicy

activationPolicy determines when this rule contributes matching candidates to preemption evaluation.

candidateSelectors
[]PreemptionConfigPreemptionCandidateSelector

candidateSelectors specifies the selection rules for workloads that are candidates for preemption. Candidates resulting from multiple selectors are summed into one set. No selectors result in an empty candidate set, thereby disallowing any preemptions with this rule.

PreemptionConfigPriorityConstraint

Appears in:

PreemptionConfigPriorityConstraint defines the requirements for the priority of preemption candidates.

FieldDescription
mode [Required]
PreemptionConfigPriorityMode

mode specifies whether priority comparison uses base or boosted (effective) priority.

comparison [Required]
NumericComparison

comparison defines how the candidate's priority compares to the preemptor's priority. For example, "LessThan" means that only workloads with lower priority will be allowed as preemption candidates.

PreemptionConfigPriorityMode

(Alias of string)

Appears in:

PreemptionConfigPriorityMode defines whether base or boosted (effective) priority is used when comparing candidates against the preemptor. Possible values are:

  • "Base": uses the raw priority value as assigned in the Workload resource (spec.priority) for both the candidate and preemptor, ignoring any priority boost.
  • "Boosted": uses the effective priority value, adjusted by the priority boost mechanism (if enabled), for both the candidate and preemptor.

PreemptionConfigSpec

Appears in:

PreemptionConfigSpec defines the desired state of PreemptionConfig

FieldDescription
rules
[]PreemptionConfigPreemptionRule

rules specifies preemption candidate selection rules.

ResourceFlavorReference

(Alias of string)

Appears in:

TODO: once we graduate this API to beta, we should leverage beta API's ResourceFlavorReference type instead of Alpha one. ResourceFlavorReference is the name of the ResourceFlavor.

SubtreeRootRefKind

(Alias of string)

Appears in: