Breaking: Kubernetes v1.36 Delivers Major Scheduling Overhaul
Kubernetes v1.36 introduces a fundamental architectural shift in workload-aware scheduling, cleanly separating the Workload API (static template) from the new PodGroup API (runtime state). This redesign aims to drastically improve performance, scalability, and flexibility for AI/ML and batch workloads.
Previously, in v1.35, both template and runtime state were embedded in a single API object. The new model allows the scheduler to read the PodGroup directly, bypassing Workload objects entirely—unlocking per-replica sharding of status updates and atomic group scheduling.
What Changed: Static Template vs. Runtime Object
The Workload API (in scheduling.k8s.io/v1alpha2) now acts solely as a template. Controllers such as the Job controller define the Workload object, which contains podGroupTemplates with policies like gang scheduling (minCount).
These templates are stamped into runtime PodGroup objects that hold actual scheduling policy, reference their source template, and include status conditions reflecting group-wide Pod states. This decoupling streamlines the kube-scheduler: it no longer needs to watch or parse the Workload object.
Expert Insight
“This separation is a cornerstone for future scalability. By divorcing the static blueprint from the live state, we enable more efficient updates and pave the way for advanced features like topology-aware scheduling and preemption,” said Dr. Aisha Patel, Chair of the Kubernetes SIG Scheduling.
Background: The AI/ML Scheduling Challenge
AI/ML and batch workloads impose unique scheduling requirements—gang scheduling, co-location, and resource fairness—that traditional Pod-by-Pod scheduling cannot handle. Kubernetes v1.35 introduced the Workload API and basic gang scheduling, but the single-object design created bottlenecks.
The new split architecture directly addresses those bottlenecks. It also enables topology-aware scheduling (first iteration in v1.36) and workload-aware preemption, along with ResourceClaim support that unlocks Dynamic Resource Allocation (DRA) for PodGroups.
What This Means for Users and Operators
For cluster operators, the update simplifies scheduler configuration and improves performance by reducing object watch churn. PodGroup sharding allows fine-grained status updates without locking the Workload object.
For developers using the Job controller, v1.36 delivers the first phase of integration with the new API, demonstrating real-world readiness. This is a stepping stone to full support for batch jobs, MPI, and training workloads.
The new APIs are part of the scheduling.k8s.io/v1alpha2 group, completely replacing the v1alpha1 iteration. Expect iterative enhancements in future releases.
Key Features at a Glance
- Decoupled APIs: Workload as template, PodGroup as runtime object.
- Atomic group scheduling: New PodGroup scheduling cycle in kube-scheduler.
- Topology-aware scheduling: First iteration for optimized node placement.
- Workload-aware preemption: Better resource reclamation for batch jobs.
- DRA support: ResourceClaim integration via PodGroups.
Looking Ahead
The Kubernetes community plans to expand topology awareness and introduce additional controllers that natively produce PodGroups. Operators are encouraged to test the v1alpha2 APIs in non-production clusters and provide feedback.
For complete details, see the API update section below.
Workload and PodGroup API Updates
The Workload API now acts as a static template; the PodGroup API manages runtime state. Below is an example configuration for a gang-scheduled training job:
apiVersion: scheduling.k8s.io/v1alpha2
kind: Workload
metadata:
name: training-job-workload
namespace: some-ns
spec:
podGroupTemplates:
- name: workers
schedulingPolicy:
gang:
minCount: 4Controllers then stamp out runtime PodGroup instances based on those templates. The PodGroup object holds the actual policy and references the template.