What's Changed

Key New Features 🎉

• downloading libnccl2 and libnccl-dev for a3u and a4h by @rachit-google in downloading libnccl2 and libnccl-dev for a3u and a4h #4680
• Add NUMA-aware scheduling in GKE clusters (enabled for G4) by @kadupoornima in Add NUMA-aware scheduling in GKE clusters (enabled for G4) #4739

Breaking Changes 🚨

• Allowing setting use_job_duration with non-exclusive partitions. by @arpit974 in Allowing setting use_job_duration with non-exclusive partitions. #4696
• Add multi-network support in TPU v6e by @agrawalkhushi18 in Add multi-network support in TPU v6e #4723
• Update vpc and cloud_router versions in VPC network module by @kadupoornima in Update vpc and cloud_router versions in VPC network module #4732

Module Improvements 🔨

• Refactoring in gke persistent module by @vikramvs-gg in Refactoring in gke persistent module #4618
• Migrate Kueue installation to use Helm chart by @shubpal07 in Migrate Kueue installation to use Helm chart #4542

Improvements 🛠

• Update nvidia DRA driver version to v25.3.0 by @parulbajaj01 in Update nvidia DRA driver version to v25.3.0 #4670
• Updated A3-mega and A4-high Slurm blueprints to adopt nvidia add repository scirpt. by @rachit-google in Updated A3-mega and A4-high Slurm blueprints to adopt nvidia add repository scirpt. #4667
• Update H4D blueprint: disable automatic updates, provide image info, and delete duplicate filestore by @Neelabh94 in Update H4D blueprint: disable automatic updates, provide image info, and delete duplicate filestore #4644
• Add Managed Lustre support in gke-a4 by @parulbajaj01 in Add Managed Lustre support in gke-a4 #4654
• Add Managed Lustre support in gke a3 ultra by @parulbajaj01 in Add Managed Lustre support in gke a3 ultra #4700
• Adds an irdma health check to h4d nodes by @samskillman in Adds an irdma health check to h4d nodes #4704
• Enable Spot VM Provisioning For H4D by @LAVEEN in Enable Spot VM Provisioning For H4D #4735
• Add slurm-gke blueprint by @ACW101 in Add slurm-gke blueprint #4607

Version Updates ⏫

• Updating google provider version upper bound to 7.6.0 - latest version by @bytetwin in Updating google provider version upper bound to 7.6.0 - latest version #4736
• Update GKE cluster and firewall rules module versions to use recent google provider by @kadupoornima in Update GKE cluster and firewall rules module versions to use recent google provider #4737

Bug fixes 🐞

• Remove superfluous addition of chs logs to cloud ops config by @abbas1902 in Remove superfluous addition of chs logs to cloud ops config #4679
• Adding "datacenter-gpu-manager-4-dev" as an additional installation in A* YAML files. by @Neelabh94 in Adding "datacenter-gpu-manager-4-dev" as an additional installation in A* YAML files. #4623
• minor bug fix on MFT version comparison by @ljqg in minor bug fix on MFT version comparison #4689
• Fix inconsistent plan on Slurm cluster reconfigure by @wiktorn in Fix inconsistent plan on Slurm cluster reconfigure #4538
• Update process to filter out starting comments in a source yaml file by @SwarnaBharathiMantena in Update process to filter out starting comments in a source yaml file #4707
• Fix gke build failures by @annuay-google in Fix gke build failures #4708
• Update machine-leaning/a3-ultragpu-8g/nemo-framework to fix segmentation fault error by @SwarnaBharathiMantena in Update machine-leaning/a3-ultragpu-8g/nemo-framework to fix segmentation fault error #4725

New Contributors

• @mufaqam-gcl made their first contribution in Add mufaqam-gcl to cluster-toolkit-writers.json #4688
• @wtempel made their first contribution in Update README.md #4705
• @nikosavola made their first contribution in Fix typo in examples/README.md for N2 description #4720
• @ACW101 made their first contribution in Add slurm-gke blueprint #4607

Full Changelog: v1.67.0...v1.68.0