-
Notifications
You must be signed in to change notification settings - Fork 32
Open
Description
Currently when a machine is deleted, slurm step is interrupted without warning. However it would be great to send to all slurm steps within the machine a SIGINT such that they can run code to clean up (copy state into GCS for example)
Specially significant for Spot VMs.
- https://cloud.google.com/compute/docs/instances/create-use-spot#handle-preemption
- https://cloud.google.com/compute/docs/instances/spot#preemption-process
I have not been able to find wether slurm currently handles it well.
Metadata
Metadata
Assignees
Labels
No labels