-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
While it is documented that cleanup is enabled by default, I did not expect resources that existed in my cluster before running skaffold to be removed. Perhaps it should be considered: If the render step uses remoteManifests, should cleanup default to false? Is there some other way to avoid a stock behavior that is destructive to resources not created by skaffold?
Steps to reproduce:
- Have a k8s cluster with some resource (deployment) already in it
- Setup skaffold.yaml with a
manifests
section referring to the existing resource - run
skaffold dev
(in my case there was some error - so it didn't even fully stabilize before cleaning up!) - Observe your pre-existing resource is deleted.
manifests:
remoteManifests:
- manifest: some-namespace:deployment/some-deployment-name
kubeContext: default
Build [someregistry.io/somerepository] succeeded
WARN[0062] k8s/*.yaml did not match any file subtask=-1 task=DevLoop
Tags used in deployment:
- someregistry.io/somerepository -> someregistry.io/somerepository:0873d67e-dirty@sha256:d..1
Starting deploy...
- deployment.apps/some-deployment-name configured
Waiting for deployments to stabilize...
- some-namespace:deployment/some-deployment-name: container some-ctr-name is waiting to start: someregistry.io/somerepository:0873d67e-dirty@sha256:d..1 can't be pulled
(repeated 2x)
Cleaning up...
- deployment.apps "some-deployment-name" deleted
1/1 deployment(s) failed
So in my case an image pulling issue took place, but the issue is in the default cleanup behavior, which is perhaps unexpected when the manifests are not coming skaffold's control.
Commentary on use case:
We don't use Skaffold for actual deployments, but want to use it for some triaging and rapid testing during development with remote clusters. So the cluster had resources deployed via Helm, and the idea is to use skaffold to "take over" the deployment object you're developing against. Works great! However, since the deployment itself is deployed with a very different tool (for other reasons, the helm chart cannot be used as the skaffold render step in this case - it is managed somewhere else with complex logic for it's values, etc).
I'll be okay knowing that I need --cleanup=false
but this might be worth a consideration.