Problem & Background

We have a deployment of ~70 functions and deploy them as part of CI at once. Whenever we deploy a random set of functions fails to deploy due to "You have exceeded your deployment quota". This used to block us completely a few weeks ago when there still was a quota on build time. Since this has been moved to Cloud Build as far as I can tell there are no more limits and you simply pay for your build time. Which is awesome, no more blocking the build due to quotas!

Unfortunately there still seems to be some other quotas/rate limits that are causing the deploy to fail. From what I can tell these might be simply too many write requests against the Cloud Functions API (or perhaps cloud build?).

The error message suggests to deploy with --only - Unfortunately it is not easy for us to split these up into separately deployed functions. It is also impossible to do when a dependency is updated or we change a data model or utility library that is used by different functions. Analyzing which function changed and has to be redeployed is not possible (for us) automatically and has to be done manually. This then brings new pains, where automated deploys after code review become impossible.

Right now this means our pipelines all fail regularily - and we manually retry them until every functions deployed successfully once.

Suggestion

Since retrying the failed functions right after always works, my suggestion would be to retry failed deploys that hit code 8 here:

I suppose this could be implemented here:

If this would be a way forward I'm happy to create a PR for this.