这是indexloc提供的服务,不要输入任何密码
Skip to content

Self Hosted Remote Cache no longer works with AWS S3 presigned URLs starting in 1.13.4 #9177

@puchm

Description

@puchm

Verify canary release

  • I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/puchm/turborepo-presigned-url-repro

What package manager are you using / does the bug impact?

npm, pnpm, Yarn v1, Yarn v2/v3/v4 (node_modules linker only)

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

2.1.3-canary.1

Describe the Bug

Starting in version 1.13.4 there is an issue where AWS S3 presigned URLs return a 501 (Not Implemented) exception for PUT requests when used as a remote cache.

This can apparently be caused by a missing Content-Length header, although I wasn't able to determine if something changed in that regard.

Expected Behavior

The way this should work is as follows:

  • Turbo makes preflight request to AWS Lambda (possibly through an API Gateway)
  • Presigned URL is generated and returned to Turbo in the Location header
  • A presigned URL allows whoever has the URL to read / write from a given key in the storage bucket without any further authorization. It is valid for a limited amount of time which in the case of the reproduction is 1 hour.
  • Turbo makes PUT / GET request towards the presigned URL
  • The error occurs in this last request to the presigned URL

This setup should work and given that it already worked before 1.13.4 I think it's fair to say that there was some kind of regression that caused this.

To Reproduce

The reproduction is a modification of this repository: https://github.com/EloB/turborepo-remote-cache-lambdagit

I put a guide at the top of the README: https://github.com/puchm/turborepo-presigned-url-repro

Remember to delete the AWS resources again after debugging the issue :)

I think a good place to start would be to compare the HTTP requests going out in 1.13.3 (where it works) to those in 1.13.4 (where it stopped working). Maybe there actually is a difference in the headers like Stackoverflow suggests. I was unfortunately not able to find my way around the Rust code and couldn't look at the headers either since I was also not able to intercept the requests - but someone who knows Rust should be able to do so pretty easily.

Additional context

I tested in different canary releases. This was not a problem in any of the 1.13.4 canary releases. It occured between 1.13.4-canary.5 and 1.13.4.

Tagging @arlyon because you worked on remote cache stuff between these two versions, maybe you have an idea what caused this? :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind: bugSomething isn't workingneeds: triageNew issues get this label. Remove it after triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions