这是indexloc提供的服务,不要输入任何密码
Skip to content

Disable linking. #1354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 9, 2022
Merged

Disable linking. #1354

merged 4 commits into from
Jun 9, 2022

Conversation

nathanhammond
Copy link
Contributor

@nathanhammond nathanhammond commented Jun 9, 2022

This is the simplest possible change to fix #1323 which addresses the root issue: turning off the hard linking behavior via function arguments.

This is an alternative to #1351.

@vercel
Copy link

vercel bot commented Jun 9, 2022

@nathanhammond is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

@vercel
Copy link

vercel bot commented Jun 9, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated
turbo-site ⬜️ Ignored (Inspect) Jun 9, 2022 at 9:55AM (UTC)

Copy link
Contributor Author

@nathanhammond nathanhammond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A tour of what is happening!

if !got {
t.Errorf("SameFile(%v, %v) got false, want true", aPath, dstAPath)
if got {
t.Errorf("SameFile(%v, %v) got true, want false", aPath, dstAPath)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This clause gets flipped because we always copy a new file.

if target != linkTarget {
t.Errorf("Readlink got %v, want %v", target, linkTarget)
}
target, err := os.Lstat(dstLinkPath)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blindly reading a link here will throw an error in a world where we always copy a file.

}
target, err := os.Lstat(dstLinkPath)
assert.NilError(t, err, "Lstat")
assert.Check(t, target.Mode().IsRegular(), "the cached file is a regular file")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead we want to make sure it is just a normal, boring file.

t.Errorf("Readlink got %v, want missing", target)
}
_, err = os.Lstat(dstBrokenLinkPath)
assert.ErrorIs(t, err, os.ErrNotExist)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This checks for approximately the same outcome. Except in this case the link is not present. This doesn't actually matter since we don't presently restore broken symlinks.

}
dstLstat, dstLstErr := os.Lstat(dstLinkPath)
assert.NilError(t, dstLstErr, "Lstat")
assert.Check(t, dstLstat.Mode().IsRegular(), "the cached file is a regular file")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still making sure it's all regular files.

fromFile, err := os.Open(from)
if err != nil {
if isSymlink {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we didn't previously error in the event of encountering a broken symlink this maintains that behavior.

We can revisit whether or not this is a good idea later, but this PR aims to maintain consistency.

@@ -41,7 +41,7 @@ func (f *fsCache) Fetch(target, hash string, _unusedOutputGlobs []string) (bool,
}

// Otherwise, copy it into position
err := fs.RecursiveCopyOrLinkFile(cachedFolder, target, fs.DirPermissions, true, true)
err := fs.RecursiveCopyOrLinkFile(cachedFolder, target, fs.DirPermissions, false, false)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This flips the toggles to say, "please don't try to link me."

@nathanhammond
Copy link
Contributor Author

(We can actively remove the linking branches in a followup PR, I started that way, but it's a bit more-invasive.)

@gsoltis gsoltis added the pr: on hold Pull requests that are "on hold" and should not be merged label Jun 9, 2022
@gsoltis
Copy link
Contributor

gsoltis commented Jun 9, 2022

The code changes look good, I think they do indeed do what is described. I think we need to get a handle on perf implications before merging though.

@kodiakhq kodiakhq bot merged commit b1210d5 into vercel:main Jun 9, 2022
@nathanhammond nathanhammond deleted the no-hard-links branch June 9, 2022 17:43
kodiakhq bot pushed a commit that referenced this pull request Jun 9, 2022
Reverts #1351

We don't need this PR, as #1354 supersedes it.
@gsoltis
Copy link
Contributor

gsoltis commented Jun 9, 2022

Ok, I inadvertently merged this, but then went and did the perf investigation.

I was unable to find significant perf differences across a range of workloads, so I'm good to leave this merged.

kodiakhq bot pushed a commit that referenced this pull request Jun 10, 2022
On #1354 Kodiak trolled us. This expands the blocking labels to ensure that the labels that look like they should be blocking are indeed blocking.
kodiakhq bot pushed a commit that referenced this pull request Aug 19, 2022
This brings back some behavior from before #1354. The filesystem cache will save all symlinks, and restore non-broken symlinks to files. Symlinks to directories continue to get restored as empty directories. Still TBD what we're going to do with broken symlinks in the filesystem cache, they currently are not restored.

For reference: the http cache saves and restores all symlinks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr: on hold Pull requests that are "on hold" and should not be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Caching doesn't seem to correctly bust when discarding git changes
2 participants