这是indexloc提供的服务,不要输入任何密码
Skip to content

Remove afero. #1339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 7, 2022
Merged

Remove afero. #1339

merged 3 commits into from
Jun 7, 2022

Conversation

nathanhammond
Copy link
Contributor

@nathanhammond nathanhammond commented Jun 6, 2022

The afero experiment was interesting but we have discovered that the package has significant deficiencies and many ecosystem projects (e.g. https://github.com/adrg/xdg) do not currently support virtual file systems. As this is not a primary portion of what we're focusing on we don't at this time see ROI from responsibility for trying to push our corner of the ecosystem in this direction.

If we want to adopt something similar in the future we would consider https://github.com/avfs/avfs.

@vercel
Copy link

vercel bot commented Jun 6, 2022

@nathanhammond is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

@nathanhammond nathanhammond force-pushed the remove-afero branch 2 times, most recently from 74dc7cd to 42c0f21 Compare June 6, 2022 10:40
@vercel
Copy link

vercel bot commented Jun 6, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated
turbo-site ⬜️ Ignored (Inspect) Jun 7, 2022 at 4:20PM (UTC)

if dirEntry.IsDir() {
return nil
}

if excludeCount == 0 {
result.Add(path)
result.Add(root + path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to double-check this line, and the equivalent below. Maybe we need a note that root includes a trailing separator, and path does not include a leading separator? If we ever move to some globbing code that doesn't depend on fs.FS, I could see missing that as a mistake we might make.

Naïvely I would've expected this to be filepath.Join. Maybe it should be anyways?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, in an earlier iteration, I had the same hypothesis thatJoin with an empty string would do the trick. But no, Go strikes again!

Really though, since Join hits the entirety of Clean underneath the hood, we should keep it as string concatenation. I'll do a better job explaining this.

for _, file := range files {
// We don't need the handle, we don't need the error.
// We'll know if it errors because the tests will not pass.
// nolint:errcheck
fs.Create(file)
fs[file[1:]] = &fstest.MapFile{Mode: 0666}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment as to why we're stripping the leading character? I understand it's due to fs.FS not being rooted, but I think we should call it out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment in place and split to separate steps to make it more clear.

"nodejs-berry": "../../../examples/basic",
"nodejs-yarn": "../../../examples/basic",
"nodejs-pnpm": "../../../examples/with-pnpm",
"nodejs-npm": filepath.Join(cwd, "../../../examples/basic"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it's changing what we're testing. Previously we were passing in relative paths, and now we're passing in absolute paths. Have we changed what the production code is doing?

Copy link
Contributor Author

@nathanhammond nathanhammond Jun 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously afero's OsFs implicitly relied on the current working directory. Eventually it delegates down to Os.Open() and it just kind of happens to work even though it's not supposed to be a valid path inside of an fs.FS filesystem (and happily broke everything when I swapped in a "valid" fs.FS implementation). In fact, the leading path traversal is also not allowed for fs.FS filesystems, which afero.OsFs claims to be.

We're lucky, however, the place that consumes this code is already set up to do the right thing:

// Get the workspaces from the package manager.
workspaces, err := c.PackageManager.GetWorkspaces(rootpath)
if err != nil {
return fmt.Errorf("workspace configuration error: %w", err)
}
// We will parse all package.json's simultaneously. We use a
// wait group because we cannot fully populate the graph (the next step)
// until all parsing is complete
parseJSONWaitGroup := &errgroup.Group{}
for _, workspace := range workspaces {
relativePkgPath, err := filepath.Rel(rootpath, workspace)
if err != nil {
return fmt.Errorf("non-nested package.json path %w", err)
}
parseJSONWaitGroup.Go(func() error {
return c.parsePackageJSON(relativePkgPath)
})
}

workspaces coming back fully-qualified immediately get relative-ized to make sure that they're children, and from that point forward the workspaces are no longer used.

So yes, in this case we are changing what we're testing, but to correct behavior, and I traced through the code to ensure that the consuming location didn't care. This change serves to lock in the current state of the world.

That this combination of edge cases and what amount to bugs in afero worked to begin with astounds me. Like playing Minesweeper with 999 mines and randomly clicking your way to success.

@nathanhammond nathanhammond force-pushed the remove-afero branch 5 times, most recently from bf9f816 to 19da3ac Compare June 7, 2022 11:19
@nathanhammond nathanhammond requested a review from gsoltis June 7, 2022 11:24
Copy link
Contributor

@gsoltis gsoltis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whew. Thanks for tackling this.

@gsoltis gsoltis added the pr: automerge Kodiak will merge these automatically after checks pass label Jun 7, 2022
@kodiakhq kodiakhq bot merged commit 1cfaff0 into vercel:main Jun 7, 2022
@nathanhammond nathanhammond deleted the remove-afero branch June 7, 2022 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr: automerge Kodiak will merge these automatically after checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants