-
Notifications
You must be signed in to change notification settings - Fork 2k
Remove afero. #1339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove afero. #1339
Conversation
@nathanhammond is attempting to deploy a commit to the Vercel Team on Vercel. A member of the Team first needs to authorize it. |
74dc7cd
to
42c0f21
Compare
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
9a46e46
to
158f819
Compare
cli/internal/globby/globby.go
Outdated
if dirEntry.IsDir() { | ||
return nil | ||
} | ||
|
||
if excludeCount == 0 { | ||
result.Add(path) | ||
result.Add(root + path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to double-check this line, and the equivalent below. Maybe we need a note that root
includes a trailing separator, and path
does not include a leading separator? If we ever move to some globbing code that doesn't depend on fs.FS
, I could see missing that as a mistake we might make.
Naïvely I would've expected this to be filepath.Join
. Maybe it should be anyways?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, in an earlier iteration, I had the same hypothesis thatJoin
with an empty string would do the trick. But no, Go strikes again!
Really though, since Join
hits the entirety of Clean
underneath the hood, we should keep it as string concatenation. I'll do a better job explaining this.
cli/internal/globby/globby_test.go
Outdated
for _, file := range files { | ||
// We don't need the handle, we don't need the error. | ||
// We'll know if it errors because the tests will not pass. | ||
// nolint:errcheck | ||
fs.Create(file) | ||
fs[file[1:]] = &fstest.MapFile{Mode: 0666} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a comment as to why we're stripping the leading character? I understand it's due to fs.FS
not being rooted, but I think we should call it out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comment in place and split to separate steps to make it more clear.
"nodejs-berry": "../../../examples/basic", | ||
"nodejs-yarn": "../../../examples/basic", | ||
"nodejs-pnpm": "../../../examples/with-pnpm", | ||
"nodejs-npm": filepath.Join(cwd, "../../../examples/basic"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like it's changing what we're testing. Previously we were passing in relative paths, and now we're passing in absolute paths. Have we changed what the production code is doing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previously afero
's OsFs
implicitly relied on the current working directory. Eventually it delegates down to Os.Open()
and it just kind of happens to work even though it's not supposed to be a valid path inside of an fs.FS
filesystem (and happily broke everything when I swapped in a "valid" fs.FS
implementation). In fact, the leading path traversal is also not allowed for fs.FS
filesystems, which afero.OsFs
claims to be.
We're lucky, however, the place that consumes this code is already set up to do the right thing:
turborepo/cli/internal/context/context.go
Lines 156 to 175 in 5ad04a3
// Get the workspaces from the package manager. | |
workspaces, err := c.PackageManager.GetWorkspaces(rootpath) | |
if err != nil { | |
return fmt.Errorf("workspace configuration error: %w", err) | |
} | |
// We will parse all package.json's simultaneously. We use a | |
// wait group because we cannot fully populate the graph (the next step) | |
// until all parsing is complete | |
parseJSONWaitGroup := &errgroup.Group{} | |
for _, workspace := range workspaces { | |
relativePkgPath, err := filepath.Rel(rootpath, workspace) | |
if err != nil { | |
return fmt.Errorf("non-nested package.json path %w", err) | |
} | |
parseJSONWaitGroup.Go(func() error { | |
return c.parsePackageJSON(relativePkgPath) | |
}) | |
} |
workspaces
coming back fully-qualified immediately get relative-ized to make sure that they're children, and from that point forward the workspaces
are no longer used.
So yes, in this case we are changing what we're testing, but to correct behavior, and I traced through the code to ensure that the consuming location didn't care. This change serves to lock in the current state of the world.
That this combination of edge cases and what amount to bugs in afero
worked to begin with astounds me. Like playing Minesweeper with 999 mines and randomly clicking your way to success.
7df949b
to
e315a8e
Compare
bf9f816
to
19da3ac
Compare
19da3ac
to
58fc24f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whew. Thanks for tackling this.
The
afero
experiment was interesting but we have discovered that the package has significant deficiencies and many ecosystem projects (e.g. https://github.com/adrg/xdg) do not currently support virtual file systems. As this is not a primary portion of what we're focusing on we don't at this time see ROI from responsibility for trying to push our corner of the ecosystem in this direction.If we want to adopt something similar in the future we would consider https://github.com/avfs/avfs.