这是indexloc提供的服务,不要输入任何密码
Skip to content

(refactor) add lockfile abstraction #1789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 31, 2022

Conversation

chris-olszewski
Copy link
Member

This PR introduces a new lockfile abstraction that will allow us to easily add more lockfile support to other package managers. It also

Notable changes:

  • Context now stores the lockfile interface instead of a yarn specific lockfile
  • PackageJson now only stores the package names as they appear in the lockfile, these will be mapped to their entries when the pruned lockfile is created
  • The initial implementation would cache the processed version of the yarn lockfile and attempt to use it. I did some light testing and I don't think this has a significant perf hit since the processing is straightforward.

From my understanding of Golang I think having Lockfile as an interface makes sense since each package manager has a different data layout for their lockfile e.g. pnpm has 6 top level entries and yarn's top level entries are the package entries themselves.

Testing:
The e2e tests cover yarn prune, but I also tested that prune produces the an identical lockfile before and after these changes on a repo created with npx create-turbo.

@vercel
Copy link

vercel bot commented Aug 26, 2022

@chris-olszewski is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

@chris-olszewski chris-olszewski force-pushed the add_lockfile_abstraction branch from 6609272 to 72513c1 Compare August 26, 2022 18:47
@chris-olszewski chris-olszewski marked this pull request as ready for review August 26, 2022 18:58
@chris-olszewski chris-olszewski requested a review from a team as a code owner August 26, 2022 18:58
Copy link
Contributor

@gaspar09 gaspar09 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good to me. Maybe @nathanhammond can provide more comments.

// ResolvePackage Given a package and version returns the key, resolved version, and if it was found
ResolvePackage(name string, version string) (string, string, bool)
// AllDependencies Given a lockfile key return all (dev/optional/peer) dependencies of that package
AllDependencies(key string) (map[string]string, bool)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the graph, resolved to a set.

import "io"

// Lockfile Interface for general operations that work accross all lockfiles
type Lockfile interface {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to maybe throw a graph library underneath this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in provide a way to access the contents as a DAG? Are there any favorite graph libraries in Go world?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, not as a DAG because npm allows cycles, but so that we're able to leverage basic graph algorithms for reduction. I think it'd be a worthwhile addition, but agreed not for this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment this interface only deals with data operations and no IO/marshalling. I went with an interface instead of a single struct with an instant for each impl as the contents of a lockfile differ greatly between package managers.

Tested pruning a yarn mono repo made with `create-turbo` and verified the lockfile produced was the same as before this commit.
Change so we build up the lockfile in memory as opposed to writing/reading an intermediate to disk.
This should allow for better testing of the lockfile marshalling logic
@chris-olszewski chris-olszewski force-pushed the add_lockfile_abstraction branch from 72513c1 to 9da959d Compare August 30, 2022 23:23
@chris-olszewski
Copy link
Member Author

Resolved most comments that were brought up. Few notes:

  • I haven't started using a graph library in the lockfile creation. I require a little more clarification about what would be desired from that
  • I haven't cut over to use go-yarnlock. I'd like to switch over to that library in another PR that includes some smoke testing for the yarn lockfile behavior

@vercel
Copy link

vercel bot commented Aug 31, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
turbo-site ✅ Ready (Inspect) Visit Preview Aug 31, 2022 at 8:34AM (UTC)

Copy link
Contributor

@nathanhammond nathanhammond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, the abstraction at Lockfile is going to make this so much nicer. Really looking forward to the followup PRs in this space, too.

@nathanhammond nathanhammond added the pr: automerge Kodiak will merge these automatically after checks pass label Aug 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr: automerge Kodiak will merge these automatically after checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants