这是indexloc提供的服务,不要输入任何密码
Skip to content

add logs command to turbo CLI #827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 42 commits into from
Closed

Conversation

chelkyl
Copy link
Contributor

@chelkyl chelkyl commented Mar 7, 2022

Replay logs of most recently run tasks sorted by task execution:
image

List all hashes found in cache folder:
image

Replay logs of specific task hashes:
image

Replay last run's logs sorted by task duration from longest to shortest:
image

Replay last run's logs sorted by task start time and showing all metadata:
image
As list:
image

Related to #822

TODO:

  • Add sort by start and end time
    • Start time uses the time value used for calculating duration in run.go
    • End time is calculated on demand from start + duration
    • If --all, default sort to start time instead of alnum
  • Refactor --list-format: rename to --include-metadata and allow multiple
    • If --list, output as csv with header. Otherwise, output as info log at end of replay, one line per requested metadata point
  • Include remote logs unnecessary because remote logs are fetched during run and it is impractical to fetch all remote logs just for this

chelkyl added 2 commits March 6, 2022 18:49
cleared with every run and repopulated even for cached tasks,
plain format with single hash per line
@vercel
Copy link

vercel bot commented Mar 7, 2022

@chelkyl is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

@vercel
Copy link

vercel bot commented Mar 7, 2022

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/vercel/turbo-site/7pfU5uTjoYPiEEUqaf1p9z6JzJdT
✅ Preview: https://turbo-site-git-fork-chelkyl-add-logs-command.vercel.sh

@weyert
Copy link
Contributor

weyert commented Mar 9, 2022

I am liking this idea :)

@chelkyl chelkyl marked this pull request as draft March 13, 2022 21:53
@chelkyl chelkyl marked this pull request as ready for review March 20, 2022 03:17
@chelkyl chelkyl marked this pull request as ready for review April 24, 2022 21:26
@chelkyl chelkyl requested a review from nathanhammond as a code owner April 24, 2022 21:26
@gsoltis
Copy link
Contributor

gsoltis commented Apr 25, 2022

@chelkyl can you say more about your use case for this command? I think we would like to avoid having hashes leak into the UX to the extent possible, so maybe there is another way to address the particular content you are interesting in replaying?

@chelkyl
Copy link
Contributor Author

chelkyl commented Apr 25, 2022

@gsoltis This command was to address dcherman's comment. The original request was for some way to view the logs that were hidden if you run with --output-logs=new-only. Over time, I expanded the scope to add some very minor task history querying capability, restrict output to the hashes you specify, show more data about the tasks, sort the output, and format the output.

I would understand if this isn't a compelling use case!

@gsoltis
Copy link
Contributor

gsoltis commented Apr 26, 2022

I can definitely see the utility in getting the logs for tasks that were run. I think the challenge is going to be in finding a good way to address them.

I like the idea of tracking recent runs. That could tie in nicely with eventual better management of the cache. I also like the idea of supplying the same arguments to turbo logs that were supplied to turbo run, along with the particular tasks of interest (web#build, api#test, etc.) to replay those logs, or an appropriate message on a cache miss.

@chelkyl
Copy link
Contributor Author

chelkyl commented Apr 27, 2022

Agreed! The hash seemed to be the best solution, especially because the cached log folders are named by their hash.
However, it would be great to support task names (in addition to hashes?), since I could include task globbing.

How do you think task names support should be implemented? My initial thought is to add another file like the last run log that is just a JSON file mapping task name to hash that is only added to during turbo run.

@gsoltis
Copy link
Contributor

gsoltis commented Apr 27, 2022

I think having each turbo run produce a manifest would be a good idea, and I think @nathanhammond was interested in something like that as well. Maybe the way to go about this is to start an RFC for what that file should look like, and then the logs command would read those to find log files?

@nathanhammond
Copy link
Contributor

Heya @chelkyl! @gsoltis is 100% correct about my opinion. I believe that each run should produce a unique identifier that can be used to go back and look up what happened. Conceptually, turbo run => ID => look up individual task logs and caches. In my imagination all of the identifiers would be opaque hashes, possibly excepting some (ignored) portion of the identifier for the run that encodes the start time, or a specified name. (Time would only be for monotonic sorting on a single box, not for absolute time. Any name would just be a human-readable prefix/suffix. Both would be ignored by the program.)

@chelkyl
Copy link
Contributor Author

chelkyl commented Apr 29, 2022

I see! I hadn't even considered uniquely identifying, storing, and isolating each turbo run call.
How would the id hash be generated? Would it be a hash on all of the tasks or their hashes for that run?

@gsoltis
Copy link
Contributor

gsoltis commented May 5, 2022

@chelkyl If we aren't using it for a cache hit, which I think we wouldn't be, you could use a uuid. A nice-to-have feature would be something easily time-sortable.

@mehulkar mehulkar added the area: logging Improvements to logging label Jan 11, 2023
@turbo-orchestrator turbo-orchestrator bot added area: docs Improvements or additions to documentation owned-by: turbopack labels Nov 9, 2023
@anthonyshew
Copy link
Contributor

This one's looking a little stale, folks. Seeing as we're nearly complete with our Rust port, I'm going to close this out as this would shortly end up being deleted code if it were to land.

@anthonyshew anthonyshew closed this Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: docs Improvements or additions to documentation area: logging Improvements to logging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants