θΏ™ζ˜―indexlocζδΎ›ηš„ζœεŠ‘οΌŒδΈθ¦θΎ“ε…₯任何密码
Skip to content

improve error messages from YT scraping #768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 21, 2024
Merged

improve error messages from YT scraping #768

merged 1 commit into from
Feb 21, 2024

Conversation

@timothycarambat
Copy link
Member

@timothycarambat timothycarambat commented Feb 21, 2024

Pull Request Type

  • ✨ feat
  • πŸ› fix
  • ♻️ refactor
  • πŸ’„ style
  • πŸ”¨ chore
  • πŸ“ docs

Relevant Issues

resolves #766

What is in this change?

Describe the changes in this PR that are impactful to the repo.

Additional Information

improve error messages from YT scraping when transcriptions are disabled
allow all youtube URLS and then parse the given URL and see if we can enforce it then instead of blocking all URLs that dont match the schema on input.

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated
  • I have tested my code functionality
  • Docker build succeeds locally

@timothycarambat
improve error messages from YT scraping
4dc580b 
parse & enforce URL to allow multiple URL schemas
@timothycarambat timothycarambat merged commit d896105 into master Feb 21, 2024
@review-agent-prime
Copy link

review-agent-prime bot commented Feb 21, 2024

collector/utils/extensions/YoutubeTranscript/index.js

Instead of returning the raw error message to the user, it would be better to log the error and return a generic error message. This will prevent potential information leakage.
Create Issue
See the diff
Checkout the fix

    const { docs, error } = await loader
      .load()
      .then((docs) => {
        return { docs, error: null };
      })
      .catch((e) => {
        console.error(e);
        return {
          docs: [],
          error: "An error occurred while loading the transcript.",
        };
      });

git fetch origin && git checkout -b ReviewBot/Impro-44upe1o origin/ReviewBot/Impro-44upe1o

@timothycarambat timothycarambat deleted the 766-better-yt-error-msg branch February 21, 2024 18:47
cabwds pushed a commit to cabwds/anything-llm that referenced this pull request Jul 3, 2025
@timothycarambat
improve error messages from YT scraping (Mintplex-Labs#768)
9de5e49 
parse & enforce URL to allow multiple URL schemas
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@review-agent-prime review-agent-prime[bot] review-agent-prime[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[FEAT]: Youtube collector - return better responses

2 participants

@timothycarambat