这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@shatfield4
Copy link
Collaborator

Pull Request Type

  • ✨ feat
  • 🐛 fix
  • ♻️ refactor
  • 💄 style
  • 🔨 chore
  • 📝 docs

Relevant Issues

resolves #628

What is in this change?

Describe the changes in this PR that are impactful to the repo.

  • Fix a bug where when embedding anything using a data connector (Youtube video transcript/Github repo) the vector cache would not delete the cache files

Additional Information

Add any other context about the Pull Request here that was not captured above.

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated
  • I have tested my code functionality
  • Docker build succeeds locally

@shatfield4 shatfield4 linked an issue Jan 20, 2024 that may be closed by this pull request
@review-agent-prime
Copy link

collector/utils/extensions/YoutubeTranscript/index.js

Instead of using forEach to concatenate the pageContent of each doc, you can use the map function to create an array of pageContent and then use the join function to concatenate them. This will reduce the number of iterations over the docs array from two to one, improving performance.
Create Issue
See the diff
Checkout the fix

    let content = docs.map(doc => doc.pageContent).join("");
git fetch origin && git checkout -b ReviewBot/Impro-jfaud94 origin/ReviewBot/Impro-jfaud94

Instead of using docs[0].metadata to get the metadata, you can destructure the metadata from the first element of the docs array. This will make the code more readable and easier to understand.
Create Issue
See the diff
Checkout the fix

    const { metadata } = docs[0];
git fetch origin && git checkout -b ReviewBot/Impro-x0xr8ll origin/ReviewBot/Impro-x0xr8ll

Instead of checking if the directory exists and then creating it if it doesn't, you can use fs.mkdirSync with the { recursive: true } option. This will create the directory if it doesn't exist, and do nothing if it does. This reduces unnecessary file system operations, improving performance.
Create Issue
See the diff
Checkout the fix

    fs.mkdirSync(outFolderPath, { recursive: true });
git fetch origin && git checkout -b ReviewBot/Impro-n7f1b3p origin/ReviewBot/Impro-n7f1b3p

frontend/src/components/Modals/MangeWorkspace/Documents/Directory/index.jsx

Instead of using item as the variable name in the map function, you can use a more descriptive name like folder. This will make the code more readable and easier to understand.
Create Issue
See the diff
Checkout the fix

    files.items.map(
      (folder, index) =>
        (folder.name === "custom-documents" ||
          (folder.type === "folder" && folder.items.length > 0)) && (
          <FolderRow
            key={index}
            item={folder}
            selected={isSelected(
              folder.id,
              folder.type === "folder" ? folder : null
            )}
            fetchKeys={fetchKeys}
            onRowClick={() => toggleSelection(folder)}
            toggleSelection={toggleSelection}
            isSelected={isSelected}
            setLoading={setLoading}
            setLoadingMessage={setLoadingMessage}
            autoExpanded={index === 0}
          />
        )
    )
git fetch origin && git checkout -b ReviewBot/Impro-7rsqa6r origin/ReviewBot/Impro-7rsqa6r

server/utils/files/purgeDocument.js

Instead of using fs.existsSync to check if the directory exists and then using fs.rmSync to delete it, you can use fs.rmSync with the { force: true } option. This will delete the directory if it exists, and do nothing if it doesn't. This reduces unnecessary file system operations, improving performance.
Create Issue
See the diff
Checkout the fix

    fs.rmSync(subFolderPath, { recursive: true, force: true });
git fetch origin && git checkout -b ReviewBot/Impro-u777ufk origin/ReviewBot/Impro-u777ufk

@timothycarambat timothycarambat merged commit 288ff0d into master Jan 22, 2024
@timothycarambat timothycarambat deleted the 628-bug-chunks-not-unembedding-after-embedding-subfolder-items branch January 22, 2024 21:03
cabwds pushed a commit to cabwds/anything-llm that referenced this pull request Jul 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] chunks not unembedding after embedding subfolder items

3 participants