bugfix: use correct page count, feature: fileworker integration with Trieve#2770

Closed

cdxker wants to merge 28 commits intomainfrom

cd/file-worker-integration

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bugfix: use correct page count, feature: fileworker integration with Trieve#2770

bugfix: use correct page count, feature: fileworker integration with Trieve#2770
cdxker wants to merge 28 commits intomainfrom
cd/file-worker-integration

Commits on Nov 12, 2024

feature: add file chunker microservice

feature: add logging and write to clickhouse

feature: added api key and file chunking status

feature: add CLI to test pdf chunking

Commits on Nov 15, 2024

cleanup: add .env.dist + use env's for model in pdf_chunk.rs

feat: use pdla boxes for chunking

cleanup: clippy

feat: add docker-compose.yml for local testing

feature: pdf to md :)

feature: add page right after its been processed

refactor: complete rename from file-chunker to pdf2md

bugfix: include snippet to handle loading env's at runtime instead of compile

feature: docker setup for pdf2md service

feature: add CI workflow to push images for pdf2md service

feature: enable chunk bulk delete as an async operation where chunks matching delete filter can be inserted safely after resp

Commits on Nov 16, 2024

feature: skeleton for pdf2md demo page

feature: working CSS for demo-ui on pdf2md

feature: finished navbar for pdf2md

Commits on Nov 18, 2024

feature: skeleton upload form for pdf2md

feature: add health check + fix redoc

feature: add skeleton template others can slot into

feature: add api key functionality to demo-ui app

feature: working API req on file form add

Commits on Nov 19, 2024

feature: add advanced options and usage metrics to each chunk

cleanup: remove creating a file on the file_handler

feature: setup fileworker to communicate with pdf2md

bugfix: correct page count using redis

feature: update to new route name

bugfix: use correct page count, feature: fileworker integration with Trieve#2770cdxker wants to merge 28 commits intomaindevflowinc/trieve:mainfrom cd/file-worker-integrationdevflowinc/trieve:cd/file-worker-integrationCopy head branch name to clipboard

Commits

Commits on Nov 12, 2024

Commits on Nov 15, 2024

Commits on Nov 16, 2024

Commits on Nov 18, 2024

Commits on Nov 19, 2024

bugfix: use correct page count, feature: fileworker integration with Trieve#2770
cdxker wants to merge 28 commits intomainfrom
cd/file-worker-integration