Remove aryn-sdk from the sycamore repo. #1306
Conversation
The Aryn SDK has moved to https://github.com/aryn-ai/aryn-sdk. This changes the dependency to the public pypi release.
There was a problem hiding this comment.
Pull Request Overview
This PR removes the bundled local aryn-sdk module in favor of the public PyPI release and updates downstream code accordingly.
- Removed all local
lib/aryn-sdkcode and tests, switching to a PyPI dependency. - Root and subproject
pyproject.tomlfiles updated to referencearyn-sdk>=0.2.5. - Updated Sycamore unit tests to mock
httpx.streamand useiter_bytesinstead ofrequests.postanditer_content.
Reviewed Changes
Copilot reviewed 32 out of 32 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pyproject.toml | Removed local aryn-sdk path dependency |
| lib/sycamore/pyproject.toml | Pin aryn-sdk to public release with version constraint |
| lib/sycamore/sycamore/tests/unit/transforms/test_aryn_partitioner.py | Adjust mocks to use httpx.stream and iter_bytes |
| lib/aryn-sdk/** | Entire local SDK directory and tests deleted |
| self.status_code = 200 | ||
|
|
||
| def iter_content(self, chunksize): | ||
| def iter_bytes(self): |
There was a problem hiding this comment.
The iter_bytes method signature no longer matches typical streaming interfaces, which often accept parameters (e.g., chunk size). Consider adding *args, **kwargs or an optional chunksize parameter to ensure compatibility with the code under test.
Copilot uses AI. Check for mistakes.
| class TestArynPDFPartitioner: | ||
| def test_partition(self, mocker) -> None: | ||
| mocker.patch("requests.post", return_value=MockResponseNoTables()) | ||
| mocker.patch("httpx.stream").return_value.__enter__.return_value = MockResponseNoTables() |
There was a problem hiding this comment.
Patching httpx.stream at the top level may not intercept calls inside the module under test. It’s more robust to patch the symbol where it’s imported (e.g., patch('aryn_sdk.partition.partition.httpx.stream')) so the mock is applied correctly.
| mocker.patch("httpx.stream").return_value.__enter__.return_value = MockResponseNoTables() | |
| mocker.patch("sycamore.transforms.detr_partitioner.httpx.stream").return_value.__enter__.return_value = MockResponseNoTables() |
Copilot uses AI. Check for mistakes.
The Aryn SDK has moved to https://github.com/aryn-ai/aryn-sdk. This changes the dependency to the public pypi release.