这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@suzhoum
Copy link
Contributor

@suzhoum suzhoum commented Jun 27, 2024

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@suzhoum
Copy link
Contributor Author

suzhoum commented Jun 27, 2024

/benchmark module=tabular preset=tabular_medium benchmark=tabular_full time_limit=1h

@suzhoum
Copy link
Contributor Author

suzhoum commented Jun 27, 2024

/benchmark module=tabular preset=tabular_medium benchmark=tabular_full time_limit=1h
Benchmark Output

Benchmark Test Result - Pass Evaluation Results Path: s3://autogluon-ci-benchmark/evaluation/tabular/upgrade_ray_2.3x
The dashboard website is: http://autogluon-staging.s3-website-us-west-2.amazonaws.com/benchmark-dashboard/upgrade_ray_2.3x/a6dcbcb8d5723275d05fe8baf0f23edf0745d6b0/index.html
Benchmark Output

Benchmark Test Result - Fail Evaluation Results Path: s3://autogluon-ci-benchmark/evaluation/tabular/upgrade_ray_2.3x
The dashboard website is: http://autogluon-staging.s3-website-us-west-2.amazonaws.com/benchmark-dashboard/upgrade_ray_2.3x/78a10468ab51121c5d16175c5df0ca582168da80/index.html

@github-actions
Copy link

Job PR-4302-82339a9 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-4302/82339a9/index.html

@suzhoum suzhoum marked this pull request as ready for review July 3, 2024 18:06
@suzhoum suzhoum requested a review from Innixma July 3, 2024 18:06
@suzhoum suzhoum changed the title [TEST] Ray upgrade Ray version upgrade Jul 3, 2024

def try_import_ray() -> ModuleType:
RAY_MAX_VERSION = "2.11.0" # sync with core/setup.py
RAY_MAX_VERSION = "2.40.0" # sync with core/setup.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.31 is the latest ray version, so this should be changed to "2.32"

core/setup.py Outdated
extras_require = {
"ray": [
"ray[default]>=2.10.0,<2.11", # sync with common/src/autogluon/common/utils/try_import.py
"ray[default]>=2.10.0,<2.40", # sync with common/src/autogluon/common/utils/try_import.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.31 is the latest ray version, so this should be changed to "2.32"

core/setup.py Outdated
],
"raytune": [
"ray[default,tune]>=2.10.0,<2.11", # sync with common/src/autogluon/common/utils/try_import.py
"ray[default,tune]>=2.10.0,<2.40", # sync with common/src/autogluon/common/utils/try_import.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.31 is the latest ray version, so this should be changed to "2.32"

@Innixma Innixma added this to the 1.2 Release milestone Jul 8, 2024

def try_import_ray() -> ModuleType:
RAY_MAX_VERSION = "2.11.0" # sync with core/setup.py
RAY_MAX_VERSION = "2.40.0" # sync with core/setup.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tabular_medium in AG-Bench I assume means medium_quality. medium_quality does not use Ray. We need to test best_quality which uses Ray.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's good to know. Let me run a benchmark with best_quality

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have rerun the benchmark with best_quality, and the benchmark evaluation looks fine. There were some errors from ray after the time limit has reached, and I have rerun the benchmark locally to verify, but it turns out that the error seems to be sporadic, and happens on master as well.

This is the error message in the python-core*.log

Error reporting task backlog information: GrpcUnavailable: RPC Error message: failed to connect to all addresses; last error: UNKNOWN: ipv4:172.31.3.111:41673: Failed to connect to remote host: Connection refused; RPC Error details:

@suzhoum suzhoum force-pushed the upgrade_ray_2.3x branch from 6628719 to 68ee339 Compare July 10, 2024 15:29
@suzhoum suzhoum added the run-multi-gpu Run multimodal multi-gpu tests label Jul 10, 2024
@suzhoum suzhoum force-pushed the upgrade_ray_2.3x branch from 39108cb to 1ad9ccc Compare July 10, 2024 18:03
@suzhoum suzhoum added the model list checked You have updated the model list after modifying multimodal unit tests/docs label Jul 10, 2024
@github-actions
Copy link

Job PR-4302-1ad9ccc is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-4302/1ad9ccc/index.html

@suzhoum suzhoum requested a review from Innixma July 11, 2024 18:06
Copy link
Contributor

@Innixma Innixma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for running the benchmark!

@suzhoum suzhoum merged commit 6513fb2 into autogluon:master Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model list checked You have updated the model list after modifying multimodal unit tests/docs run-multi-gpu Run multimodal multi-gpu tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants