Update model name in getting_started_with_genai_eval_sdk.ipynb #2180
Conversation
The preview version of the model name for Gemini 2.5 flash no longer works.
There was a problem hiding this comment.
Summary of Changes
Hello @jsondai, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request addresses a broken example in the getting_started_with_genai_eval_sdk.ipynb notebook by updating an outdated model name. The change ensures the notebook correctly uses the current gemini-2.5-flash model, allowing users to successfully run the GenAI evaluation SDK example.
Highlights
- Model Name Update: Updated the
modelparameter in thegetting_started_with_genai_eval_sdk.ipynbnotebook to use the stablegemini-2.5-flashmodel instead of the deprecatedgemini-2.5-flash-preview-05-20.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request correctly updates a deprecated preview model name for gemini-2.5-flash to its generally available version within the getting started notebook. The change is accurate and necessary. I have added one review comment to highlight a maintainability improvement opportunity by refactoring the hardcoded model names into constants, which would make future updates easier and less error-prone.
| @@ -284,7 +284,7 @@ | |||
| "})\n", | |||
| "\n", | |||
| "eval_dataset = client.evals.run_inference(\n", | |||
| " model=\"gemini-2.5-flash-preview-05-20\",\n", | |||
| " model=\"gemini-2.5-flash\",\n", | |||
There was a problem hiding this comment.
While this change correctly updates the model name, it highlights a maintainability issue with using hardcoded strings (often called "magic strings") for model identifiers.
This model name, gemini-2.5-flash, is also used on line 374. Similarly, gemini-2.0-flash is hardcoded on lines 364 and 538.
To improve maintainability and reduce the risk of errors during future updates, I recommend defining these model names as constants in a shared setup cell and reusing them throughout the notebook.
For example:
# In a setup cell (e.g., after the client is initialized)
GEMINI_2_5_FLASH = "gemini-2.5-flash"
GEMINI_2_0_FLASH = "gemini-2.0-flash"
# Then, use the constant in this cell
eval_dataset = client.evals.run_inference(
model=GEMINI_2_5_FLASH,
# ...
)Since this refactoring is outside the scope of the current diff, it could be addressed in a follow-up. This would make the notebook more robust and easier to maintain.
The preview version of the model name for Gemini 2.5 flash no longer works.
Description
Thank you for opening a Pull Request!
Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
CONTRIBUTINGGuide.CODEOWNERSfor the file(s).nox -s formatfrom the repository root to format).Fixes #<issue_number_goes_here> 🦕