8000 Implement a retry mechanism for Google GenAI calls by xpomul · Pull Request #15783 · eclipse-theia/theia · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Implement a retry mechanism for Google GenAI calls #15783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Implement a retry mechanism for Google GenAI calls #15783

wants to merge 3 commits into from

Conversation

xpomul
Copy link
Contributor
@xpomul xpomul commented Jun 6, 2025 •
edited
Loading

What it does

See #15718:

Google imposes rate limits for its LLMs. Especially with lower tiers (including the free tier - see https://ai.google.dev/gemini-api/docs/rate-limits?hl=de ), it can quickly happen that an agent with tool calls (such as the Coder agent) hits the Requests Per Minute (RPM) rate limit and the agent execution terminates with an error.
Also, in longer conversations, it can happen that the LLM does not return proper JSON which leads to an error in the GenAI API client, or reports 500 Internal Server Errors occasionally.

To make the Google LanguageModel implementation in Theia more robust against these errors, a retry mechanism is now implemented that can resend the last request after a configurable delay in case of an error.

The retry mechanism can be configured using preferences:

  • maxRetriesOnErrors configures the maximum number of retries per request after which to give up. Defaults to 3. If smaller than 1, then the retry logic is disabled.
  • retryDelayOnRateLimitError configures the delay in seconds to wait in case of a rate limit error. Defaults to 60 (1 minute). If negative, then no retry is attempted and the error is propagated.
  • retryDelayOnOtherErrors configures the delay in seconds to wait in case of any other error. Defaults to -1 (disabled). If negative, then no retry is attempted and the error is propagated.

How to test

  • Get a Google GenAI API key for the free tier.
  • Set up Theia with that key and configure the Coder agent to use google/gemini-2.0-flash
  • Give a complex instruction to the Coder agent (e.g. with the Theia codebase loaded, ask it something like
@Coder Change the Breakpoint widget so that multi-selection of breakpoints is possible.

This means that the TreeWidget of the Breakpoint view should be changed to support multiSelect. If multiple breakpoints are selected, then in the context menu, there should be menu items:

* Enable all selected
* Disable all selected
* Delete all selected

which should behave like the counterpart commands for single breakpoints just applied to multiple ones.

Without the change, you should see an message 429 Rate Limit Exceeded after 10 tool calls.
With the change, the conversation will stall for some time in the middle, but eventually continue after 60s of waiting.

Follow-ups

Breaking changes

  • This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Review checklist

Reminder for reviewers

@xpomul
Implement a retry mechanism for Google GenAI calls
70f68d0 
In case of errors (either due to rate limit exceed or other unexpected errors) the Google LanguageModel now resends the last request after a configurable delay.

Signed-off-by: Stefan Winkler <stefan@winklerweb.net>
@github-project-automation github-project-automation bot moved this to Waiting on reviewers in PR Backlog Jun 6, 2025
eneufeld
eneufeld reviewed Jun 10, 2025
View reviewed changes
Copy link
Contributor
@eneufeld eneufeld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good overall.
I suggest to move the retrysettings to the frontend and pass them down from there as we do with other similar settings. (see anthropic package or openai package)

@xpomul
Changed retry settings to preferences
e07f3f9 
Signed-off-by: Stefan Winkler <stefan@winklerweb.net>
@xpomul xpomul requested a review from eneufeld June 10, 2025 19:24
@eneufeld
Copy link
Contributor

@sdirix any opinion on this? I think the change is fine to be merged.

@sdirix
Copy link
Member
sdirix commented Jun 16, 2025

Conceptually this looks good to me, however I did not test it or review it in detail

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eneufeld eneufeld Awaiting requested review from eneufeld

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
PR Backlog
Status: Waiting on reviewers
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xpomul @eneufeld @sdirix
0