Open
Description
The feature: Agent env integration
Building on top of torchrl.envs.llm.ChatEnv, we should expand the environment integrations to support a wider range of agent-tool interactions. This would enable training models that can interact with various tools (browsers, shells, games) through LLMs.
## Proposed envs
Here are the key environments and frameworks to consider integrating:
- MLGym (facebookresearch/mlgym) for structured ML experimentation
- BrowserGym for web interaction environments
- MineDojo for rich Minecraft-based tasks with natural language specifications
- AgentVerse for multi-agent interactions and tool usage
- WebArena\
- SWE-bench
The implementation would extend ChatEnv with a standardized interface for tool integration, consistent reward transformations, and unified observation spaces. This would greatly benefit research in tool-using LLM agents and provide standardized benchmarking across different environments.
Metadata
Metadata
Assignees
Labels
No labels