AgentLens is an open-source debugging toolkit for AI agents that enables offline recording, replay, failure analysis, and cost tracking. Built for developers who need efficient, local-first debugging without consuming API credits.
When developing AI agents, debugging is costly and time-consuming. Each iteration requires API calls, resources, and careful tracking of what happened. AgentLens provides:
- Offline debugging - Record and replay agent runs without making additional API calls
- Failure analysis - Identify common failure patterns automatically
- Cost tracking - Monitor token usage and estimated API costs
- Local-first workflow - Everything runs on your machine with no external dependencies
pip install auriel-agentlensfrom agentlens import AgentLens
# Initialize AgentLens
lens = AgentLens()
# 1. Record agent runs with a decorator
@lens.record
def my_agent_function(query):
# Your agent implementation
return result
# Use your agent normally - AgentLens records in the background
response = my_agent_function("Process this data")
# 2. Or use the context manager for more control
with lens.context_record(model="gpt-4") as recording:
# Your agent code here
result = some_function()
recording.log_run(
input_data=query,
output_data=result,
token_usage={"prompt_tokens": 10, "completion_tokens": 20}
)
# 3. Replay the last recorded run
lens.replay()
# 4. Analyze failures in the last run
lens.analyze()
# 5. Track estimated costs
lens.costs(all_runs=True)from agentlens import AgentLens
from agentlens.integrations.langchain import LangChainLens
# Initialize AgentLens and LangChainLens
lens = AgentLens()
lc_lens = LangChainLens(lens=lens)
# Wrap a LangChain LLM, Chain or Agent
llm = OpenAI(temperature=0.7)
wrapped_llm = lc_lens.wrap_llm(llm)
# Use wrapped components as normal
response = wrapped_llm("What are three best practices for writing clean code?")
# Wrap a LangChain Chain
chain = LLMChain(llm=llm, prompt=prompt)
wrapped_chain = lc_lens.wrap_chain(chain)
# Wrap a LangChain Agent
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)
wrapped_agent = lc_lens.wrap_agent(agent)from agentlens import AgentLens
from agentlens.integrations.openai import OpenAILens
import openai
# Initialize AgentLens and OpenAILens
lens = AgentLens()
openai_lens = OpenAILens(lens=lens)
# Wrap the OpenAI client
client = openai.OpenAI()
wrapped_client = openai_lens.wrap_client(client)
# Use the wrapped client as normal
response = wrapped_client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain debugging in one paragraph."}
]
)Records agent inputs, outputs, tool calls, and token usage to local storage without requiring additional API calls.
@lens.record
def agent_function(input_query):
# Agent implementation
return resultOr using the context manager for more control:
with lens.context_record(model="gpt-4") as recording:
# Custom agent code
result = process_query(query)
recording.log_run(
input_data=query,
output_data=result,
token_usage=token_count,
tool_calls=tools_used
)Reproduces agent runs locally, allowing you to inspect each step without consuming API credits.
# Replay the most recent run
lens.replay()
# Replay a specific run by ID
lens.replay(run_id=3)Automatically identifies common issues like timeouts, empty outputs, and potential hallucinations based on output patterns.
# Analyze the most recent run
lens.analyze()
# Analyze a specific run
lens.analyze(run_id=3)Estimates API usage costs based on token counts and configurable pricing models.
# Track costs for the most recent run
lens.costs()
# Track costs for all runs
lens.costs(all_runs=True)
# Track costs for a specific run
lens.costs(run_id=3)AgentLens includes a CLI for working with recorded runs:
# Replay a run
agentlens replay --file runs.jsonl
# Analyze a run
agentlens analyze --id 3 --file runs.jsonl
# Calculate costs
agentlens costs --all --file runs.jsonl- Development iteration - Debug agents locally without repeatedly hitting APIs
- Cost optimization - Identify expensive or inefficient patterns in agent behavior
- Regression testing - Verify agent behavior after code changes
- Educational purposes - Study and analyze agent decision patterns
Currently supporting:
- Direct Python function decorators and context managers
- LangChain (LLMs, Chains, and Agents)
- OpenAI Python client (both modern and legacy versions)
- Planning future support for CrewAI and other frameworks
AgentLens is focused specifically on offline debugging and development iteration, complementing production monitoring tools like LangSmith or Helicone.
| Feature | AgentLens | Production Monitors |
|---|---|---|
| Deployment | Local-only | Cloud/hosted |
| Focus | Development/debugging | Production monitoring |
| Cost | Free, open-source | Freemium/paid tiers |
| Integration | Single decorator | Platform-specific |
| Analysis | Offline-first | Real-time analytics |
To set up AgentLens for development:
# Clone the repository
git clone https://github.com/auriel-ai/agentlens.git
cd agentlens
# Run the setup script
./setup_and_run.shCheck out the examples/ directory for full examples including:
- Basic usage with the decorator and context manager
- LangChain integration examples
- OpenAI client integration
- Command line interface examples
Contributions are welcome! See CONTRIBUTING.md for details.