A Streamlit-based application that uses local LLMs and LangGraph orchestration to verify claims by scraping evidence from the web, simulating a multi-round debate, and producing a final structured judgment.
- Structured AI debate with Verifier, Counter-Explainer, and Judge agents
- Web scraping with error handling and content extraction
- Final JSON-based verdict with confidence and evidence analysis
- LM Studio for local, private LLM inference
- Fully state-driven flow using LangGraph
- Frontend & Application: Streamlit – For interactive UI
- Language Models: LM Studio – Runs local models via OpenAI-compatible API
- Debate & Graph Framework: LangGraph – For stateful, multi-agent orchestration
- Web Scraping:
requests,BeautifulSoup– For extracting content from URLs - Message Handling:
langchain_core,openai,dataclasses,pydantic– Structured message and state management
-
User Input
- Enter a claim and a list of URLs.
- Choose how many rounds the AI agents will debate.
-
Web Scraping
- Each URL is scraped using
requestsand parsed withBeautifulSoup. - Content and metadata (title, content, full text) are extracted.
- Failed or invalid sources are logged with retry support.
- Each URL is scraped using
-
LangGraph Workflow Begins
-
The system constructs a LangGraph with these nodes:
scrape_evidenceverifier_turncounter_explainer_turncheck_roundsjudge_decision
-
-
Verifier Turn
- LM Studio generates an argument supporting the claim using the Qwen model.
- It references extracted evidence.
-
Counter-Explainer Turn
- Another response is generated (same or different model) to present counterpoints.
-
Rounds Loop
- The debate continues for the defined number of rounds.
-
Judge Decision
- Judge agent (using a reasoning-focused model like Phi-4-Mini) evaluates the full debate.
- Produces a 5-part analysis and structured JSON verdict.
-
Scoring & Display
-
The judge analysis is transformed into JSON verdict.
-
Output includes:
- Final verdict (TRUE / FALSE / INSUFFICIENT_EVIDENCE)
- Confidence score
- Evidence quality
- Winning side
- Key reasoning
-
{
"verdict": "TRUE",
"confidence": 0.85,
"evidence_quality": "STRONG",
"winning_side": "verifier",
"reasoning": "The verifier presented stronger evidence and rebutted counterpoints."
}-
Python 3.9+
-
LM Studio installed and running
-
Models downloaded:
Qwen/Qwen1.5-1.7B-Chatmicrosoft/phi-2orphi-4-mini
git clone https://github.com/yourusername/claim-verifier.git
cd claim-verifierpython -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activatepip install -r requirements.txtOr manually:
pip install streamlit langgraph langchain-core openai beautifulsoup4 requests pandas- Launch LM Studio from https://lmstudio.ai
- Download and start the models listed above
- Confirm it’s listening at:
http://localhost:1234/v1
streamlit run main.pyAccess the app at http://localhost:8501
-
Enter your claim (e.g. "Video games improve cognitive function")
-
Add relevant URLs (studies, articles, etc.)
-
Set number of debate rounds
-
Click "Start Verification"
-
View:
- Arguments from each side
- Final structured verdict
- Judge analysis and scoring
-
LM Studio not running:
- Ensure it's open and models are loaded
- Check port
1234is active
-
Scraping issues:
- Some sites block bots — try using more accessible URLs
-
Model issues:
- Restart LM Studio and re-load the models