KAQG is an open-source educational AI framework that integrates Knowledge Graphs (KG), Retrieval-Augmented Generation (RAG), and Assessment Theory to generate high-quality, difficulty-calibrated exam questions. It supports explainable multi-step reasoning and aligns generated items with Bloom’s Taxonomy and Item Response Theory (IRT).
- Knowledge Graph Construction: Extracts SPO triples and concept hierarchies from multimodal educational materials.
- Semantic Retrieval: Uses KG-aware fact chaining for more coherent context selection.
- Question Generation: Employs LLMs and PageRank for ranking concepts, aligned with cognitive difficulty levels.
- Assessment Integration: Quantifies difficulty using Bloom’s levels and validates questions through IRT.
- Multi-Agent System (MAS): Distributed agents coordinate retrieval, generation, and evaluation via DDS-based pub-sub.
The system comprises:
KAQG-Retriever: Builds domain-isolated KGs from documents (PDF, video, web).KAQG-Generator: Creates and validates exam questions with psychometric calibration.AI Agents Framework: Manages autonomous tasks using DDS communication.
kaqg/
├── apps/ # Utility tools
├── doc/ # Documents
├── know-edit/ # KG editor
├── src/ # Source codes
├── evluation/ # Question assessment
├── generation/ # Question generation
├── knowsys/ # KG library
├── retrieval/ # Material retrieval
└── services/ # Services agents
├── unit_test/ # Unit tests
└── README.mdgit clone https://github.com/mfshiu/kaqg.git
cd kaqg
pip install -r requirements.txtYou’ll also need:
- Neo4j 5.x
- OpenAI-compatible LLMs (e.g., GPT-4 or LLaMA2)
- DDS middleware (e.g., Eclipse Cyclone DDS)
-
Build Knowledge Graphs
python retriever/build_graph.py --input data/textbooks/
-
Generate Questions
python generator/generate_questions.py --subject "Environmental Science" -
Evaluate with IRT
python generator/evaluate_questions.py --model irt
Validated against ACT reading passages with over 90 system-generated questions at three difficulty levels. Questions were evaluated by domain experts and showed high alignment with psychometric expectations.
- 🤖 AgentFlow: Multi-agent architecture for distributed coordination and task delegation in KAQG.
- Ching Han Chen – Professor, National Central University
- Ming Fang Shiu – Ph.D. Candidate, NCU
For questions, contact the maintainer: 108582003@cc.ncu.edu.tw
Would you like a version of this saved as README.md for direct upload?