A neuro-symbolic complex-valued transformer model pushing the boundaries of AI architecture
SillyAI is a complex-valued neuro-symbolic transformer model that combines systems design principles with modern deep learning techniques and reasoning engines. Its unique architecture sets it apart from other models, making it particularly powerful for fields such as reasoning, physics simulations, and quantum system analysis.
Real-time visualization of wavefunction evolution during training
Resource usage while training
- Infinite context window through structured matrix compression
- Dynamic key-value pair streaming and compression
- Support for various matrix structures (Toeplitz, block Toeplitz, Kronecker, circulant, etc.)
- Eliminates traditional context limitations
- Energy-weighted concept relationships
- Dynamic LFU-based updates and pruning
- Concept tagging and aliasing
- Entangled concept relationships
- Field-specific subgraphs
- Bytecode generation from graph traversal
- LZ4 compression for efficient storage
- Lock-free async operations
- Batched I/O operations
- SVD decomposition for dimensionality reduction
- Dynamic ternary to FP16 quantization
- Geometric algebra optimizations
- JIT compilation of critical paths
- Complex-valued parametric ReLU with phase preservation
- Multi-token async prediction
- Response synthesis through candidate scoring
- Task complexity-based routing
- Self-training capabilities
- Dynamic plugin system
- VLIW/EPIC bytecode VM for reasoning
# Clone the repository
git clone https://github.com/bumbelbee777/sillyai.git
cd sillyai
# Create a virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txtfrom sillyai import SillyAI, ModelConfig
from sillyai.ops import MultivectorOps
# Initialize configuration
config = ModelConfig(
d_model=64,
n_heads=4,
n_layers=2,
max_seq_len=64
)
# Create model
model = SillyAI(config, ops=MultivectorOps().compile())
# Train the model
model.train()- Quantum system simulation
- Wavefunction prediction
- Physics-based learning
- Signal processing
- Complex system modeling
- Equation solving
- NLP
We welcome all contributions! Please see our Contributing Guide for details.
This project is licensed under the MIT License - see the LICENSE file for details.
- PyTorch team for the amazing deep learning framework
- Google's InfiniContext paper for the attention layer inspiration
- AlphaZero for self-learning