Web Recon Tool (WRT)

Version: 0.5.0 - Modular Pattern Configuration

Web Recon Tool (WRT) is a powerful Python command-line script designed for ethical security reconnaissance and debugging. It provides comprehensive web application analysis by scanning URLs, detecting API endpoints, and identifying potential secrets using configurable pattern files.

🚀 Key Features

Enhanced Secret Detection

85+ Comprehensive Patterns: Covers major cloud services, payment APIs, communication platforms, databases, and more
Confidence-Based Filtering: High/Medium/Low confidence levels with entropy analysis
False Positive Reduction: Advanced pattern matching with context analysis
Modular Pattern Files: Easy-to-edit JSON configuration files

Advanced API Discovery

Categorized Endpoint Detection: Authentication, Admin, Monitoring, Webhooks, Documentation
JavaScript Framework Support: Fetch, Axios, jQuery, Angular, React patterns
Intelligent URL Validation: Enhanced filtering and deduplication

Flexible Configuration

Separate Pattern Files: secret_patterns.json, api_patterns.json, js_api_patterns.json
Hot Reload: Reload patterns without restarting
Pattern Validation: Built-in validation and error checking
Community Friendly: Easy JSON editing for contributions

Professional Output

Rich Terminal Interface: Colorful tables, progress bars, syntax highlighting
Categorized Results: Organized by endpoint type and confidence level
Context Display: Show surrounding code for better analysis
Export Options: Save detailed reports to files

📁 Project Structure

webrecon/
├── webrecon.py              # Main reconnaissance tool
├── patterns/
│   ├── secret_patterns.json     # 85+ secret detection patterns
│   ├── api_patterns.json        # API endpoint patterns by category
│   ├── js_api_patterns.json     # JavaScript API call patterns
│   ├── custom_secrets.json      # Optional custom patterns
│   └── custom_apis.json         # Optional custom patterns
├── LICENSE
└── README.md

🛠 Installation

Prerequisites

Python 3.7+
pip package manager

Quick Setup

Clone or download the tool:

git clone <repository-url>
cd web-recon-tool

Create virtual environment (recommended):

python3 -m venv wrtenv
source wrtenv/bin/activate  # On Windows: wrtenv\Scripts\activate

Install dependencies:

pip install typer rich requests beautifulsoup4 jsbeautifier

Initialize pattern files (first run):
```
python webrecon.py init-patterns
```

🎯 Usage

Basic Commands

Scan a website:

python webrecon.py scan https://example.com

Advanced scanning:

python webrecon.py scan https://example.com \
  --depth 2 \
  --min-confidence high \
  --show-context \
  --output report.txt

Pattern Management

View loaded patterns:

python webrecon.py patterns --list           # Secret patterns
python webrecon.py patterns --list-api       # API patterns
python webrecon.py patterns --list-js        # JavaScript patterns
python webrecon.py patterns --stats          # Statistics

Validate pattern files:

python webrecon.py patterns --validate

Initialize default patterns:

python webrecon.py init-patterns --overwrite

Command Reference

Scan Command

python webrecon.py scan [OPTIONS] TARGET_URL

Options:

--depth, -d INTEGER: Crawling depth (0 = target only, default: 1)
--scan-js / --no-scan-js: Include JavaScript analysis (default: enabled)
--scan-html / --no-scan-html: Include HTML analysis (default: enabled)
--min-confidence [low|medium|high]: Filter secrets by confidence (default: low)
--show-context / --no-context: Show code context for secrets (default: disabled)
--categorize-apis / --no-categorize-apis: Group API endpoints (default: enabled)
--reload-patterns: Reload pattern files before scanning
--output, -o FILE: Save results to file

Pattern Command

python webrecon.py patterns [OPTIONS]

Options:

--list, -l: List secret patterns
--list-api: List API endpoint patterns
--list-js: List JavaScript patterns
--stats, -s: Show pattern statistics
--validate, -v: Validate all pattern files
--category, -c CATEGORY: Filter by category

📊 Sample Output

🔧 Configuration

Secret Patterns (`secret_patterns.json`)

Add new secret detection patterns:

{
  "Custom API Key": {
    "pattern": "(customapi_[a-zA-Z0-9]{32})",
    "value_group": 1,
    "desc": "Custom service API key",
    "confidence": "high",
    "category": "custom_service"
  }
}

API Patterns (`api_patterns.json`)

Define new endpoint categories:

{
  "payment_endpoints": {
    "description": "Payment processing endpoints",
    "patterns": [
      "/payment(?:[/\\?#]|$)",
      "/billing(?:[/\\?#]|$)",
      "/checkout(?:[/\\?#]|$)"
    ]
  }
}

JavaScript Patterns (`js_api_patterns.json`)

Add framework-specific API call patterns:

{
  "vue_patterns": {
    "description": "Vue.js HTTP patterns",
    "patterns": [
      "this\\.$http\\.\\w+\\s*\\(\\s*['\"]([^'\"]+)['\"]"
    ]
  }
}

🔍 Detected Patterns

Secret Categories

Cloud Services: AWS, Google Cloud, Azure (12 patterns)
Version Control: GitHub, GitLab, Bitbucket (6 patterns)
Communication: Slack, Discord, Teams, Telegram (8 patterns)
Payment: Stripe, PayPal, Square (6 patterns)
Databases: MongoDB, Redis, PostgreSQL, MySQL (8 patterns)
Authentication: JWT, SSH keys, OAuth tokens (7 patterns)
Development Tools: Docker, NPM, PyPI (6 patterns)
Social Media: Facebook, Twitter, YouTube (8 patterns)
Productivity: Linear, Notion, Jira, Asana (8 patterns)
Infrastructure: Twilio, SendGrid, Cloudflare (12 patterns)
Generic: API keys, passwords, hex keys (4 patterns)

API Endpoint Categories

Standard API Paths: /api, /v1, /rest, /graphql
Admin Interfaces: /admin, /dashboard, /panel
Authentication: /auth, /oauth, /login, /token
File Operations: /upload, /download, /export
Monitoring: /health, /status, /metrics
Documentation: /docs, /swagger, /openapi
Webhooks: /webhook, /callback, /notify

⚠️ Ethical Use and Disclaimer

🔴 FOR AUTHORIZED SECURITY TESTING ONLY 🔴

This tool is designed for legitimate security research, penetration testing, and debugging purposes. You must have explicit written permission from system owners before scanning.

⚠️ Important Warnings:

Unauthorized scanning is illegal and unethical
Pattern-based detection can produce false positives
Always manually verify potential secrets
Rate limiting may occur with aggressive scanning
Developers are not responsible for misuse

🤝 Contributing

We welcome contributions! The modular design makes it easy to add new patterns:

Adding Secret Patterns

Edit secret_patterns.json
Add your pattern with proper metadata
Test with python webrecon.py patterns --validate
Submit a pull request

Adding API Patterns

Edit api_patterns.json or js_api_patterns.json
Follow the existing structure
Validate your patterns
Submit with test cases

Contribution Guidelines

Test thoroughly to minimize false positives
Document patterns with clear descriptions
Include metadata (confidence, category, description)
Follow JSON formatting standards

🐛 Known Limitations

Pattern-Based Detection: May produce false positives/negatives
Static Analysis: Limited to content available via HTTP requests
Dynamic Content: JavaScript-rendered content may be missed
Rate Limiting: Aggressive scanning may trigger protections
Encoding: Assumes UTF-8 content encoding

📈 Recent Updates (v0.5.0)

✅ Modular Pattern Files: Separate JSON files for easy editing
✅ 85+ Secret Patterns: Comprehensive coverage of modern services
✅ Confidence Scoring: High/Medium/Low classification
✅ Enhanced Validation: Built-in pattern validation
✅ Hot Reload: Reload patterns without restarting
✅ Better Categorization: Organized by service type
✅ Improved CLI: Enhanced commands and options

📄 57BF License

This project is licensed under the MIT License - see the LICENSE file for details.

Happy ethical hacking! 🔐✨

Name		Name	Last commit message	Last commit date
Latest commit History 11 Commits
patterns		patterns
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
recon-demo.png		recon-demo.png
requirements.txt		requirements.txt
webrecon.py		webrecon.py

License

adityasanehi/web-recon-cli

Folders and files

Latest commit

History

Repository files navigation