diff --git a/.cursor/rules/documentation-sync.mdc b/.cursor/rules/documentation-sync.mdc new file mode 100644 index 000000000..9817125e3 --- /dev/null +++ b/.cursor/rules/documentation-sync.mdc @@ -0,0 +1,36 @@ +--- +description: when making code changes or adding documentation +globs: ["*.py", "*.md"] +alwaysApply: true +--- + +- When making code changes: + - Update related documentation files to reflect the changes + - Check docstrings and type hints are up to date + - Update any example code in markdown files + - Review README.md if the changes affect installation or usage + +- When creating new markdown files: + - Add the file to mkdocs.yml under the appropriate section + - Follow the existing hierarchy and indentation + - Use descriptive nav titles + - Example: + ```yaml + nav: + - Home: index.md + - Guides: + - Getting Started: guides/getting-started.md + - Your New File: guides/your-new-file.md + ``` + +- For API documentation: + - Ensure new functions/classes are documented + - Include type hints and docstrings + - Add usage examples + - Update API reference docs if auto-generated + +- Documentation Quality: + - Write at grade 10 reading level (see simple-language.mdc) + - Include working code examples + - Add links to related documentation + - Use consistent formatting and style \ No newline at end of file diff --git a/.cursor/rules/followups.mdc b/.cursor/rules/followups.mdc new file mode 100644 index 000000000..29b9a9f0e --- /dev/null +++ b/.cursor/rules/followups.mdc @@ -0,0 +1,8 @@ +--- +description: when AI agents are collaborating on code +globs: "*" +alwaysApply: true +--- +Make sure to come up with follow-up hot keys. They should be thoughtful and actionable and result in small additional code changes based on the context that you have available. + +using [J], [K], [L] diff --git a/.cursor/rules/new-features-planning.mdc b/.cursor/rules/new-features-planning.mdc new file mode 100644 index 000000000..3ab5e7501 --- /dev/null +++ b/.cursor/rules/new-features-planning.mdc @@ -0,0 +1,45 @@ +--- +description: when asked to implement new features or clients +globs: *.py +alwaysApply: true +--- + +- When being asked to make new features, make sure that you check out from main a new branch and make incremental commits + - Use conventional commit format: `(): ` + - Types: feat, fix, docs, style, refactor, perf, test, chore + - Example: `feat(validation): add email validation function` + - Keep commits focused on a single change + - Write descriptive commit messages in imperative mood + - Use `git commit -m "type(scope): subject" -m "body" -m "footer"` for multiline commits +- If the feature is very large, create a temporary `todo.md` +- And start a pull request using `gh` + - Create PRs with multiline bodies using: + ```bash + gh pr create --title "feat(component): add new feature" --body "$(cat < --add-reviewer jxnl,ivanleomk` + - Or include `-r jxnl,ivanleomk` when creating the PR +- use `gh pr view --comments | cat` to view all the comments +- For PR updates: + - Do not directly commit to an existing PR branch + - Instead, create a new PR that builds on top of the original PR's branch + - This creates a "stacked PR" pattern where: + 1. The original PR (base) contains the initial changes + 2. The new PR (stack) contains only the review-related updates + 3. Once the base PR is merged, the stack can be rebased onto main diff --git a/.cursor/rules/readme.md b/.cursor/rules/readme.md new file mode 100644 index 000000000..8ade85640 --- /dev/null +++ b/.cursor/rules/readme.md @@ -0,0 +1,100 @@ +# Cursor Rules + +Cursor rules are configuration files that help guide AI-assisted development in the Cursor IDE. They provide structured instructions for how the AI should behave in specific contexts or when working with certain types of files. + +## What is Cursor? + +[Cursor](https://cursor.sh) is an AI-powered IDE that helps developers write, understand, and maintain code more efficiently. It integrates AI capabilities directly into the development workflow, providing features like: + +- AI-assisted code completion +- Natural language code generation +- Intelligent code explanations +- Automated refactoring suggestions + +## Understanding Cursor Rules + +Cursor rules are defined in `.mdc` files within the `.cursor/rules` directory. Each rule file follows a specific naming convention: lowercase names with the `.mdc` extension (e.g., `simple-language.mdc`). + +Each rule file contains: + +1. **Metadata Header**: YAML frontmatter that defines: + ```yaml + --- + description: when to apply this rule + globs: file patterns to match (e.g., "*.py", "*.md", or "*" for all files) + alwaysApply: true/false # whether to apply automatically + --- + ``` + +2. **Rule Content**: Markdown-formatted instructions that guide the AI's behavior + +## Available Rules + +Currently, the following rules are defined: + +### `simple-language.mdc` +- **Purpose**: Ensures documentation is written at a grade 10 reading level +- **Applies to**: Markdown files (*.md) +- **Auto Apply**: No +- **Key Requirements**: + - Write at grade 10 reading level + - Ensure code blocks are self-contained with complete imports + +### `new-features-planning.mdc` +- **Purpose**: Guides feature implementation workflow +- **Applies to**: Python files (*.py) +- **Auto Apply**: Yes +- **Key Requirements**: + - Create new branch from main + - Make incremental commits + - Create todo.md for large features + - Start pull requests using GitHub CLI (`gh`) + - Include "This PR was written by [Cursor](https://cursor.sh)" in PRs + +### `followups.mdc` +- **Purpose**: Ensures thoughtful follow-up suggestions +- **Applies to**: All files +- **Auto Apply**: Yes +- **Key Requirements**: + - Generate actionable hotkey suggestions using: + - [J]: First follow-up action + - [K]: Second follow-up action + - [L]: Third follow-up action + - Focus on small, contextual code changes + - Suggestions should be thoughtful and actionable + +### `documentation-sync.mdc` +- **Purpose**: Maintains documentation consistency with code changes +- **Applies to**: Python and Markdown files (*.py, *.md) +- **Auto Apply**: Yes +- **Key Requirements**: + - Update docs when code changes + - Add new markdown files to mkdocs.yml + - Keep API documentation current + - Maintain documentation quality standards + +## Creating New Rules + +To create a new rule: + +1. Create a `.mdc` file in `.cursor/rules/` using lowercase naming +2. Add YAML frontmatter with required metadata: + ```yaml + --- + description: when to apply this rule + globs: file patterns to match + alwaysApply: true/false + --- + ``` +3. Write clear, specific instructions in Markdown +4. Test the rule with relevant file types + +## Best Practices + +- Keep rules focused and specific +- Use clear, actionable language +- Test rules thoroughly before committing +- Document any special requirements or dependencies +- Update rules as project needs evolve +- Use consistent file naming (lowercase with .mdc extension) +- Ensure globs patterns are explicit and documented diff --git a/.cursor/rules/simple-language.mdc b/.cursor/rules/simple-language.mdc new file mode 100644 index 000000000..48a9b191f --- /dev/null +++ b/.cursor/rules/simple-language.mdc @@ -0,0 +1,8 @@ +--- +description: when writing documentation +globs: *.md +alwaysApply: false +--- + +- When writing documents and concepts make sure that you write at a grade 10 reading level +- make sure every code block has complete imports and makes no references to previous code blocks, each one needs to be self contained diff --git a/.github/workflows/claude-code-review.yml b/.github/workflows/claude-code-review.yml new file mode 100644 index 000000000..5bf8ce595 --- /dev/null +++ b/.github/workflows/claude-code-review.yml @@ -0,0 +1,78 @@ +name: Claude Code Review + +on: + pull_request: + types: [opened, synchronize] + # Optional: Only run on specific file changes + # paths: + # - "src/**/*.ts" + # - "src/**/*.tsx" + # - "src/**/*.js" + # - "src/**/*.jsx" + +jobs: + claude-review: + # Optional: Filter by PR author + # if: | + # github.event.pull_request.user.login == 'external-contributor' || + # github.event.pull_request.user.login == 'new-developer' || + # github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR' + + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + issues: read + id-token: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 1 + + - name: Run Claude Code Review + id: claude-review + uses: anthropics/claude-code-action@beta + with: + claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} + + # Optional: Specify model (defaults to Claude Sonnet 4, uncomment for Claude Opus 4) + # model: "claude-opus-4-20250514" + + # Direct prompt for automated review (no @claude mention needed) + direct_prompt: | + Please review this pull request and provide feedback on: + - Code quality and best practices + - Potential bugs or issues + - Performance considerations + - Security concerns + - Test coverage + + Be constructive and helpful in your feedback. + + # Optional: Use sticky comments to make Claude reuse the same comment on subsequent pushes to the same PR + # use_sticky_comment: true + + # Optional: Customize review based on file types + # direct_prompt: | + # Review this PR focusing on: + # - For TypeScript files: Type safety and proper interface usage + # - For API endpoints: Security, input validation, and error handling + # - For React components: Performance, accessibility, and best practices + # - For tests: Coverage, edge cases, and test quality + + # Optional: Different prompts for different authors + # direct_prompt: | + # ${{ github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR' && + # 'Welcome! Please review this PR from a first-time contributor. Be encouraging and provide detailed explanations for any suggestions.' || + # 'Please provide a thorough code review focusing on our coding standards and best practices.' }} + + # Optional: Add specific tools for running tests or linting + # allowed_tools: "Bash(npm run test),Bash(npm run lint),Bash(npm run typecheck)" + + # Optional: Skip review for certain conditions + # if: | + # !contains(github.event.pull_request.title, '[skip-review]') && + # !contains(github.event.pull_request.title, '[WIP]') + diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml new file mode 100644 index 000000000..3fcc56f51 --- /dev/null +++ b/.github/workflows/claude.yml @@ -0,0 +1,64 @@ +name: Claude Code + +on: + issue_comment: + types: [created] + pull_request_review_comment: + types: [created] + issues: + types: [opened, assigned] + pull_request_review: + types: [submitted] + +jobs: + claude: + if: | + (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) || + (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) || + (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) || + (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + issues: read + id-token: write + actions: read # Required for Claude to read CI results on PRs + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 1 + + - name: Run Claude Code + id: claude + uses: anthropics/claude-code-action@beta + with: + claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} + + # This is an optional setting that allows Claude to read CI results on PRs + additional_permissions: | + actions: read + + # Optional: Specify model (defaults to Claude Sonnet 4, uncomment for Claude Opus 4) + # model: "claude-opus-4-20250514" + + # Optional: Customize the trigger phrase (default: @claude) + # trigger_phrase: "/claude" + + # Optional: Trigger when specific user is assigned to an issue + # assignee_trigger: "claude-bot" + + # Optional: Allow Claude to run specific commands + # allowed_tools: "Bash(npm install),Bash(npm run build),Bash(npm run test:*),Bash(npm run lint:*)" + + # Optional: Add custom instructions for Claude to customize its behavior for your project + # custom_instructions: | + # Follow our coding standards + # Ensure all new code has tests + # Use TypeScript for new files + + # Optional: Custom environment variables for Claude + # claude_env: | + # NODE_ENV: test + allowed_tools: "Bash(git commit), Bash(uv sync), Bash(uv run)" diff --git a/.github/workflows/pyright.yml b/.github/workflows/pyright.yml index 229201019..f2c0a5421 100644 --- a/.github/workflows/pyright.yml +++ b/.github/workflows/pyright.yml @@ -12,12 +12,7 @@ env: jobs: Pyright: - runs-on: ${{ matrix.os }} - strategy: - matrix: - os: [ubuntu-latest, macos-latest] - python-version: ["3.9", "3.10", "3.11"] - + runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3 @@ -26,7 +21,7 @@ jobs: with: enable-cache: true - name: Set up Python - run: uv python install ${{ matrix.python-version }} + run: uv python install 3.11 - name: Install the project run: uv sync --all-extras - name: Run pyright diff --git a/.github/workflows/ruff.yml b/.github/workflows/ruff.yml index 802a13bb3..f93ffb2f2 100644 --- a/.github/workflows/ruff.yml +++ b/.github/workflows/ruff.yml @@ -13,10 +13,7 @@ env: jobs: Ruff: - runs-on: ${{ matrix.os }} - strategy: - matrix: - os: [ubuntu-latest, macos-latest] + runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3 @@ -29,7 +26,7 @@ jobs: - name: Install the project run: uv sync --all-extras - name: Run Continuous Integration Action - uses: astral-sh/ruff-action@v1 + uses: astral-sh/ruff-action@v3 - name: Upload Artifacts uses: actions/upload-artifact@v4 with: diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 3321f0bff..60d9b7e51 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -6,13 +6,11 @@ on: - main jobs: - release: + # Core tests without LLM providers + core-tests: + name: Core Tests runs-on: ubuntu-latest - strategy: - matrix: - python-version: ["3.9", "3.10", "3.11"] - steps: - uses: actions/checkout@v2 - name: Install uv @@ -20,29 +18,82 @@ jobs: with: enable-cache: true - name: Set up Python - run: uv python install ${{ matrix.python-version }} + run: uv python install 3.11 - name: Install the project run: uv sync --all-extras - - name: Run tests - if: matrix.python-version != '3.11' - run: uv run pytest tests/ -k 'not llm and not openai and not gemini and not anthropic and not cohere and not vertexai' + - name: Run core tests + run: uv run pytest tests/ -k 'not llm and not openai and not gemini and not anthropic and not cohere and not vertexai and not mistral and not xai and not docs' env: + INSTRUCTOR_ENV: CI OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }} + XAI_API_KEY: ${{ secrets.XAI_API_KEY }} + + # Provider tests run in parallel + provider-tests: + name: ${{ matrix.provider.name }} Tests + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + provider: + - name: Openai + env_key: OPENAI_API_KEY + test_path: tests/llm/test_openai + - name: Anthropic + env_key: ANTHROPIC_API_KEY + test_path: tests/llm/test_anthropic + - name: Gemini + env_key: GOOGLE_API_KEY + test_path: tests/llm/test_gemini + - name: Google GenAI + env_key: GOOGLE_API_KEY + test_path: tests/llm/test_genai + - name: Cohere + env_key: COHERE_API_KEY + test_path: tests/llm/test_cohere + - name: XAI + env_key: XAI_API_KEY + test_path: tests/llm/test_xai - - name: Run Gemini Tests - if: matrix.python-version == '3.11' - run: uv run pytest tests/llm/test_gemini + steps: + - uses: actions/checkout@v2 + - name: Install uv + uses: astral-sh/setup-uv@v4 + with: + enable-cache: true + - name: Set up Python + run: uv python install 3.11 + - name: Install the project + run: uv sync --all-extras + - name: Run ${{ matrix.provider.name }} tests + run: uv run pytest ${{ matrix.provider.test_path }} -n auto env: - GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }} + INSTRUCTOR_ENV: CI + ${{ matrix.provider.env_key }}: ${{ secrets[matrix.provider.env_key] }} + + # Auto client needs multiple providers + auto-client-test: + name: Auto Client Tests + runs-on: ubuntu-latest - - name: Generate coverage report - if: matrix.python-version == '3.11' - run: | - uv run coverage run -m pytest tests/ -k "not docs and not anthropic and not gemini and not cohere and not vertexai and not fireworks" - uv run coverage report - uv run coverage html + steps: + - uses: actions/checkout@v2 + - name: Install uv + uses: astral-sh/setup-uv@v4 + with: + enable-cache: true + - name: Set up Python + run: uv python install 3.11 + - name: Install the project + run: uv sync --all-extras + - name: Run Auto Client tests + run: uv run pytest tests/test_auto_client.py env: + INSTRUCTOR_ENV: CI OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} + GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }} + COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }} ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + XAI_API_KEY: ${{ secrets.XAI_API_KEY }} diff --git a/.github/workflows/test_docs.yml b/.github/workflows/test_docs.yml index 212144964..2ef1f094d 100644 --- a/.github/workflows/test_docs.yml +++ b/.github/workflows/test_docs.yml @@ -1,9 +1,7 @@ name: Test Docs on: - pull_request: - push: - branches: - - master + schedule: + - cron: '0 0 1 * *' # Runs at 00:00 on the 1st of every month jobs: release: runs-on: ubuntu-latest @@ -33,6 +31,6 @@ jobs: - name: Install the project run: uv sync --all-extras - name: Run tests - run: poetry run pytest tests/llm/test_openai/docs + run: uv run pytest tests/docs env: OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} diff --git a/.gitignore b/.gitignore index a3e394bf9..6972cc34e 100644 --- a/.gitignore +++ b/.gitignore @@ -173,3 +173,7 @@ tutorials/results.jsonlines tutorials/schema.json wandb/settings math_finetunes.jsonl + +pr_body.md + +check_zero_width_chars.py diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index c4b8eb9cd..210562ac3 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -8,3 +8,21 @@ repos: files: ^(instructor|tests|examples)/ - id: ruff-format # Run the formatter. name: Run Formatter (Ruff) + + - repo: local + hooks: + - id: uv-lock-check + name: Check uv.lock is up-to-date + entry: uv + args: [lock, --check] + language: system + files: ^(pyproject\.toml|uv\.lock)$ + pass_filenames: false + + - id: uv-sync-check + name: Verify dependencies can be installed + entry: uv + args: [sync, --check] + language: system + files: ^(pyproject\.toml|uv\.lock)$ + pass_filenames: false diff --git a/AGENT.md b/AGENT.md new file mode 100644 index 000000000..f84c1b612 --- /dev/null +++ b/AGENT.md @@ -0,0 +1,27 @@ +# AGENT.md + +## Commands +- Install: `uv pip install -e ".[dev]"` or `poetry install --with dev` +- Run tests: `uv run pytest tests/` +- Run single test: `uv run pytest tests/path_to_test.py::test_name` +- Skip LLM tests: `uv run pytest tests/ -k 'not llm and not openai'` +- Type check: `uv run pyright` +- Lint: `uv run ruff check instructor examples tests` +- Format: `uv run ruff format instructor examples tests` +- Build docs: `uv run mkdocs serve` (local) or `./build_mkdocs.sh` (production) + +## Architecture +- **Core**: `instructor/` - Pydantic-based structured outputs for LLMs +- **Base classes**: `Instructor` and `AsyncInstructor` in `client.py` +- **Providers**: Client files (`client_*.py`) for OpenAI, Anthropic, Gemini, Cohere, etc. +- **Factory pattern**: `from_provider()` for automatic provider detection +- **DSL**: `dsl/` directory with Partial, Iterable, Maybe, Citation extensions +- **Key modules**: `patch.py` (patching), `process_response.py` (parsing), `function_calls.py` (schemas) + +## Code Style +- **Typing**: Strict type annotations, use `BaseModel` for structured outputs +- **Imports**: Standard lib → third-party → local +- **Formatting**: Ruff with Black conventions +- **Error handling**: Custom exceptions from `exceptions.py`, Pydantic validation +- **Naming**: `snake_case` functions/variables, `PascalCase` classes +- **No mocking**: Tests use real API calls diff --git a/CLAUDE.md b/CLAUDE.md index 959d3975a..d3182f40d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,32 +1,80 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + # Instructor Development Guide ## Commands -- Install deps: `poetry install --with dev,anthropic` -- Run tests: `poetry run pytest tests/` -- Run specific test: `poetry run pytest tests/path_to_test.py::test_name` -- Skip LLM tests: `poetry run pytest tests/ -k 'not llm and not openai'` -- Type check: `poetry run pyright` -- Lint: `ruff check instructor examples tests` -- Format: `black instructor examples tests` -- Generate coverage: `poetry run coverage run -m pytest tests/ -k "not docs"` then `poetry run coverage report` +- Install deps: `uv pip install -e ".[dev,anthropic]"` or `poetry install --with dev,anthropic` +- Run tests: `uv run pytest tests/` +- Run specific test: `uv run pytest tests/path_to_test.py::test_name` +- Skip LLM tests: `uv run pytest tests/ -k 'not llm and not openai'` +- Type check: `uv run pyright` +- Lint: `uv run ruff check instructor examples tests` +- Format: `uv run ruff format instructor examples tests` +- Generate coverage: `uv run coverage run -m pytest tests/ -k "not docs"` then `uv run coverage report` +- Build documentation: `uv run mkdocs serve` (for local preview) or `./build_mkdocs.sh` (for production) + +## Installation & Setup +- Fork the repository and clone your fork +- Install UV: `pip install uv` +- Create virtual environment: `uv venv` +- Install dependencies: `uv pip install -e ".[dev]"` +- Install pre-commit: `uv run pre-commit install` +- Run tests to verify: `uv run pytest tests/ -k "not openai"` ## Code Style Guidelines - **Typing**: Use strict typing with annotations for all functions and variables - **Imports**: Standard lib → third-party → local imports -- **Formatting**: Follow Black's formatting conventions +- **Formatting**: Follow Black's formatting conventions (enforced by Ruff) - **Models**: Define structured outputs as Pydantic BaseModel subclasses - **Naming**: snake_case for functions/variables, PascalCase for classes - **Error Handling**: Use custom exceptions from exceptions.py, validate with Pydantic - **Comments**: Docstrings for public functions, inline comments for complex logic +## Conventional Commits +- **Format**: `type(scope): description` +- **Types**: feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert +- **Examples**: + - `feat(anthropic): add support for Claude 3.5` + - `fix(openai): correct response parsing for streaming` + - `docs(README): update installation instructions` + - `test(gemini): add validation tests for JSON mode` + +## Core Architecture +- **Base Classes**: `Instructor` and `AsyncInstructor` in client.py are the foundation +- **Factory Pattern**: Provider-specific factory functions (`from_openai`, `from_anthropic`, etc.) +- **Unified Access**: `from_provider()` function in auto_client.py for automatic provider detection +- **Mode System**: `Mode` enum categorizes different provider capabilities (tools vs JSON output) +- **Patching Mechanism**: Uses Python's dynamic nature to patch provider clients for structured outputs +- **Response Processing**: Transforms raw API responses into validated Pydantic models +- **DSL Components**: Special types like Partial, Iterable, Maybe extend the core functionality + ## Provider Architecture -- **Supported Providers**: OpenAI, Anthropic, Gemini, Cohere, Mistral, Groq, VertexAI, Fireworks, Cerebras, Writer, Databricks, Anyscale, Together, LiteLLM +- **Supported Providers**: OpenAI, Anthropic, Gemini, Cohere, Mistral, Groq, VertexAI, Fireworks, Cerebras, Writer, Databricks, Anyscale, Together, LiteLLM, Bedrock, Perplexity - **Provider Implementation**: Each provider has a dedicated client file (e.g., `client_anthropic.py`) with factory functions - **Modes**: Different providers support specific modes (`Mode` enum): `ANTHROPIC_TOOLS`, `GEMINI_JSON`, etc. - **Common Pattern**: Factory functions (e.g., `from_anthropic`) take a native client and return patched `Instructor` instances - **Provider Testing**: Tests in `tests/llm/` directory, define Pydantic models, make API calls, verify structured outputs - **Provider Detection**: `get_provider` function analyzes base URL to detect which provider is being used +## Key Components +- **process_response.py**: Handles parsing and converting LLM outputs to Pydantic models +- **patch.py**: Contains the core patching logic for modifying provider clients +- **function_calls.py**: Handles generating function/tool schemas from Pydantic models +- **hooks.py**: Provides event hooks for intercepting various stages of the LLM request/response cycle +- **dsl/**: Domain-specific language extensions for specialized model types +- **retry.py**: Implements retry logic for handling validation failures +- **validators.py**: Custom validation mechanisms for structured outputs + +## Testing Guidelines +- Tests are organized by provider under `tests/llm/` +- Each provider has its own conftest.py with fixtures +- Standard tests cover: basic extraction, streaming, validation, retries +- Evaluation tests in `tests/llm/test_provider/evals/` assess model capabilities +- Use parametrized tests when testing similar functionality across variants +- **IMPORTANT**: No mocking in tests - tests make real API calls + ## Documentation Guidelines - Every provider needs documentation in `docs/integrations/` following standard format - Provider docs should include: installation, basic example, modes supported, special features @@ -34,7 +82,216 @@ - Example code should include complete imports and environment setup - Tutorials should progress from simple to complex concepts - New features should include conceptual explanation in `docs/concepts/` - +- **Writing Style**: Grade 10 reading level, all examples must be working code + +## Branch and Development Workflow +1. Fork and clone the repository +2. Create feature branch: `git checkout -b feat/your-feature` +3. Make changes and add tests +4. Run tests and linting +5. Commit with conventional commit message +6. Push to your fork and create PR +7. Use stacked PRs for complex features + +## Adding New Providers + +### Step-by-Step Guide +1. **Update Provider Enum** in `instructor/utils.py`: + ```python + class Provider(Enum): + YOUR_PROVIDER = "your_provider" + ``` + +2. **Add Provider Modes** in `instructor/mode.py`: + ```python + class Mode(enum.Enum): + YOUR_PROVIDER_TOOLS = "your_provider_tools" + YOUR_PROVIDER_JSON = "your_provider_json" + ``` + +3. **Create Client Implementation** `instructor/client_your_provider.py`: + - Use overloads for sync/async variants + - Validate mode compatibility + - Return appropriate Instructor/AsyncInstructor instance + - Handle provider-specific edge cases + +4. **Add Conditional Import** in `instructor/__init__.py`: + ```python + if importlib.util.find_spec("your_provider_sdk") is not None: + from .client_your_provider import from_your_provider + __all__ += ["from_your_provider"] + ``` + +5. **Update Auto Client** in `instructor/auto_client.py`: + - Add to `supported_providers` list + - Implement provider handling in `from_provider()` + - Update `get_provider()` function if URL-detectable + +6. **Create Tests** in `tests/llm/test_your_provider/`: + - `conftest.py` with client fixtures + - Basic extraction tests + - Streaming tests + - Validation/retry tests + - No mocking - use real API calls + +7. **Add Documentation** in `docs/integrations/your_provider.md`: + - Installation instructions + - Basic usage examples + - Supported modes + - Provider-specific features + +8. **Update Navigation** in `mkdocs.yml`: + - Add to integrations section + - Include redirects if needed + +## Contributing to Evals +- Standard evals for each provider test model capabilities +- Create new evals following existing patterns +- Run evals as part of integration test suite +- Performance tracking and comparison + +## Pull Request Guidelines +- Keep PRs small and focused +- Include tests for all changes +- Update documentation as needed +- Follow PR template +- Link to relevant issues + +## Type System and Best Practices + +### PyRight Configuration +- **Type Checking Mode**: Basic (not strict) for gradual typing adoption +- **Python Version**: 3.9 for compatibility +- **Settings in pyrightconfig.json**: + - `reportMissingImports = "warning"` - Missing imports are warnings + - `reportMissingTypeStubs = false` - Type stubs optional + - Exclusions for certain client files (bedrock, cerebras) +- Run `uv run pyright` before committing - zero errors required + +### Type Patterns +- **Bounded TypeVars**: Use `T = TypeVar("T", bound=Union[BaseModel, ...])` for constraints +- **Version Compatibility**: Handle Python 3.9 vs 3.10+ typing differences explicitly +- **Union Type Syntax**: Use `from __future__ import annotations` to enable Python 3.10+ union syntax (`|`) in Python 3.9 +- **Simple Type Detection**: Special handling for `list[Union[int, str]]` patterns +- **Runtime Type Handling**: Graceful fallbacks for compatibility + +### Pydantic Integration +- Heavy use of `BaseModel` for structured outputs +- `TypeAdapter` used internally for JSON schema generation +- Field validators and custom types +- Models serve dual purpose: validation and documentation + +## Building Documentation + +### Setup +```bash +# Install documentation dependencies +pip install -r requirements-doc.txt +``` + +### Local Development +```bash +# Serve documentation locally with hot reload +uv run mkdocs serve + +# Build documentation for production +./build_mkdocs.sh +``` + +### Documentation Features +- **Material Theme**: Modern UI with extensive customization +- **Plugins**: + - `mkdocstrings` - API documentation from docstrings + - `mkdocs-jupyter` - Notebook integration + - `mkdocs-redirects` - URL management + - Custom hooks for code processing +- **Custom Processing**: `hide_lines.py` removes code marked with `# <%hide%>` +- **Redirect Management**: Comprehensive redirect maps for moved content + +### Writing Documentation +- Follow templates in `docs/templates/` for consistency +- Grade 10 reading level for accessibility +- All code examples must be runnable +- Include complete imports and environment setup +- Progressive complexity: simple → advanced + +## Project Structure +- `instructor/` - Core library code + - Base classes (`client.py`): `Instructor` and `AsyncInstructor` + - Provider clients (`client_*.py`): Factory functions for each provider + - DSL components (`dsl/`): Partial, Iterable, Maybe, Citation extensions + - Core logic: `patch.py`, `process_response.py`, `function_calls.py` + - CLI tools (`cli/`): Batch processing, file management, usage tracking +- `tests/` - Test suite organized by provider + - Provider-specific tests in `tests/llm/test_/` + - Evaluation tests for model capabilities + - No mocking - all tests use real API calls +- `docs/` - MkDocs documentation + - `concepts/` - Core concepts and features + - `integrations/` - Provider-specific guides + - `examples/` - Practical examples and cookbooks + - `learning/` - Progressive tutorial path + - `blog/posts/` - Technical articles and announcements + - `templates/` - Templates for new docs (provider, concept, cookbook) +- `examples/` - Runnable code examples + - Feature demos: caching, streaming, validation, parallel processing + - Use cases: classification, extraction, knowledge graphs + - Provider examples: anthropic, openai, groq, mistral + - Each example has `run.py` as the main entry point +- `typings/` - Type stubs for untyped dependencies + +## Documentation Structure +- **Getting Started Path**: Installation → First Extraction → Response Models → Structured Outputs +- **Learning Patterns**: Simple Objects → Lists → Nested Structures → Validation → Streaming +- **Example Organization**: Self-contained directories with runnable code demonstrating specific features +- **Blog Posts**: Technical deep-dives with code examples in `docs/blog/posts/` + +## Example Patterns +When creating examples: +- Use `run.py` as the main file name +- Include clear imports: stdlib → third-party → instructor +- Define Pydantic models with descriptive fields +- Show expected output in comments +- Handle errors appropriately +- Make examples self-contained and runnable + +## Dependency Management + +### Core Dependencies +- **Minimal core**: `openai`, `pydantic`, `docstring-parser`, `typer`, `rich` +- **Python requirement**: `<4.0,>=3.9` +- **Pydantic version**: `<3.0.0,>=2.8.0` (constrained for stability) + +### Optional Dependencies +Provider-specific packages as extras: +```bash +# Install with specific provider +pip install "instructor[anthropic]" +pip install "instructor[google-generativeai]" +pip install "instructor[groq]" +``` + +### Development Dependencies +```bash +# Install all development dependencies +uv pip install -e ".[dev]" +``` +Includes: +- `pyright<2.0.0` - Type checking +- `pytest` and `pytest-asyncio` - Testing +- `ruff` - Linting and formatting +- `coverage` - Test coverage +- `mkdocs` and plugins - Documentation + +### Version Constraints +- **Upper bounds on all dependencies** for stability +- **Provider SDK versions** pinned to tested versions +- **Test dependencies** include evaluation frameworks -Library enables structured LLM outputs using Pydantic models across multiple providers with type safety. +### Managing Dependencies +- Update `pyproject.toml` for new dependencies +- Test with multiple Python versions (3.9-3.12) +- Run full test suite after dependency updates +- Document any provider-specific version requirements +The library enables structured LLM outputs using Pydantic models across multiple providers with type safety. \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..1007edcb2 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,383 @@ +# Contributing to Instructor + +Thank you for considering contributing to Instructor! This document provides guidelines and instructions to help you contribute effectively. + +## Table of Contents + +- [Contributing to Instructor](#contributing-to-instructor) + - [Table of Contents](#table-of-contents) + - [Code of Conduct](#code-of-conduct) + - [Getting Started](#getting-started) + - [Environment Setup](#environment-setup) + - [Development Workflow](#development-workflow) + - [Dependency Management](#dependency-management) + - [Using UV](#using-uv) + - [Using Poetry](#using-poetry) + - [Working with Optional Dependencies](#working-with-optional-dependencies) + - [How to Contribute](#how-to-contribute) + - [Reporting Bugs](#reporting-bugs) + - [Feature Requests](#feature-requests) + - [Pull Requests](#pull-requests) + - [Writing Documentation](#writing-documentation) + - [Contributing to Evals](#contributing-to-evals) + - [Code Style Guidelines](#code-style-guidelines) + - [Conventional Comments](#conventional-comments) + - [Conventional Commits](#conventional-commits) + - [Types](#types) + - [Examples](#examples) + - [Testing](#testing) + - [Branch and Release Process](#branch-and-release-process) + - [Using Cursor for PR Creation](#using-cursor-for-pr-creation) + - [License](#license) + +## Code of Conduct + +By participating in this project, you agree to abide by our code of conduct: treat everyone with respect, be constructive in your communication, and focus on the technical aspects of the contributions. + +## Getting Started + +### Environment Setup + +1. **Fork the Repository**: Click the "Fork" button at the top right of the [repository page](https://github.com/instructor-ai/instructor). + +2. **Clone Your Fork**: + ```bash + git clone https://github.com/YOUR-USERNAME/instructor.git + cd instructor + ``` + +3. **Set up Remote**: + ```bash + git remote add upstream https://github.com/instructor-ai/instructor.git + ``` + +4. **Install UV** (recommended): + ```bash + # macOS/Linux + curl -LsSf https://astral.sh/uv/install.sh | sh + + # Windows PowerShell + powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" + ``` + +5. **Install Dependencies**: + ```bash + # Using uv (recommended) + uv pip install -e ".[dev,docs,test-docs]" + + # Using poetry + poetry install --with dev,docs,test-docs + + # For specific providers, add the provider name as an extra + # Example: uv pip install -e ".[dev,docs,test-docs,anthropic]" + ``` + +6. **Set up Pre-commit**: + ```bash + pip install pre-commit + pre-commit install + ``` + +### Development Workflow + +1. **Create a Branch**: + ```bash + git checkout -b feature/your-feature-name + ``` + +2. **Make Your Changes and Commit**: + ```bash + git add . + git commit -m "Your descriptive commit message" + ``` + +3. **Keep Your Branch Updated**: + ```bash + git fetch upstream + git rebase upstream/main + ``` + +4. **Push Changes**: + ```bash + git push origin feature/your-feature-name + ``` + +### Dependency Management + +We support both UV and Poetry for dependency management. Choose the tool that works best for you: + +#### Using UV + +UV is a fast Python package installer and resolver. It's recommended for day-to-day development in Instructor. + +```bash +# Install uv +curl -LsSf https://astral.sh/uv/install.sh | sh + +# Install project and development dependencies +uv pip install -e ".[dev,docs]" + +# Adding a new dependency (example) +uv pip install new-package +``` + +Key UV commands: +- `uv pip install -e .` - Install the project in editable mode +- `uv pip install -e ".[dev]"` - Install with development extras +- `uv pip freeze > requirements.txt` - Generate requirements file +- `uv self update` - Update UV to the latest version + +#### Using Poetry + +Poetry provides more comprehensive dependency management and packaging. + +```bash +# Install Poetry +curl -sSL https://install.python-poetry.org | python3 - + +# Install dependencies including development deps +poetry install --with dev,docs + +# Add a new dependency +poetry add package-name + +# Add a new development dependency +poetry add --group dev package-name +``` + +Key Poetry commands: +- `poetry shell` - Activate the virtual environment +- `poetry run python -m pytest` - Run commands within the virtual environment +- `poetry update` - Update dependencies to their latest versions + +### Working with Optional Dependencies + +Instructor uses optional dependencies to support different LLM providers. When adding integration for a new provider: + +1. **Update pyproject.toml**: Add your provider's dependencies to both `[project.optional-dependencies]` and `[dependency-groups]`: + + ```toml + [project.optional-dependencies] + # Add your provider here + my-provider = ["my-provider-sdk>=1.0.0,<2.0.0"] + + [dependency-groups] + # Also add to dependency groups + my-provider = ["my-provider-sdk>=1.0.0,<2.0.0"] + ``` + +2. **Create Provider Client**: Implement your provider client in `instructor/clients/client_myprovider.py` + +3. **Add Tests**: Create tests in `tests/llm/test_myprovider/` + +4. **Document Installation**: Update the documentation to include installation instructions: + ``` + # Install with your provider support + uv pip install "instructor[my-provider]" + # or + poetry install --with my-provider + ``` + +## How to Contribute + +### Reporting Bugs + +If you find a bug, please create an issue on [our issue tracker](https://github.com/instructor-ai/instructor/issues) with: + +1. A clear, descriptive title +2. A detailed description including: + - The `response_model` you are using + - The `messages` you are using + - The `model` you are using + - Steps to reproduce the bug + - The expected behavior and what went wrong + - Your environment (Python version, OS, package versions) + +### Feature Requests + +For feature requests, please create an issue describing: + +1. The problem your feature would solve +2. How your solution would work +3. Alternatives you've considered +4. Examples of how the feature would be used + +### Pull Requests + +1. **Create a Pull Request** from your fork to the main repository. +2. **Fill out the PR template** with details about your changes. +3. **Address review feedback** and make requested changes. +4. **Wait for CI checks** to pass. +5. Once approved, a maintainer will merge your PR. + +### Writing Documentation + +Documentation improvements are always welcome! Follow these guidelines: + +1. Documentation is written in Markdown format in the `docs/` directory +2. When creating new markdown files, add them to `mkdocs.yml` under the appropriate section +3. Follow the existing hierarchy and structure +4. Use a grade 10 reading level (simple, clear language) +5. Include working code examples +6. Add links to related documentation + +### Contributing to Evals + +We encourage contributions to our evaluation tests: + +1. Explore existing evals in the [evals directory](https://github.com/instructor-ai/instructor/tree/main/tests/llm/test_openai/evals) +2. Contribute new evals as pytest tests +3. Evals should test specific capabilities or edge cases of the library or models +4. Follow the existing patterns for structuring eval tests + +## Code Style Guidelines + +We use automated tools to maintain consistent code style: + +- **Ruff**: For linting and formatting +- **PyRight**: For type checking +- **Black**: For code formatting (enforced by Ruff) + +General guidelines: + +- **Typing**: Use strict typing with annotations for all functions and variables +- **Imports**: Standard lib → third-party → local imports +- **Models**: Define structured outputs as Pydantic BaseModel subclasses +- **Naming**: snake_case for functions/variables, PascalCase for classes +- **Error Handling**: Use custom exceptions from exceptions.py, validate with Pydantic +- **Comments**: Docstrings for public functions, inline comments for complex logic + +### Conventional Comments + +We use conventional comments in code reviews and commit messages. This helps make feedback clearer and more actionable: + +``` +