Add AI task structured output #148083
Open
Add AI task structured output #148083
+303
−7
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
There was a problem hiding this comment.
Pull Request Overview
This PR adds optional structured output support to the AI Task integration, enabling callers to supply a schema for the AI to generate data in a typed, field-based format.
- Introduces a new
structureparameter inasync_generate_data, guarded by a newGENERATE_STRUCTURED_DATAentity feature - Validates user‐supplied structure definitions via a new
_validate_structurehelper andSTRUCTURE_FIELD_SCHEMA - Extends tests and mocks to cover structured‐output generation and invalid‐schema handling
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/components/ai_task/test_task.py | Added tests for unsupported structured‐output cases |
| tests/components/ai_task/test_init.py | Added tests for valid and invalid structure schemas |
| tests/components/ai_task/test_entity.py | Added structured‐output test on entity API |
| tests/components/ai_task/conftest.py | Updated mock to advertise and handle structured data |
| homeassistant/helpers/service.py | Registered ai_task in base component lookup |
| homeassistant/components/ai_task/task.py | Added structure param and feature check in API |
| homeassistant/components/ai_task/const.py | Defined ATTR_STRUCTURE, ATTR_REQUIRED, and new feature flag |
| homeassistant/components/ai_task/init.py | Added service schema and _validate_structure |
| homeassistant/components/ai_task/services.yaml | Declared structure parameter for generate_data |
| homeassistant/components/ai_task/strings.json | Added localization for the structure parameter |
Comments suppressed due to low confidence (2)
homeassistant/components/ai_task/services.yaml:23
- The
examplevalue is not valid JSON and has mismatched braces. Update it to a valid JSON string, e.g.:example: '{"name": {"selector": {"text": {}}, "description": "Name of the user", "required": true}, "age": {"selector": {"number": {}}, "description": "Age of the user"}}'.
example: '{ "name": { "selector": { "text": }, "description": "Name of the user", "required": "True" } } }, "age": { "selector": { "number": }, "description": "Age of the user" } }'
homeassistant/components/ai_task/services.yaml:24
- The
object:selector is specified without an explicit mapping. It should beobject: {}to be valid YAML.
selector:
balloob
reviewed
Jul 4, 2025
| @@ -32,3 +34,6 @@ class AITaskEntityFeature(IntFlag): | |||
|
|
|||
| GENERATE_DATA = 1 | |||
| """Generate data based on instructions.""" | |||
|
|
|||
| GENERATE_STRUCTURED_DATA = 2 | |||
There was a problem hiding this comment.
No need for an extra supported feature. It's folded into GENERATE_DATA and we won't allow AI Task entities that cannot do that.
balloob
reviewed
Jul 4, 2025
| init_components: None, | ||
| mock_ai_task_entity: MockAITaskEntity, | ||
| ) -> None: | ||
| """Test the entity can generate structured data with a top level object schemea.""" |
There was a problem hiding this comment.
Suggested change
| """Test the entity can generate structured data with a top level object schemea.""" | |
| """Test the entity can generate structured data with a top level object schema.""" |
balloob
reviewed
Jul 4, 2025
Comment on lines
+131
to
+151
|
|
||
|
|
||
| async def test_run_task_structure_unsupported_feature( | ||
| hass: HomeAssistant, | ||
| init_components: None, | ||
| mock_ai_task_entity: MockAITaskEntity, | ||
| ) -> None: | ||
| """Test running a task with an unknown entity.""" | ||
|
|
||
| mock_ai_task_entity.supported_features = AITaskEntityFeature.GENERATE_DATA | ||
| with pytest.raises( | ||
| HomeAssistantError, | ||
| match="AI Task entity ai_task.test_task_entity does not support generating structured data", | ||
| ): | ||
| await async_generate_data( | ||
| hass, | ||
| task_name="Test Task", | ||
| instructions="Test prompt", | ||
| entity_id=TEST_ENTITY_ID, | ||
| structure=vol.Schema({vol.Required("name"): selector.TextSelector()}), | ||
| ) |
There was a problem hiding this comment.
We can remove this one
Suggested change
| async def test_run_task_structure_unsupported_feature( | |
| hass: HomeAssistant, | |
| init_components: None, | |
| mock_ai_task_entity: MockAITaskEntity, | |
| ) -> None: | |
| """Test running a task with an unknown entity.""" | |
| mock_ai_task_entity.supported_features = AITaskEntityFeature.GENERATE_DATA | |
| with pytest.raises( | |
| HomeAssistantError, | |
| match="AI Task entity ai_task.test_task_entity does not support generating structured data", | |
| ): | |
| await async_generate_data( | |
| hass, | |
| task_name="Test Task", | |
| instructions="Test prompt", | |
| entity_id=TEST_ENTITY_ID, | |
| structure=vol.Schema({vol.Required("name"): selector.TextSelector()}), | |
| ) |
balloob
approved these changes
Jul 4, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to co
7220
mment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed change
Add AI Task structured output support. The generate data service accepts an input schema and APIs to allow an AI task entity to generate a response with an object conforming to that structure. Integrations are responsible for converting the schema (based on selectors) to the appropriate type for the API (e.g. json schema)
Architecture discussion: home-assistant/architecture#1216 - has been approved by core team according to balloob, but not yet marked as approved.
Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all.To help with the load of incoming pull requests: