schema: Add prompt validation check
#3564
Conversation
for more information, see https://pre-commit.ci
Unit Test Results 6 files ±0 6 suites ±0 1h 24m 17s ⏱️ - 3m 53s For more details on these failures, see this check. Results for commit 4fe83e6. ± Comparison against base commit 6dfc68f. ♻️ This comment has been updated with latest results. |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
prompt.template validation checkprompt validation check
for more information, see https://pre-commit.ci
|
@jeffkinnison @arnavgarg1 @justinxzhao PR is complete and affected tests have succeeded. Remaining test failures seem unrelated but they've happened on several runs now. |
| # return | ||
| from ludwig.schema.llms.prompt import PromptConfig, RetrievalConfig | ||
|
|
||
| if config.prompt == PromptConfig(): |
There was a problem hiding this comment.
Is the idea here that if it's the default prompt config, we don't need to check prompt requirements?
There was a problem hiding this comment.
Exactly. It's a confusing check, which is why ideally I think prompt should be null by default.
ludwig/config_validation/checks.py
Outdated
| "A template must contain at least one reference to a column or, in the case of zero/few-shot " | ||
| "learning, at least the sample keyword `{__sample__}`!" |
There was a problem hiding this comment.
How about:
"A template must contain at least one reference to a column or the sample keyword {__sample__} for a JSON-serialized representation of non-output feature columns."
Since {__sample__} can be used for regular fine-tuning as well.
CC: @arnavgarg1
No description provided.