Description
Reproduction
Issue discription
The current code type hint of formatting_func argument in SFTTrainer is formatting_func: Optional[Union[Callable[[dict], str], Callable[[dict], list[str]]]] = None, is not correct. It should be formatting_func: Optional[Union[Callable[[dict], str]] = None,. Because it does not support Callable[[dict], list[str]]]] now.
Reason
This function will be sent to _prepare_dataset. As type hint in _prepare_dataset, it is formatting_func: Optional[Callable[[dict], str]],.
Why it happen
In the history of designing this argument, it firstly supported a callable function which can return a list of string. See here. In the following development, as description of this PR, it supported a callable function which return a string. And now in the current version, it does not support return a list of string anymore. I think it is a slient change between different trl versions without clear documentation. By the way, I spent a lot of time on upgrade the trl to run an old script based on older trl, and finally find the return type of this argument has changed.
Now the only document to show the return of formatting_func is:
def formatting_func(example):
text = f"### Question: {example['question']}\n ### Answer: {example['answer']}"
return text
training_args = SFTConfig(packing=True)
trainer = SFTTrainer(
"facebook/opt-350m",
train_dataset=dataset,
args=training_args,
formatting_func=formatting_func
)
trainer.train()System Info
trl=0.18.1
Checklist
- I have checked that my issue isn't already filed (see open issues)
- I have included my system information
- Any code provided is minimal, complete, and reproducible (more on MREs)
- Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
- Any traceback provided is complete