Open
Description
Hello,
I would like to report a problem with the translated prompts for HumanEval benchmark task 140.
This is the original Python prompt:
def fix_spaces(text):
"""
Given a string text, replace all spaces in it with underscores,
and if a string has more than 2 consecutive spaces,
then replace all consecutive spaces with -
fix_spaces("Example") == "Example"
fix_spaces("Example 1") == "Example_1"
fix_spaces(" Example 2") == "_Example_2"
fix_spaces(" Example 3") == "_Example-3"
"""Consider the translation in Ruby (p.s., this problem is present in all translations, from what I noticed).
# Given a string text, replace all spaces in it with underscores,
# and if a string has more than 2 consecutive spaces,
# then replace all consecutive spaces with -
# >>> fix_spaces.call(" Example") <----- Extra space here. The Input string should be "Example"
# "Example"
# >>> fix_spaces.call(" Example 1") <----- Extra space here. The Input string should be "Example 1"
# "Example_1"
# >>> fix_spaces.call(" Example 2")
# "_Example_2"
# >>> fix_spaces.call(" Example 3") <----- Missing spaces here. After "Example" there are 3 whitespaces
# "_Example-3"
def fix_spaces(text)As you can see, the first two doctests have some extra spaces in their input string, while the last one requires 3 whitespaces instead of 1, after the word "Example".
This is a problem because adding or removing one space alters the expected behavior of the problem and may have a negative impact on the model’s completion.
Metadata
Metadata
Assignees
Labels
No labels