Open
Description
I'm trying to isolate a task in its own directory because it can generate a large number of files. I saw tempdir and thought that it could serve this purpose, but when I run the task with tempdir=True it returns an empty list, whereas by omitting this switch it does return all files.
The task:
@task(tempdir=True)
def splitter(data: File) -> List[File]:
return script(
f'''
split -d -l 2 mydata.txt part.
''',
inputs = [data.stage('mydata.txt')],
outputs = [File(f).stage(str(f)) for f in glob_file('part.*')],
)
Maybe tempdir=True has a different purpose? In any case, what is the recommended approach to isolate tasks in their own directory. (I'm running redun locally in my own laptop)
Thanks.
Metadata
Metadata
Assignees
Labels
No labels