8000 TypeError: DaskTaskRunner.submit() got an unexpected keyword argument 'key' · Issue #6641 · PrefectHQ/prefect · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

TypeError: DaskTaskRunner.submit() got an unexpected keyword argument 'key' #6641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
4 tasks done
mikkelduif opened this issue Aug 31, 2022 · 7 comments
Closed
4 tasks done

TypeError: DaskTaskRunner.submit() got an unexpected keyword argument 'key' #6641

mikkelduif opened this issue Aug 31, 2022 · 7 comments
Assignees
@anticorrelator
Labels
bug Something isn't working

Comments

@mikkelduif
Copy link

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I searched the Prefect documentation for this issue.
  • I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

Seems like you can no longer submit DaskTask in prefect-2.3.0

I think it's related to this commit:
1f10bd8

Reproduction

from prefect import flow, task
from prefect_dask.task_runners import DaskTaskRunner


@task
def mytask():
    pass


@flow(task_runner=DaskTaskRunner())
def myflow():
    mytask.submit()


if __name__ == "__main__":
    myflow()

Error

Traceback (most recent call last):
  File ".../test.py", line 16, in <module>
    myflow()
  File ".../.venv/lib/python3.10/site-packages/prefect/flows.py", line 384, in __call__
    return enter_flow_run_engine_from_flow_call(
  File ".../.venv/lib/python3.10/site-packages/prefect/engine.py", line 158, in enter_flow_run_engine_from_flow_call
    return anyio.run(begin_run)
  File ".../.venv/lib/python3.10/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File ".../.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File ".../.pyenv/versions/3.10.5/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File ".../.pyenv/versions/3.10.5/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File ".../.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File ".../.venv/lib/python3.10/site-packages/prefect/client.py", line 103, in with_injected_client
    return await fn(*args, **kwargs)
  File ".../.venv/lib/python3.10/site-packages/prefect/engine.py", line 231, in create_then_begin_flow_run
    state = await begin_flow_run(
  File ".../.venv/lib/python3.10/site-packages/prefect/engine.py", line 339, in begin_flow_run
    async with AsyncExitStack() as stack:
  File ".../.pyenv/versions/3.10.5/lib/python3.10/contextlib.py", line 714, in __aexit__
    raise exc_details[1]
  File ".../.pyenv/versions/3.10.5/lib/python3.10/contextlib.py", line 217, in __aexit__
    await self.gen.athrow(typ, value, traceback)
  File ".../.venv/lib/python3.10/site-packages/prefect/engine.py", line 1338, in report_flow_run_crashes
    raise exc from None
  File ".../.venv/lib/python3.10/site-packages/prefect/engine.py", line 1321, in report_flow_run_crashes
    yield
  File ".../.pyenv/versions/3.10.5/lib/python3.10/contextlib.py", line 697, in __aexit__
    cb_suppress = await cb(*exc_details)
  File ".../.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File ".../.venv/lib/python3.10/site-packages/prefect/engine.py", line 938, in create_task_run_then_submit
    await submit_task_run(
  File ".../.venv/lib/python3.10/site-packages/prefect/engine.py", line 994, in submit_task_run
    future = await task_runner.submit(
TypeError: DaskTaskRunner.submit() got an unexpected keyword argument 'key'

Versions

Version:             2.3.0
API version:         0.8.0
Python version:      3.10.5
Git commit:          8d9316c0
Built:               Tue, Aug 30, 2022 5:30 PM
OS/Arch:             darwin/arm64
Profile:             mikkelduif-localhost
Server type:         cloud


In [1]: import prefect_dask
In [2]: prefect_dask.__version__
Out[2]: '0.1.2'

Additional context

No response
@mikkelduif mikkelduif added bug Something isn't working status:triage labels Aug 31, 2022
@bunchesofdonald
Copy link
Contributor

This is definitely a bug, we'll get it looked at soon. Thanks for the report @mikkelduif!

@bunchesofdonald
Copy link
Contributor

CC: @zangell44 / @anticorrelator

@zanieb
Copy link
Contributor
zanieb commented Aug 31, 2022

Ah yes we need to be coordinating changes to the collection-based task runner alongside changes like this. cc @ahuang11 as well.

I think that the changes might not even be compatible with the way the Dask task runner optimizes futures. We'll need to ensure that the passed partial keyword arguments can be scanned.

I think all major changes to task runners should be replicated in Dask alongside the core pull request to ensure that we're not going to change implementation details that are relevant for distributed runners.

@anticorrelator anticorrelator mentioned this issue Sep 2, 2022
Merged
1 task
@BitTheByte
Copy link
Contributor

is there an ETA for this? I started seeing this in our production flows

@desertaxle
Copy link
Member

prefect-dask 0.2.0 has been released which resolves this issue.

@desertaxle desertaxle mentioned this issue Sep 5, 2022
Closed
@zanieb zanieb closed this as completed Sep 5, 2022
@MuFaheemkhan
Copy link

tried prefect-dask 0.2.0 got the following error after running same flow multiple times, first time it ran fine.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 101, in with_injected_client
async with client_context as client:
File "/usr/local/lib/python3.10/site-packages/prefect/client.py", line 1929, in aexit
return await self._exit_stack.aexit(*exc_info)
File "/usr/local/lib/python3.10/contextlib.py", line 714, in aexit
raise exc_details[1]
File "/usr/local/lib/python3.10/contextlib.py", line 697, in aexit
cb_suppress = await cb(*exc_details)
File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1997, in aexit
await self._transport.aexit(exc_type, exc_value, traceback)
File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 332, in aexit
await self._pool.aexit(exc_type, exc_value, traceback)
File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 326, in aexit
await self.aclose()
File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 312, in aclose
raise RuntimeError(
RuntimeError: The connection pool was closed while 10 HTTP requests/responses were still in-flight.
ne.

@desertaxle
Copy link
Member

@MuFaheemkhan could you please open a new issue in the prefect-dask repository with a minimal reproducible example where you are seeing this issue so that we can investigate the exception that you're seeing?

This was referenced Sep 9, 2022
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@anticorrelator anticorrelator

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@bunchesofdonald @anticorrelator @zanieb @desertaxle @BitTheByte @mikkelduif @MuFaheemkhan
0