Description
Motivation: Why do you think this is important?
A user requested that Flyte be able to inject environment variables at execution launch time to a workflow run that would get propagated into all downstream tasks, subworkflows, and launch plans.
I think this feature is possibly reasonable, but warrants discussion. The reason is that this breaks referential transparency kinda. The injected environment variable are designed to alter the behavior of the task. However. the idea is that these env vars will not affect caching at all, so that is a bit strange.
The original use-case for this would be to direct service queries. One may run the same task against production services (non-Flyte stuff) or development services for example.
Goal: What should the final outcome look like, ideally?
Not entirely clear, but basically at all the points where users can run workflows (flytectl, console, flyte-cli, flyteremote, etc.) the user would be able to pass along a dict that would get set at the top execution level, and this propagate through all nodes/subexecutions.
Describe alternatives you've considered
It feels more natural to use either a task's domain or the execution's domain to alter behavior, but perhaps not.
Another alternative is to add an explicit service environment variable to all the tasks but this is cumbersome and inelegant.
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
- Yes
Have you read the Code of Conduct?
- Yes