<@U021WU8QDST> This PR may be of interest to your ...
# dev-metaflow
s
@rough-terabyte-71304 This PR may be of interest to your team - https://github.com/Netflix/metaflow/pull/541
🙌 1
🚀 2
d
This would be so helpful for us too! We have been thinking about it and how to go over the problem of overwriting step functions. I think this would solve it
❤️ 1
v
yep, this is a killer feature for larger projects!
there's one more aspect of
@project
that will become handy when we introduce triggering between workflows on SFN. You will be able to create isolated chains/ensembles of workflows using
@project
.
you can use it to A/B experiment complex projects which consists of multiple workflows
pretty much all flows powering content modeling at Netflix use
@project
❤️ 1
d
oh wow! Yeah I am pretty sure we have uses cases that might need this. So excited for this woohoo
👍 1
e
I just finished a read through of the linked doc. This looks like it would be useful for us. At the moment we are using the new
name
option to override the name of the state function and adding a git branch derived suffix. At the moment I think we'd probably switch to using the branch option with the value based on the git branch. I noticed in the Custom Branches section of the doc, that the
better_version
name doesn't include the flow name ("ProjectFlow") - is that intentional?
s
Good catch! I will fix that in the docs.
e
cool thanks for clarifying.. so an additional request would be to allow the decorator to take an override parameter that allows the step function name to be specified. This would basically do the same thing as the
--name
flag, but allow it to be set in the flow itself instead of as an option.
s
Got it. I think for specifying
name
and other step-functions related arguments,
@project
might not be the right place since
--name
and
@project
is mutually exclusive at the moment. We have been thinking of a programmatic API to Metaflow which would enable capturing CLI args in code.
Would a flow level
step_functions
decorator with all the necessary args be helpful for your use case?
e
Yep, that would definitely work