Hello everyone! I have a question regarding the G...
# ask-metaflow
a
Hello everyone! I have a question regarding the GitHub issue mentioned in this link: https://github.com/Netflix/metaflow/issues/245 My use case is as follows: We need to build a complex flow that satisfies various requirements. To illustrate, let's call this complicated flow "FLOW B". It comprises five steps, and it can either be activated manually or triggered within another flow that we'll refer to as "FLOW A". While we could manually duplicate and transfer flow logic, it would be quite tedious, and any modifications made to FLOW B would not be reflected in FLOW A whenever we "copy & paste" elements. Could you suggest an alternative method for composing and embedding flows within flows, where any updates or changes made in FLOW B would not require additional implementation in FLOW A? Thank you for your help, I really appreciate it!
s
@adorable-oxygen-86530 when a flow is deployed to
argo-workflows/step-functions/airflow
, the logic is locked. to update the logic, you will have to deploy the flow again - in this case both flowA and flowB. what is the nature of logic that is common to both flowA and flowB - can you tease them out into a package and invoke the logic out of that package?
a
Hi @square-wire-39606 thanks for your reply. Let me try to give an example: flowA is tightly connected to a build process which generates several binaries required for our business use case.- Due to the nature of the build process this requires a substantial amount of time (somwhat between 30 - 90 minutes;this can be optimized to be faster but at the moment we land somewhere in this time range). One part of the development team is concerned about that task of developing new features into the binaries and at the same time implements improvements into flowA which might change flow structure. A second team requires that flowA is being executed before their flowB is being executed - hence the latest and greatest builds are being used in flowB. We considered using the new @on_trigger decorator but if feels not right, because DevTeamA executes flowA numerous times and flowB must not be executed each time. Only if flowB is being executed flowA shall be executed (ideally without checking that flowA has changed but including the newest changes ofcourse ;) ) Hope this helps to understand my use case better
Hi @ancient-application-36103, do you have an idea how we could solve such a setup using metaflow?