https://outerbounds.com/ logo
Join Slack
Powered by
Hmm, I guess what I am looking for is conditional ...
# ask-metaflow
b
Hmm, I guess what I am looking for is conditional flows - there is a years-long github issue with relevant information https://github.com/Netflix/metaflow/issues/71
✅ 1
s
sorry for the delayed reply!
while conditional branches are not supported directly currently, there are a few pretty decent workarounds https://outerbounds-community.slack.com/archives/C020U025QJK/p1641948076018500?thread_ts=1641946316.015000&cid=C020U025QJK
if you can tell more about your use case, I can provide more specific ideas
b
I think the root of the issue is that my flow depends on a config file which basically defines a 
foreach
of the hyperparameters to try for each library. So some (valid) config files can result in a 
foreach
list of size 0 for library 1, but a 
foreach
list of size > 1 for library 2. I think ideally, there would be some conditional flow logic to just skip all the stuff in the 
library 1
branch, and the 
start
method could handle that.
What I’ve had to do is identify that state in the library branch, make a dummy list of size 1 to give to 
foreach
, and then keep creating dummy attributes so the downstream tasks all pass. I can’t really share the flow file directly, but if it would help I could take some time tomorrow to remove any proprietary stuff and send it along for reference
s
makes sense. I'd handle it by having an indicator variable inside foreach denoting whether the task is doing actual useful work or not, say 
is_valid=True
. Then in the join step, I would collect the results from foreach and exclude dummy tasks like:
Copy code
self.results = [task.result for task in inputs if task.is_valid]
clearly it is not optimal but executing one no-op task shouldn't be too bad either
b
Yep, that is pretty much exactly what I ended up doing (my indicator is called 
no_model
)
👍 1
s
old thread but conditionals are now GA
11 Views
Open in Slack
PreviousNext
Powered by