Hello, what is the right approach if I want to schedule 2 flows (on Argo) that have exactly the same code but differ in 1 parameter? For example, something like the following I want to run hourly by

--model val1

and run another

--model val2

. They should be independent from each other, however, they share the same code.

@project(name="hello_flow")
@schedule(hourly=True)
class HelloWorld(FlowSpec):

    model = Parameter(
        "model",
        required=True,
        help="e.g. val1, val2, etc.",
    )

    @step
    def start(self) -> None:
        <http://logger.info|logger.info>("Hello World run started!")
        self.next(self.greet)

    @step
    def greet(self) -> None:
        <http://logger.info|logger.info>(" ####### Hello World from %s! #######", self.model_slug)
        self.next(self.end)

    @step
    def end(self) -> None:
        <http://logger.info|logger.info>("Hello World run completed!")


if __name__ == "__main__":
    HelloWorld()

Look at configs. Instead of parameter, use config.

oh thats new! you mean this, right?

Yes. And yes new.

cool thanks! will try it and let you know how it goes. Appreciate it

@dry-beach-38304 to confirm my understanding, every time we execute

poetry run python hello_flow/flow.py --model <someval> argo-workflows create

it will overwrite the Argo cron workflow definition, with the only change being the value for

model

? it will not create

k

Argo cron workflows if I provide

k

different values for

model

. we would try on our side, but need to update

metaflow

version to test.

You can name the workflow diffrentely. You can use project/brwnch for that purpose (I think for Argo it mangles that in rhr name).

> You can name the workflow diffrentely. hmmm, ideally, these k versions should live on

master

and be deployed as "production" via

--production

. not sure if we want to manage branches for this use case as we want to treat them as independent flows that happen to use the same flow definition. branches feel targeted towards use case for experimentation where at some point you want to merge back to some production branch.

@hundreds-wire-22547 would you like to see

k

different workflow templates being deployed? if so, you can change the name of the deployed template using

--name

-

python flow.py argo-workflows --name foo create

yeah, in our case

k

cron workflows

yep

--name

might suffice. of course, you can also just create a single deployment and trigger it using different parameter values.

cc @great-egg-84692

Sorry. Am out so a bit slow. To continue on this though — agreed with @square-wire-39606 re usage of branch. You can actually even have —branch and —production at the same time. It will do something like name.prod.branch instead of name.test.branch for a regular branch or name.prod for just production. Deployer works great and you can double it up wirh configs too. One thing that comes to mind as I am writing this is that I don’t think the project decorator takes branch and production flags (it only does through cli). That’s something we could add though (need to discuss but makes a lot of sense in light of the addition of configs). You could thrn simply iterate on your configs and everything would be taken care of that way.

@dry-beach-38304 possible to share simple example of deployer + configs?

let me try to get something out soon. Just got back in so catching up.

Thanks @dry-beach-38304! Could I also iterate through configs with different arg for

--name

as an alternative to branch?

you can already do that without 2200 yes.

got it, and whether done programmatically or through file, setting

name

will override any value set currently in the

@project

decorator?

well, I wouldn’t say “override”. Your flow does need to be written to take into account configs. It would look something like this:

from metaflow import project, FlowSpec, config_expr, Config

@project(name=config_expr("config.model")
@schedule(hourly=True)
class HelloWorld(FlowSpec):

    config = Config("config", default_value={"model": "my_default_model"})

In this case, if you have no config passed in, the name of the project would be

my_default_model

and if you did have a config it would use whatever was in that. So in that sense, it would override the default but there is no

@project(name="my_default")

and then magically we override

my_default

if you provide a config. You have to say that the name comes from the config (but can have a default config).

makes sense, thanks for all the help here!

sure, let me know if you have more questiosn. It’s a new feature so there may still be quirks and things that don’t work so feel free to ask for help (and don’t assume that you are doing something wrong 🙂 )

@ripe-alarm-8919 cc