Hola, good folk of Metaflow 👋 I'm having an issue in running `FlowSpec`s in Argo Workflows. Any pointers on what I'm missing or how to deal with this would be much appreciated! 🙏 • I've defined a

FlowSpec

which accepts an optional parameter as follows:

class TestFlow(FlowSpec):
    only_sites = Parameter(
        "only-sites",
        default=None,
        type=str,
        help="Comma-separated list of sites to test (e.g. 'id_1.geojson,id_2.geojson')",
    )
    ...

• Subsequent code inspects the type of this parameter - where the value is

None

, we need certain behaviour. • This Flow works fine when executed locally: the

self.only_sites

attribute is set to

None

, as expected. • When run in Argo Workflows, however, the behaviour is different. • In particular, the WorkflowTemplate has an

only_sites

parameter which is a string value of

"null"

:

{
  "name": "only-sites",
  "value": "null",
  "description": "Comma-separated list of sites to test (e.g. 'id_1.geojson,id_2.geojson')"
}

• This is then passed to the resulting Workflow, which causes unexpected behaviour within the Flow for users that do not pass any value for this parameter. I'd really rather not need to add

if self.only_sites is None or self.only_sites == "null"

throughout my code to handle this difference in behaviour between environments. What more elegant ways are there for handling this?

I believe it is like this because of the manner in which Argo Workflows handles parameters. It is not possible to have an unspecified parameter in a workflow. In Metaflow the parameters are managed by Click under-the-hood. I believe you can pass in

kwargs

to Metaflow parameters and these are then added to the corresponding option in click. So I wonder if you can use this to handle this case? Perhaps you can specify a callback in the parameter kwargs and use the callback to set the parameter to None when the value is "null"?

Thanks for the suggestion! It does appear that I can add a

callback

into `Parameter`s:

class TestFlow(FlowSpec):
    only_sites = Parameter(
        "only-sites",
        default=None,
        type=str,
        help="Comma-separated list of sites to test (e.g. 'id_1.geojson,id_2.geojson')",
        callback=lambda _, x: None if x == "null" else x,
    )

That said, it does seem that this solution is equivalent to me adding

if self.only_sites is None or self.only_sites == "null"

conditionals: the user is still left to manage the scenario where not passing any value should be handled by the interface if I have declared a default value of

None

.

If it works, perhaps the Metaflow people would be willing to add it under the hood?

I just stumbled on this same issue, does anyone know if there is a built in way to handle a default None for metaflow running on argo?