Hi is it possible to further enhance the `cli` to add extra Outerbounds #dev-metaflow

Hi, is it possible to further enhance the `cli` to...

fast-vr-44972

12/11/2023, 6:32 PM

Hi, is it possible to further enhance the

cli

to add extra options just for our internal usage? We have some stuff like what metaflow deployment the job should run on and currently we are hiding all

cli

options you see and only allowing the

env

and

username

since we call it inside

subprocess

. I was thinking if there was some function that I could import which has more or less all the expected arguments I can further add our options on top of it.

Copy code

python flow_name.py argo-workflows create --help

dry-beach-38304

12/11/2023, 8:59 PM

you want to add something to

argo-workflows create

or do you want to add a completely separate command like

my-argo-workflows

fast-vr-44972

12/12/2023, 10:03 AM

Basically separate command which adds some extra options but underneath it's still calling

argo-workflows create

Copy code

python deploy_flow.py --env <prd|stg|dev>

dry-beach-38304

12/12/2023, 10:37 AM

Separate command is easier I suppose. I’ll post tomorrow but you can build an extension and add your own cli. Under the hood you can parse your arguments and then call rhr relevant Argo functions.

thankyou 1

dry-beach-38304

12/12/2023, 10:37 AM

If it is a separate command like this, it’s pretty straightforward.

dry-beach-38304

12/12/2023, 5:13 PM

OK, so you can look here: https://github.com/Netflix/metaflow-extensions-template

dry-beach-38304

12/12/2023, 5:13 PM

you then want to add things to

CLIS_DESC

in this file: https://github.com/Netflix/metaflow-extensions-template/blob/master/metaflow_extensions/org/plugins/mfextinit_org.py

dry-beach-38304

12/12/2023, 5:15 PM

it will look something like this:

Copy code

CLIS_DESC = [("mycli", ".mycli.mycli_cli.cli")]

to add something to do

myflow.py mycli …

dry-beach-38304

12/12/2023, 5:16 PM

You then need to have in the

plugins

folder there a file like

mycli/mycli_cli.py

and inside that have something like:

Copy code

@click.group()
def cli():
    pass


@cli.group(help="Commands related to my awesome CLI")
def mycli():
    pass

dry-beach-38304

12/12/2023, 5:17 PM

Define your options/etc there as you would with any other click based CLI and you should be good to go.

dry-beach-38304

12/12/2023, 5:18 PM

If you want to augment argo stuff, you can look at how argo is implemented starting here: https://github.com/Netflix/metaflow/blob/master/metaflow/plugins/argo/argo_workflows_cli.py#L55 and pick and choose what you need from there.

dry-beach-38304

12/12/2023, 5:18 PM

Note that the extension mechanism is not officially supported and there is no guarantee of it not having breaking change.

dry-beach-38304

12/12/2023, 5:18 PM

That being said, it’s been stable for a while and we use it so it’s not going to randomly stop working and/or provide less functionality than it currently is.

dry-beach-38304

12/12/2023, 5:19 PM

Let me know if you have more questions. This is just a high level overview but should at least get you started.

8 Views

Open in Slack

Previous Next