https://outerbounds.com/ logo
Join Slack
Powered by
Thoughts on us introducing `@image()` (container i...
# dev-metaflow
c
Thoughts on us introducing 
@image()
(container image*) vs 
@kfp_step
decorators:
Copy code
@image("foo")
@step
def start(self):
   pass
Copy code
@kfp_step(image="foo")
@step
def start(self):
   pass
https://github.com/zillow/metaflow/pull/120/files
✅ 1
We wouldn't support 
@image()
decorator for the default local runtime (I don't see the need yet: imagine it using local docker or k8s).
a
there's some discussion on this topic https://github.com/Netflix/metaflow/issues/489
c
so it seems that current thinking is:
...and just not add 
@image
 yet, so we defer the decision on this until there is a way in Metaflow to run flows locally in docker via 
--environment=docker
 or something.
and
require (by convention) that all compute decorators that use docker (
@batch
, 
@awslambda
, 
@kubernetes
..) accept 
image=
 parameter and respect 
METAFLOW_DEFAULT_IMAGE
https://github.com/Netflix/metaflow/issues/489#issuecomment-833861470 in our case 
@kfp_step(image="foo")
a
yep
c
Can we have the same Metaflow decorator "name" on a Flow and step level (from the user's perspective)*?
Copy code
@kfp
class HelloFlow(FlowSpec):
    @kfp(image="foo")
    @step
    def start(self, fee):
        pass
we feel forced to introduce 
@kfp_step()
Copy code
@kfp
class HelloFlow(FlowSpec):
    @kfp_step(image="foo")
    @step
    def start(self, fee):
        pass
a
yes I think same name is fine -- we actually plan to implement this for @kubernetes too so you can use it at both levels
and using it at flow level has the same semantics as if you added it to every step in the flow
i.e. it doesn't do anything special at the flow level per se
the only nit I think I'll tweak in the image proposal is maybe it should be called 
METAFLOW_CONTAINER_IMAGE
not 
METAFLOW_DEFAULT_IMAGE
because we already have 
METAFLOW_BATCH_CONTAINER_IMAGE
so the naming is consistent
Open in Slack
PreviousNext
Powered by