fancy-mouse-14245
12/07/2023, 10:01 AMclass SampleFlow(FlowSpec):
@step
def start(self):
pass
self.next(self.a)
@resources(cpu=16, memory=60000)
@step
def a(self):
pass
self.next(self.train_and_infer)
@resources(gpu=1, cpu=32, memory=124000)
@step
def train_and_infer(self):
pass
self.next(self.c)
@resources(cpu=32, memory=62000)
@step
def c(self):
pass
self.next(self.end)
@step
def end(self):
pass
When we are running the flow. the whole flow is running in a gpu instance. Even tho we have specified to use gpu in only one step.
We are using aws batch with step functions and we have deployed metaflow using terraform with stop instances.