Hi everybody! I am getting an error, with Kubernetes + GCP. I created and publish to docker Hub my image. https://hub.docker.com/layers/simonprudhomme/noibu_container/latest/images/sha256-a5[…]08462196692c75e51fa1ca5d8a0846d4334d0e909e6da9aba0?tab=layers Here is a copy of the Dockerfile:

FROM --platform=linux/amd64 python:3.9-slim

RUN mkdir /logs && chown 1000 /logs
RUN mkdir /metaflow && chown 1000 /metaflow
ENV HOME=/metaflow
WORKDIR /metaflow
USER 1000

COPY requirements.txt requirements.txt
RUN pip install --no-cache-dir --upgrade pip && \
    pip install --no-cache-dir -r requirements.txt

I have sucessfully configured : metaflow configure gcp and metaflow configure kubernetes However, when I try to use my image, for example in this basic Flow:

from metaflow import FlowSpec, step, kubernetes, conda, Parameter
import pandas as pd
import numpy as np

class test():
    def __init__(self):
        self.dataframes = {}

    def create_random_dataframes(self):
        for i in range(10):
            self.dataframes[i] = pd.DataFrame(np.random.randint(0,100,size=(10, 10)))

class LinearFlow(FlowSpec):
    @step
    def start(self):
        self.test = test()
        self.test.create_random_dataframes()
        self.next(self.a)

    @kubernetes(image='simonprudhomme/noibu_container:latest')
    @step
    def a(self):
        for key, values in self.test.dataframes.items():
            print(values.shape)
        self.next(self.end)

    @step
    def end(self):
        pass

if __name__ == '__main__':
    LinearFlow()

I am getting an error:

2023-01-13 10:50:52.059 [116/a/494 (pid 18466)] Kubernetes error:

Here are the pod logs:

{
insertId: "odoettx5jbnlsvfz"
labels: {5}
logName: "projects/noibu-data-science/logs/stdout"
receiveTimestamp: "2023-01-13T15:50:50.095900795Z"
resource: {2}
severity: "INFO"
textPayload: "Failed to download code package from <gs://storage-bucket-metaflow-datascience-metaflow-default/tf-full-stack-sysroot/LinearFlow/data/67/6789fe275d9673188a489797e409014bbcf2304c> after 6 tries. Exiting..."
timestamp: "2023-01-13T15:50:48.531286261Z"
}

I am using a MacbookPro M1, Python 3.9.6, Metaflow 2.7.18.