For anyone running metaflow on a local kubernetes cluster on a Mac with an M-chip, has anyone run into or resolved an issue like this?

│ Setting up task environment.                                                                                                                                                                                    │
│ Downloading code package...                                                                                                                                                                                     │
│ Code package downloaded.                                                                                                                                                                                        │
│ Task is starting.                                                                                                                                                                                               │
│ Bootstrapping virtual environment...                                                                                                                                                                            │
│ Bootstrap failed while executing: set -e;                                                                                                                                                                       │
│             tmpfile=$(mktemp);                                                                                                                                                                                  │
│             echo "@EXPLICIT" > "$tmpfile";                                                                                                                                                                      │
│             ls -d /metaflow/.pkgs/conda/*/* >> "$tmpfile";                                                                                                                                                      │
│             export PATH=$PATH:$(pwd)/micromamba;                                                                                                                                                                │
│             export CONDA_PKGS_DIRS=$(pwd)/micromamba/pkgs;                                                                                                                                                      │
│             export MAMBA_NO_LOW_SPEED_LIMIT=1;                                                                                                                                                                  │
│             export MAMBA_USE_INDEX_CACHE=1;                                                                                                                                                                     │
│             export MAMBA_NO_PROGRESS_BARS=1;                                                                                                                                                                    │
│             export CONDA_FETCH_THREADS=1;                                                                                                                                                                       │
│             micromamba create --yes --offline --no-deps                 --safety-checks=disabled --no-extra-safety-checks                 --prefix /metaflow/linux-64/77bf24d977d68e1 --file "$tmpfile"         │
│             rm "$tmpfile"                                                                                                                                                                                       │
│ Stdout:                                                                                                                                                                                                         │
│ Stderr: qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory                                                                                                                    │

Errors like this are pretty common IMO when working on a M1/2/3, and are usually resolved by building a custom --platform=linux/amd64 image, then making that available to your cluster. I am going to try that now and report back, but if anyone has a more elegant fix I'm interested

This is likely because metsflow assumes that remote runs run on Linux. I’ll check if there is a way to override that but I don’t think so. Could be added though.

Do you know if there is a way to define a default image via a decorator? I could override

METAFLOW_DEFAULT_CONTAINER_IMAGE

, but I would prefer to do something like

@kubernetes(image="my-custom_image:v0.0.1")
class MyFlow(FlowSpec):

    @step