hi folks! For context we’ve been using Metaflow for a couple of years on and off at my work. But i’m trying to run the flow in a different way to usual. The flow runs fine locally but fails on AWS Batch (with custom docker container) with an obtuse

NameError

message (using metaflow 2.8.1 – not sure what other information to provide).

File "/app/metaflow/metaflow/_vendor/click/types.py", line 89, in __init__
2023-04-03 07:39:50.134 [1345/start/34057 (pid 1095)] [ddbe8294-cd98-411e-8c5c-1f1ab59aaf87]     self.name = func.__name__
2023-04-03 07:39:50.134 [1345/start/34057 (pid 1095)] [ddbe8294-cd98-411e-8c5c-1f1ab59aaf87]   File "/usr/local/lib/python3.9/typing.py", line 711, in __getattr__
2023-04-03 07:39:50.134 [1345/start/34057 (pid 1095)] [ddbe8294-cd98-411e-8c5c-1f1ab59aaf87]     raise AttributeError(attr)
2023-04-03 07:39:50.135 [1345/start/34057 (pid 1095)] [ddbe8294-cd98-411e-8c5c-1f1ab59aaf87] AttributeError: __name__

@echoing-kangaroo-34845 to rule out any AWS Batch related issues - does the step fail even when it isn't executed on AWS Batch?

no it’s fine when run locally

Do you see the same error when you use Python 3.7 rather than 3.9?

interesting, I haven’t tried that. not sure our code is 3.7 compatible (but would work with 3.8 probably – would that be helpful?). Is there some reason we’d see errors relating to python version in docker but not running locally?

It could be the case. We will create an issue and debug this. Cc @bulky-afternoon-92433

@echoing-kangaroo-34845 trying to reproduce this issue now and thought of some follow-up questions regarding your setup: • is the custom docker container on batch being successfully used for other flows, or is it purpose-built for this one that is failing? • if public, what is the base image for the container? • python version, metaflow version, arch of the machine invoking the

run --with batch

• what kind of environment was the custom docker container built on?

@bulky-afternoon-92433 it’s a custom image (not public). here’s an (edited) version of the dockerfile with what I think are the relevant lines (old failing version – works under 3.8)

FROM --platform=linux/amd64 python:3.9-slim

RUN apt-get update && apt-get install -y git curl

ENV VIRTUAL_ENV=/opt/venv
RUN python -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

RUN pip install 'poetry<1.5'

WORKDIR /app
COPY . /app/
RUN poetry install --without=dev

Heya, unfortunately this has been on a bit of a back-burner for me due to not being able to reproduce the issue, and as I understood from our prior discussions that the issue was resolved for you for the time being with a version change? The hard dependency for a more recent python version makes sense though, as versions get sunset, so I'll put some more work into this definitely. Getting back to trying to reproduce the issue, some further questions on your setup: • have you ruled out possible conflicts in the installed poetry dependencies from the built Docker image? I notice that the venv is added to the path, so there might be some unintended rewiring that happens here • you mentioned the flow ran fine locally (presumably

--with batch

, and using the same custom image). Have you tried deploying it as a step-function? Wondering if the issue is limited to sagemaker or not