Hi I am curious if anyone has done the work already to convert this to AWS CDK ? github.com/outerbounds/terraform-aws-metaflow

Hi! I’m preparing to contribute to Metaflow and have read the contribution guidelines. I’m looking for a small, well-scoped bug suitable for a first contribution. The current good first issue queue appears empty, and the available help wanted issues seem large or already being discussed by other contributors. Could a maintainer suggest an open, unassigned issue I may investigate? I’ll share a reproduction and proposed approach before implementing anything.

Hello! 👋 We are standardizing our forecasting project layout to meet enterprise standards. We want to separate our orchestration flows from our core business/feature engineering logic. We know Metaflow's remote packaging mechanism natively targets files relative to the flow script's location. If train_flow.py is nested in flows/, running argo create leaves the src/ directory behind, causing a ModuleNotFoundError on the cluster. My Questions: What is the currently recommended pattern to handle this layout? Is creating a symbolic link still the standard workaround for this, or is keeping the flow files strictly at the project root folder considered the cleanest path of least resistance? Are there newer alternative approaches (like specific flags or native configurations) to cleanly include sibling/parent directories during cluster packaging?

Is it possible to launch Metaflow steps with fractional GPUs? We have Multi-Instance GPU (MIG) set up. A k8s equivalent to:

resources:
  limits:
    nvidia.com/mig-1g.24gb: "1"
  requests:
    nvidia.com/mig-1g.24gb: "1"

Hello folks. I'm aware that Netflix has internally developed the capability to create live endpoints from within Metaflow that serve metaflow artifacts. As I understand it, the necessary service infrastructure for that is managed by Metaflow. This also seems to have been on one version of the GSoC ideas list. I see that there are relatively new additions in the metaflow-netflix-extensions (and this open PR github.com/Netflix/metaflow-nflx-extensions/pull/86) that implement many of the mentioned steps. Are there any plans to make the full FastAPI backend infrastructure also available?

Hi everyone, I just came across the Google Summer of Code announcement for Metaflow and I'd love to contribute. I'm interested in joining the project and was wondering how I should get started. Is there an onboarding process, a list of beginner issues, or any home assessment/task that contributors are expected to complete before getting onboarded? I'd appreciate any guidance on the best way to start contributing.

Hi, I'm currently a senior product manager at mlops platform. Is there anyway for me to contribute to metaflow?

Hi, is there a minimum argo workflows version required for recursion to work correctly? Trying the example in docs on

2.19.9

and it never reaches the

end

step

Hello all, I'm JJ and I'm an ML Platform Engineer. We have deployed multiple instances of metaflow in our stakeholders' accounts using terraform and now I am working on a centralized offering on k8s. I am trying to figure out auth and multi-tenancy and was hoping to get some guidance from experts in here. I am using the current helm charts and wondering what is the recommended helm approach to implement multi-tenancy? are there future plans for OSS multi-tenancy to be supported? Would highly appreciate your insights on this :)

Hi, I’m trying to spin up github.com/outerbounds/terraform-aws-metaflow but I’m getting two errors, one in the backend container in ECS and one in the service container. In the backend, I’m getting

psycopg2.errors.UndefinedTable: relation "public.flows_v3" does not exist

and in the service I’m getting

Failed to get env version

. These are using the latest version of the Netflix image netflixoss/metaflow_metadata_service. Are these errors familiar to anyone?

What is the canonical way to access outerbounds integration secrets from an outerbounds work station? I realize from a flow we can use the

@secrets

decorator and that I could use the outerbounds cli, but this is a little cumbersome to pull into a python script. Is there a slick way to do this with the python SDK that I am missing?

Hi, I'm new and exploring . I'm interested in Metaflow and ML systems. Can someone guide me or Direct me where i should start with then a follow up ?

aah, super useful 👌 i made some minimal tweaks and added a dev container (i'm on a windows machine) to run flows against the metadata service and see them appear in the UI, with artifacts appearing in the minio that's bind-mounted on local:

# <https://docs.docker.com/reference/compose-file/>
---
name: metaflow-development
services:
  metaflow-metadata:
    image: docker.io/netflixoss/metaflow_metadata_service:v2.5.0
    container_name: metaflow-metadata
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      MF_METADATA_DB_HOST: postgres
      MF_METADATA_DB_PORT: 5432
      MF_METADATA_DB_USER: metaflow_user
      MF_METADATA_DB_PSWD: metaflow_passw0rd
      MF_METADATA_DB_NAME: metaflow_db
      MF_METADATA_PORT: 8080
      MF_METADATA_HOST: 0.0.0.0
    ports:
      - '127.0.0.1:8080:8080'
    healthcheck:
      test: ["CMD", "curl", "-f", "<http://localhost:8080/ping>"]
      interval: 10s
      timeout: 5s
      retries: 5

  metaflow-ui:
    image: docker.io/netflixoss/metaflow_metadata_service:v2.5.0
    container_name: metaflow-ui
    depends_on:
      metaflow-metadata:
        condition: service_healthy
      minio:
        condition: service_healthy
      postgres:
        condition: service_healthy
    environment:
      MF_METADATA_DB_HOST: postgres
      MF_METADATA_DB_PORT: 5432
      MF_METADATA_DB_USER: metaflow_user
      MF_METADATA_DB_PSWD: metaflow_passw0rd
      MF_METADATA_DB_NAME: metaflow_db
      MF_UI_METADATA_PORT: 8083
      MF_UI_METADATA_HOST: 0.0.0.0
      UI_ENABLED: 1
      AWS_ACCESS_KEY_ID: minio_user
      AWS_SECRET_ACCESS_KEY: minio_passw0rd
      FEATURE_ARTIFACT_SEARCH: 1
      FEATURE_ARTIFACT_TABLE: 1
      METAFLOW_DEFAULT_DATASTORE: s3
      METAFLOW_DATASTORE_SYSROOT_S3: <s3://metaflow/>
      METAFLOW_S3_ENDPOINT_URL: <http://minio:9000/>
    ports:
      - '127.0.0.1:8083:8083'
    command: ["/opt/latest/bin/python3", "-m", "services.ui_backend_service.ui_server"]

  minio:
    image: docker.io/minio/minio:RELEASE.2025-09-07T16-13-09Z
    container_name: metaflow-minio
    volumes:
      - type: bind
        source: minio-data
        target: /data
        read_only: false
    environment:
      MINIO_ROOT_USER: minio_user
      MINIO_ROOT_PASSWORD: minio_passw0rd
    ports:
      - '127.0.0.1:9000:9000'
      - '127.0.0.1:9001:9001'
    command: server /data --console-address ":9001"
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 10s
      timeout: 5s
      retries: 5

  minio-initialize:
    image: docker.io/minio/mc:RELEASE.2025-08-13T08-35-41Z
    container_name: metaflow-minio-initialize
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      mc alias set myminio <http://minio:9000> minio_user minio_passw0rd;
      mc mb myminio/metaflow || true;
      mc anonymous set download myminio/metaflow;
      exit 0;
      "
  postgres:
    image: docker.io/library/postgres:18.1-trixie
    container_name: metaflow-postgres
    volumes:
      - type: bind
        source: postgres-data
        target: /var/lib/postgresql
        read_only: false
    environment:
      POSTGRES_USER: metaflow_user
      POSTGRES_PASSWORD: metaflow_passw0rd
      POSTGRES_DB: metaflow_db
    ports:
      - '127.0.0.1:5432:5432'
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U metaflow_user -d metaflow_db"]
      interval: 10s
      timeout: 5s
      retries: 5

  runner:
    image: python:3.14.5-slim
    container_name: metaflow-runner
    working_dir: /flow
    volumes:
      - ./flows:/flow
    environment:
      METAFLOW_DEFAULT_METADATA: service
      METAFLOW_SERVICE: <http://metaflow-metadata:8080>
      METAFLOW_SERVICE_URL: <http://metaflow-metadata:8080>
      METAFLOW_DEFAULT_DATASTORE: s3
      METAFLOW_DATASTORE_SYSROOT_S3: <s3://metaflow/>
      METAFLOW_S3_ENDPOINT_URL: <http://minio:9000>
      AWS_ACCESS_KEY_ID: minio_user
      AWS_SECRET_ACCESS_KEY: minio_passw0rd
    command: tail -f /dev/null

i've put the example

my_flow.py

in the ./flows dir, so i can invoke it from inside the

runner

service container:

(quickstart) C:\Users\scher\Repositories\quickstart>docker ps
CONTAINER ID   IMAGE                                         COMMAND                  CREATED          STATUS                    PORTS                                NAMES
d04af86107a1   netflixoss/metaflow_metadata_service:v2.5.0   "/opt/latest/bin/pyt…"   55 seconds ago   Up 32 seconds             127.0.0.1:8083->8083/tcp             metaflow-ui
a25571c665df   netflixoss/metaflow_metadata_service:v2.5.0   "/bin/sh -c 'python3…"   55 seconds ago   Up 43 seconds (healthy)   127.0.0.1:8080->8080/tcp             metaflow-metadata
b7e8a5c52219   python:3.14.5-slim                            "tail -f /dev/null"      55 seconds ago   Up 54 seconds                                                  metaflow-runner
8b2789cd15f8   minio/minio:RELEASE.2025-09-07T16-13-09Z      "/usr/bin/docker-ent…"   55 seconds ago   Up 54 seconds (healthy)   127.0.0.1:9000-9001->9000-9001/tcp   metaflow-minio
1bb10594a2f5   postgres:18.1-trixie                          "docker-entrypoint.s…"   55 seconds ago   Up 54 seconds (healthy)   127.0.0.1:5432->5432/tcp             metaflow-postgres

(quickstart) C:\Users\scher\Repositories\quickstart>docker exec -it b7e8a5c52219 bash
root@b7e8a5c52219:/flow# pip install metaflow
Collecting metaflow
  Downloading metaflow-2.19.32-py2.py3-none-any.whl.metadata (7.1 kB)
Collecting requests (from metaflow)
  Downloading requests-2.34.2-py3-none-any.whl.metadata (4.8 kB)
Collecting boto3 (from metaflow)
  Downloading boto3-1.43.25-py3-none-any.whl.metadata (6.6 kB)
Collecting botocore<1.44.0,>=1.43.25 (from boto3->metaflow)
  Downloading botocore-1.43.25-py3-none-any.whl.metadata (5.6 kB)
Collecting jmespath<2.0.0,>=0.7.1 (from boto3->metaflow)
  Downloading jmespath-1.1.0-py3-none-any.whl.metadata (7.6 kB)
Collecting s3transfer<0.19.0,>=0.18.0 (from boto3->metaflow)
  Downloading s3transfer-0.18.0-py3-none-any.whl.metadata (1.7 kB)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.44.0,>=1.43.25->boto3->metaflow)
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.44.0,>=1.43.25->boto3->metaflow)
  Downloading urllib3-2.7.0-py3-none-any.whl.metadata (6.9 kB)
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.44.0,>=1.43.25->boto3->metaflow)
  Downloading six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
Collecting charset_normalizer<4,>=2 (from requests->metaflow)
  Downloading charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (40 kB)
Collecting idna<4,>=2.5 (from requests->metaflow)
  Downloading idna-3.18-py3-none-any.whl.metadata (6.1 kB)
Collecting certifi>=2023.5.7 (from requests->metaflow)
  Downloading certifi-2026.5.20-py3-none-any.whl.metadata (2.5 kB)
Downloading metaflow-2.19.32-py2.py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 25.8 MB/s  0:00:00
Downloading boto3-1.43.25-py3-none-any.whl (140 kB)
Downloading botocore-1.43.25-py3-none-any.whl (15.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.2/15.2 MB 46.7 MB/s  0:00:00
Downloading jmespath-1.1.0-py3-none-any.whl (20 kB)
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Downloading s3transfer-0.18.0-py3-none-any.whl (88 kB)
Downloading urllib3-2.7.0-py3-none-any.whl (131 kB)
Downloading six-1.17.0-py2.py3-none-any.whl (11 kB)
Downloading requests-2.34.2-py3-none-any.whl (73 kB)
Downloading charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (215 kB)
Downloading idna-3.18-py3-none-any.whl (65 kB)
Downloading certifi-2026.5.20-py3-none-any.whl (134 kB)
Installing collected packages: urllib3, six, jmespath, idna, charset_normalizer, certifi, requests, python-dateutil, botocore, s3transfer, boto3, metaflow
Successfully installed boto3-1.43.25 botocore-1.43.25 certifi-2026.5.20 charset_normalizer-3.4.7 idna-3.18 jmespath-1.1.0 metaflow-2.19.32 python-dateutil-2.9.0.post0 requests-2.34.2 s3transfer-0.18.0 six-1.17.0 urllib3-2.7.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: <https://pip.pypa.io/warnings/venv>. Use the --root-user-action option if you know what you are doing and want to suppress this warning.

[notice] A new release of pip is available: 26.1.1 -> 26.1.2
[notice] To update, run: pip install --upgrade pip
root@b7e8a5c52219:/flow# export USERNAME=test
root@b7e8a5c52219:/flow# python my_flow.py run
Metaflow 2.19.32 executing TestFlow for user:test
Validating your flow...
    The graph looks good!
Running pylint...
    Pylint not found, so extra checks are disabled.
2026-06-08 23:29:12.659 Workflow starting (run-id 1):
2026-06-08 23:29:13.106 [1/start/2 (pid 42)] Task is starting.
2026-06-08 23:29:13.631 [1/start/2 (pid 42)] hello from metaflow
2026-06-08 23:29:14.922 [1/start/2 (pid 42)] Task finished successfully.
2026-06-08 23:29:15.131 [1/end/3 (pid 79)] Task is starting.
2026-06-08 23:29:15.643 [1/end/3 (pid 79)] done
2026-06-08 23:29:16.877 [1/end/3 (pid 79)] Task finished successfully.
2026-06-08 23:29:16.981 Done!

see screenshot 👇 for the UI showing the flow/run. you can also check the artifact download from minio is working:

root@b7e8a5c52219:/flow# python
Python 3.14.5 (main, May 19 2026, 23:46:18) [GCC 14.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from metaflow import Task
>>> Task('TestFlow/1/start/2', attempt=0)['message'].data
'hello artifact world'
>>>

the

myflow.py

in question:

from metaflow import FlowSpec, step

class TestFlow(FlowSpec):

    @step
    def start(self):
        print("hello from metaflow")

        # 👇 This becomes an artifact
        self.message = "hello artifact world"

        self.next(self.end)

    @step
    def end(self):
        print("done")

if __name__ == "__main__":
    TestFlow()

@brief-father-83029 has left the channel

I'm trying to understand the

argo-workflows --enable-heartbeat-daemon

feature. 1. What does the heartbeat daemon do? Does it monitor the flow for status, being more reliable and robust to K8s errors that may arise during pod initialization of a task pod? Or does is it more outward focused, reporting task status based on the argo template configuration of heartbeat task and step task operating at the same time? 2. When is it a good idea to use the heartbeat daemon? 3. When is it not a good idea to use it?

Hey! can someone explain me what metaflow is designed for like what actually it does. I've tried reading up on it, but I'm still a bit confused. I'd really appreciate a simple breakdown!

Hi @rhythmic-controller-77489 @hundreds-rainbow-67050, I wanted to understand what happens to projects that are not accepted in GSoC 2026 and what the plan of action would be going forward. I had applied for the VS Code extension project and was wondering if there are still opportunities for me to contribute to it or continue working on it.

This message was deleted.

Hey guys! Hope you are all well, I have a security concern. I reported a vulnerability in

metaflow-service

to Huntr back on March 24, and just got told by their admin that they tried reaching the maintainers but didn't get through, so they suggested I follow up here directly. It's a pre-auth issue, CVSS 10.0 with a chain I confirmed end-to-end today during re-validation. Full details, PoC, and code references are in the Huntr report: https://huntr.com/bounties/0a0fceee-b352-4295-894a-2c48003c6f6c Happy to share more privately. Who's the right person to loop in?

@wide-waiter-31986 has left the channel

Hi folks! I’ve noticed a credential leak issue in

metaflow-nflx-extensions

when resolving mixed conda/pypi environments against a private index using embedded credentials. Unlike pure conda or pypi environments (which strip credentials properly), mixed environments preserve auth tokens from the private index URL and leak them into the datastore URI. Example: • Index URL:

<https://user:token@private-pypi.example.com/simple/>

• Datastore URI:

<gs://some_bucket/backend/conda_env/packages/pypi/user:token@private-pypi.example.com/packages/some_private_package>

I think I've tracked this back to the

conda-lock

invocation here. Passing the

--strip-auth

flag might resolve it, assuming that auth isn't explicitly needed downstream. If it is, stripping the credentials during

make_partial_cache_url

would be a great alternative. While I can personally work around this using a different auth pattern like

.netrc

, I’m building tooling for a broader team and need to ensure their credentials stay out of plaintext regardless of their auth method. Let me know if you need any more details or if I've missed anything!

Hi, I’m new and exploring contributions. I’m interested in Metaflow and ML systems. Can someone suggest an issue to start with?

👋 Hello, team!

Hello folks. I've been figuring out how to set up Metaflow for AWS using the "Deploying to AWS Managed Services"::"Cloud Formation" approach. As has been pointed out, that method is not used much, but I decided to give it a go. And I did get things running. But, I happened to check my AWS Billing page, and in the just over a month of successfully demo'ing on AWS, I've racked up a $1200 bill. OOPS! Apparently what happens is that, in the course of setting up in this way, or maybe running code, a whole bunch of stuff is left running (forever if one doesn't catch it.) In my case I had running instances from ECC, SageMaker Notebooks, Relational Database Services. Those where the big charges, but I had a lot more smaller charges from various tools. I'm presuming that such an arrangement, where once services are brought up they stay running, is not what anyone would want. So I have two questions: 1. Is there a "shutdown" command that I'm not using but should? 2. Should I switch gears and us the method "Deploying to AWS with Kubernetes"? 3. Or any other advice or comments you might make? Thanks!!

Hello! We are currently trying to deploy Metaflow in the AWS + Kubernetes Cloud Configuration found here: https://docs.outerbounds.com/engineering/deployment/aws-k8s/deployment/ . We are able to deploy all but a few of the resources through the IaC package that is provided. However, we are running into an issue with the following during creation: kubernetes_namespace_v1.argo, helm_release.cluster_autoscaler Both are returning: Error: Kubernetes cluster unreachable: Get https:// i/o timeout(s) Any help would be greatly appreciated. Thanks!

Hey! As we grow metaflow adoption internally at my org one common area of feedback is that it is unclear about what artifacts are passed between steps and what the types of those artifacts may be. For example to check an artifacts existence in a later step typically I need a

getattr(self, "attribute", None)

which feels not great because the type checker cannot interpret this and infer the type. Also at the end step it would be great if I could output an overall output of the flow using

return

. Is this feedback known already? Is there already a way to do this using modern python typecheckers? I am big fan of typed outputs and inputs and have taken inspiration from other python graph libraries like https://pydantic.dev/docs/ai/graph/graph/#graph Love the tool. Let me know if this is the wrong place to give this feedback. I can also make a PR/Issue with some ideas I have if that is helpful

Hey is metaflow listing projects for lfx 2026 term 2

Hi, is there a way to install custom Python packages from a private artifact registry when deploying flows to Argo? I’d prefer to avoid using Docker images if possible

did you try testing your approach, also would be helpful if you could attach a link to what you are specifically doubtful about

Hi everyone, I’ve been working on the Metaflow codebase and trying to fix some issues. While exploring the FlowSpec and CLI parts, I noticed that the parameter handling logic is a bit confusing — especially around how parameters are collected and returned internally (like in get_parameters / related methods). I tried understanding and modifying it locally, but I’m not fully sure if my approach is correct or if I might be missing some edge cases. Could someone please guide me on how this part is expected to work or what would be the correct way to approach such optimizations? Thanks a lot!