Hi <@U030E1YLYGK> Sir and Everyone ..... I'm inter...
# dev-metaflow
w
Hi @wooden-dusk-90720 Sir and Everyone ..... I'm interested in contributing to Metaflow. I've thoroughly read the Contributing Guide, the Core Runtime requirements, the AI usage policy, and the GSoC 2026 contributor guidance. I've identified a significant performance issue in the Client API - an N+1 HTTP query problem in the metadata provider layer that causes cascading request storms for common operations like listing flows and finding the latest successful run. I've traced the root cause through core.py, metadata.py, and service.py with exact line references. I'm about to open a detailed issue with a proposed layered fix and would like to follow up with a PR. Looking forward to your feedback...... Issue Link: https://github.com/Netflix/metaflow/issues/2942
f
Just a drive-by comment, but is this an actual issue? I reckon if Netflix uses Metaflow, things like "_cascading request storms for common operations like listing flows and finding the latest successful run_" would have been noticed. Not sure how I feel about possibly AI-generated changes to a core interface to solve an unreported performance issue. 🤷 But I'm not a maintainer or anything.
👍 1
@wonderful-plastic-25010 feel free to correct me about the AI usage stuff. I'm just basing this on the volume of issues & PRs opened, e.g. 38 issues in 1 week for ControlCore listed in your contributions + the LLM-esque structure of the messages you write. Given that you've read the AI Tool Usage Policy, I'm sure you know that you need to disclose AI use in any contribution.
💯 1
w
@famous-airline-14628 - appreciate the honest pushback. You're right to question this. I'll be upfront: 1. AI disclosure: I used AI tools during my research. I should have stated that from the start per the contribution policy. My mistake. 2. What I got wrong: The _is_in_namespace N+1 pattern is intentional - there's a comment at line 2660 of core.py that explicitly explains the design choice. I missed that and overstated it as a bug. Same with _apply_filter - the filters dict is already passed into _get_object_internal, so server-side implementations can and likely do filter there. I was wrong to frame the client-side filter as the whole story. 3. What I still stand by: latest_successful_run iterates all runs and checks run.successful one by one, which chains through end_task → step → task → artifact lookup. With 10 consecutive failed runs, that's ~31 HTTP requests to find one successful run. This isn't a design choice with a code comment - it's just how the property was written. Whether it matters in practice at Netflix's scale, I can't say. I'll narrow this issue to just point 3, or close it entirely if the maintainers don't find it useful. Either way - lesson learned on doing better due diligence before opening issues.