Hi <@U030E1YLYGK> Sir and Everyone ..... I'm inter...
# ask-metaflow
w
Hi @wooden-dusk-90720 Sir and Everyone ..... I'm interested in contributing to Metaflow. I've thoroughly read the Contributing Guide, the Core Runtime requirements, the AI usage policy, and the GSoC 2026 contributor guidance. I've identified a significant performance issue in the Client API - an N+1 HTTP query problem in the metadata provider layer that causes cascading request storms for common operations like listing flows and finding the latest successful run. I've traced the root cause through core.py, metadata.py, and service.py with exact line references. I'm about to open a detailed issue with a proposed layered fix and would like to follow up with a PR. Looking forward to your feedback...... Issue Link: https://github.com/Netflix/metaflow/issues/2942
h
Hi, we're aware of the scalability issues regarding the metadata service / client api, and we already have two gsoc ideas that are related to that theme. You can include your improvements as part of your proposal to those ideas
w
Thanks @hundreds-rainbow-67050 Sir That's really helpful. I see the "Metadata service request improvements" and "Agent-Friendly Metaflow Client" ideas - both are closely related to what I identified here. I'm very interested in working on Metadata service request improvements. I'll study the existing codebase and metadata service more carefully and put together a proposal. Should I share the draft approach here or in a new thread on Slack?
h
this thread is fine
w
Hello @hundreds-rainbow-67050 Sir Here's my planned approach for Metadata service request improvements: Service-side (metaflow-service): Add
_limit
,
_offset
,
_order
query params to all collection endpoints (e.g., /flows/{id}/runs?_limit=50&_offset=0) 1. Add
_tag
,
_system_tag
,
_any_tag
query params for server-side tag filtering via SQL WHERE clauses 2. When no pagination params are sent, behavior is identical to today (plain JSON array) - full backwards compatibility Client-side (metaflow): 4. Add
_supports_pagination
version check in ServiceMetadataProvider, following the same pattern as _supports_attempt_gets (service >= 2.0.6) and _supports_tag_mutation (service >= 2.3.0) 5. Update _get_object_internal to pass filters as query params when the service supports them, skip _apply_filter to avoid double-filtering 6. Add optional `limit`/`offset` to MetadataProvider.get_object so the Client API can paginate Compatibility matrix: New client + old service falls back to current behavior. Old client + new service gets full responses (no new params sent). No breaking changes. Happy to go into more detail on any of these. I'll write up the full proposal once we align on direction.
Hello @hundreds-rainbow-67050 Sir I'm also interested in the Metaflow VS Code Extension project (mentored by @rhythmic-controller-77489 Sir and you). I have experience with TypeScript and VS Code extensions. Would it be okay to submit proposals for both this and the Metadata service request improvements, or should I focus on one?
h
For the metadata service one you need to contact Sakari. Although I will just say off the top of my head, that proposal is exceptionally light and needs more thought (theres way more to be done than just that)
Regarding multiple proposals, I think you should focus on one rather than dividing your efforts -- one strong proposal is better than two half baked ones
w
Hello @rhythmic-controller-77489 Sir & @hundreds-rainbow-67050 Sir I’ve drafted my GSoC proposal for the Metaflow VS Code Extension project and wanted to thank you for designing such a practical and impactful project idea. Working on improving developer experience within IDEs strongly aligns with my interests and skills in frontend development and tooling. If you have some time, I would be grateful if you could review my proposal and share your feedback or suggestions for improvement. I would especially appreciate guidance on whether the project scope, milestones, and technical approach align well with your expectations. Thank you for your time and support.
l
@hundreds-rainbow-67050 , sir are you sure if we can submit our proposal/draft publically on slack ?
h
there should be an official link to do the submissions somewhere on the gsoc website
@wonderful-plastic-25010 you probably don't want to share your submission publicly in case you leak your alpha