Hello I get this error whenever I want to load any artifact Outerbounds #ask-metaflow

Hello, I get this error whenever I want to load a...

bulky-artist-49977

08/15/2023, 8:35 PM

Hello, I get this error whenever I want to load any artifact from MetaFlow client.

Copy code

from metaflow import Task
Task('SAFlow/29/start/212', attempt=0)['x'].data

TypeError: expected str, bytes or os.PathLike object, not NoneType

I have used the metaflow service to deploy localy. https://github.com/Netflix/metaflow-service I think this problem may be similar to this issue. https://github.com/Netflix/metaflow/issues/950 Is there anyway to have a proper local storage and not use S3 to load dags and artifacts properly with locally deployed metaflow service?

✅ 1

ancient-application-36103

08/15/2023, 9:53 PM

@bulky-artist-49977 the UI expects that the artifact storage is a blob store. one work around would be to use cards instead of the UI to visualize your flows.

bulky-artist-49977

08/15/2023, 11:21 PM

Is there any way to use any other blob storage like minio with the UI service?

bulky-artist-49977

08/16/2023, 12:18 AM

I did some searching and it seems that it is possible to use metaflow with other datastores. https://github.com/Netflix/metaflow/issues/682 But I don't see a straightforward guide to whether how to setup minio as a datastore. I have the following docker compose file. Can you help me with which minio image and configuration or env variables for minio, metaflow service and metflow ui I should use? I’ve seen this https://github.com/outerbounds/metaflow-with-airflow-minio Isn’t there any simpler format using only minio docker image and setting env variables?

docker-compose-dev.yml

square-wire-39606

08/16/2023, 1:24 AM

here is a thread that goes into some of the details for configuring minio

16 Views

Open in Slack

Previous Next