Hello again, I am trying to run a flow on my kuber...
# ask-metaflowp
powerful-dress-75267
11/23/2022, 2:57 PMHello again, I am trying to run a flow on my kubernetes cluster. I have setup the config file using "metaflow configure kubernetes" but launching a flow with --with kubernetes raises the following error:
python flow.py run --with kubernetes
Metaflow 2.7.14 executing WineClassificationPOC for user:username
Validating your flow...
The graph looks good!
Running pylint...
Pylint is happy!
Kubernetes error:
The @kubernetes decorator requires --datastore=s3 or --datastore=azure at the moment.
✅ 1
a
ancient-application-36103
11/23/2022, 2:59 PM@powerful-dress-75267 You would need to also configure a blob store - like s3, azure blob store or minio for kubernetes
p
powerful-dress-75267
11/23/2022, 3:00 PMoh OK, thank you
powerful-dress-75267
11/23/2022, 3:24 PMHi again, apparently I have minio installed, how can I configure metaflow to use it ? it is said here https://outerbounds.com/engineering/operations/configure-metaflow/ that we must run configure aws before configure kubernetes (as I did). Should I run configure aws and give the info of my minio blob store instead of aws s3 info ?
u
user
11/23/2022, 4:41 PMHere's a quick way to test your flow with Kubernetes:
Copy code
METAFLOW_S3_ENDPOINT_URL=<http://my-minio-endpoint> METAFLOW_DATASTORE_SYSROOT_S3=<s3://mf-test-bucket> python3 a.py --datastore=s3 run --with kubernetesmf-test-bucket🙌 1
p
powerful-dress-75267
11/23/2022, 4:51 PMThank you that is what I was looking for ! I will try it 🙂
👍 1
72 Views