how can we write a test for a flow that uses @batc...
# ask-metaflow
b
how can we write a test for a flow that uses @batch decorator without setting s3 as the default METAFLOW_DEFAULT_DATASTORE in config_test.json?
1
v
you can do
python myflow.py --datastore=local run
or if it's easier,
METAFLOW_DATASTORE=local python myflow.py run
oh, I missed the
@batch
part. Batch requires S3 since it needs to move data around. For testing purposes you can setup a separate
METAFLOW_PROFILE
with a different bucket so test data stays separate
you can do
python myflow.py --metadata=local run
to keep metadata separate for testing