https://outerbounds.com/ logo
Join Slack
Powered by
Hi everyone, I have just started using metaflow. W...
# ask-metaflow
f
Hi everyone, I have just started using metaflow. When trying to run a flow on aws using step-functions, I get this error
Copy code
when deploying this flow to AWS Step Functions for the first time.
See "Organizing Results" at <https://docs.metaflow.org/> for more information about production tokens.

    Flow failed:
    AWS Step Functions requires --datastore=s3.
I tried searching the docs but didn't find any info about what does this mean? I see only some release notes where it's mentioned.
✅ 1
s
@fast-vr-44972 - does your metaflow config (in 
~/.metaflowconfig/config.json
) include 
METAFLOW_DATASTORE_SYSROOT_S3
and 
METAFLOW_DEFAULT_DATASTORE=s3
?
When deploying to AWS Step Functions, you need to specify an S3 bucket and point metaflow's datastore to s3. That can be achieved by either setting the above values in your config or through the CLI - 
python flow.py --datastore-root=path-to-s3-folder --datastore=s3 step-functions create
❤️ 1
f
No, I was following this tutorial https://docs.metaflow.org/production/scheduling-metaflow-flows/scheduling-with-aws-step-functions But just searched this slack and I see there are more docs from outerbounds itself https://outerbounds.com/engineering/operations/configure-metaflow/ Which docs do people normally follow?
s
both doc sites should be consistent. the metaflow docs have user-facing documentation and the outerbounds docs deal with infrastructure related concerns.
7 Views
Open in Slack
PreviousNext
Powered by