Hey Metaflow team, we're bumping into the "Container Overrides length must be at most 8192" error on AWS Step Functions from time to time. We're tracking a few extra things for each flow (like git info and also some custom config like our Conda channel) from deploy-time parameters, which makes the overrides a bit larger. I'm happy to share an example in a DM. However, I noticed that the JSON sent to Step Functions is prettified (maybe from here? https://github.com/Netflix/metaflow/blob/06a76c8a2a0e9f3ae12ab5e5bf9ee788beb6ae94/metaflow/plugins/aws/step_functions/step_functions.py#L85 or is it just shown this way in the AWS console?), minifying it would save around 1000 chars for us. Do you have a suggestion on what we could try to better fit the limit?

@billowy-exabyte-36648

pretty=True

is used to pretty print the json if you use the

--only-json

flag.

Thanks for the answer, I'll share via DM! https://github.com/Netflix/metaflow/blob/06a76c8a2a0e9f3ae12ab5e5bf9ee788beb6ae94/metaflow/plugins/aws/step_functions/step_functions.py#LL12[…]27C36 this still looks to me like the pretty JSON string is used in the SFN client, I didn't check if the boto3 client is minimizing it for transport though