Hey Metaflow team, we're bumping into the "Container Overrides length must be at most 8192" error on AWS Step Functions from time to time. We're tracking a few extra things for each flow (like git info and also some custom config like our Conda channel) from deploy-time parameters, which makes the overrides a bit larger. I'm happy to share an example in a DM.
However, I noticed that the JSON sent to Step Functions is prettified (maybe from here?
https://github.com/Netflix/metaflow/blob/06a76c8a2a0e9f3ae12ab5e5bf9ee788beb6ae94/metaflow/plugins/aws/step_functions/step_functions.py#L85 or is it just shown this way in the AWS console?), minifying it would save around 1000 chars for us.
Do you have a suggestion on what we could try to better fit the limit?