Hi, I'm using extensions and aws batch/step functions. Given that all the parameters are in ssm, it seems like a useful thing to do something like this in the config:

BATCH_JOB_QUEUE = from_aws("BATCH_JOB_QUEUE")

That's all good and fairly straightforward except that boto3 isn't loaded at that point It will work if I add boto3 to the conda_base libraries but that's not ideal (behaves slightly differently if local/batch vs step function/batch) I just wondered if anyone else has done something similar and/or has a better idea about installing boto3? Thanks

@rhythmic-beach-70913 boto3 gets automatically injected into the conda environment created per step but that conda environment is not used to trigger AWS Batch jobs. As long as you have boto3 available in the environment that is being used to launch the flow, things should just work.

It took me a while to work out, but I'll put it here in case it's helpful. In short, if you don't configure anything, and run with

--environment=conda run

then boto3 isn't loaded into the conda environment (and no batch decorator) That seems fair enough

@rhythmic-beach-70913 boto3 is automatically added in to the environment. Are you using the default conda decorator or the one that comes in through metaflow extensions?

The default There's no way to know boto3 is required in this scenario, since you are not telling the flow to use any AWS resources - no METAFLOW_ parameters set Once you tell it that you want to use AWS, then it's all fine I just needed to be a bit more careful in the way that I constructed the config thanks