We don't extensively use EFS with Batch currently (we do use it for other usecases), but I expect we would if support existed in Metaflow. Our current workaround for disk allocations in Metaflow batch CEs is to go the launch template route and attach EBS volumes to instances as they are provisioned.

In most cases users would not be able to access a resource like an EFS volume directly from their laptop, just given our typical security policy. We do have research users that work on remote servers inside our VPC, and syncing content from one of those servers to an EFS volume would probably be convenient. (indeed, these remote servers themselves have a shared EFS volume, I think!)

Do you foresee using EFS on Batch in a read-only mode only? I want to make sure we have consistency in experience when the compute moves from laptop to the cloud.

It probably would not be read-only.

Akin to other resource requirements, things like this are inherently hard to translate between local and remote -- I imagine the distinction would have to leak into user code, at least absent a local scheduler for mem/disk/cpu/etc.

In our feature/kfp branch we added

volume

to

@resources

https://github.com/zillow/metaflow/blob/feature/kfp/metaflow/plugins/resources_decorator.py#L65-L78