The human-centric platform for production ML & AI

Outerbounds

Hi all, is it possible to use metaflow to run jobs on a different cloud provider? We don't want to lose our access to the past runs and also don't want to change service urls every time we need to run jobs on different providers.
For instance, keep the DB, service and any related stuff in AWS. But use GCP to run the job (it will access gcp buckets to read the data and store the results).

So that when the API is used, past Runs on both AWS and GCP can be seen.
If such setup is not possible, what would be the next best thing? Keeping the postgres in sync between GCP and AWS?

you could have one metadata service and then multiple configs targeting different clouds (but the same metadata service).

You can switch between the environments by changing `METAFLOW_PROFILE`

if you need a more seamless solution for multi-cloud compute, I'm happy to demo how we do it in Outerbounds (<https://outerbounds.com/blog/six-steps-to-cost-optimization/#6-where-should-the-workloads-execute|see a teaser screenshot here>)