Is anyone aware of any documentation about

METAFLOW_SERVICE_URL

vs

METAFLOW_SERVICE_INTERNAL_URL

?

The only docs I could spot for this are during the Metaflow profile configuration:

metaflow configure kubernetes

Metaflow can use a remote Metadata Service to track and persist flow execution metadata.
Would you like to configure the Metadata Service? [Y/n]:
[METAFLOW_SERVICE_URL] URL for Metaflow Service.: <http://service.url.example.com|service.url.example.com>
[METAFLOW_SERVICE_INTERNAL_URL] (optional) URL for Metaflow Service (Accessible only within VPC [AWS] or a Kubernetes cluster [if the service runs in one]). [<http://service.url.example.com|service.url.example.com>]:

my understanding of why there is such distinguishment is that when user runs a flow locally, it calls

METAFLOW_SERVICE_URL

, and it runs flows in k8s, it calls

METAFLOW_SERVICE_INTERNAL_URL

. Do you have insight into why there isn't similar a distinguishment for METAFLOW_S3_ENDPOINT_URL, given that when the user runs the flow locally or in k8s, it also needs to access diff urls, e.g.

METAFLOW_S3_ENDPOINT_INTERNAL_URL

(which doesn't exist) would be only accessible within the k8s cluster.

There is a good example of this in the local metaflow-dev setup file: https://github.com/Netflix/metaflow/blob/master/devtools/Tiltfile#L147 basically, if we need to provide a custom endpoint url for S3, then this would usually come with also a requirement for authentication. It is therefore easier to seed the execution environment with all the required AWS environment variables for S3 access than to try providing these through the client library.