Hello folks, We are working on integrating with

KubeRay

and our first stab at it will be a tad hacky. Here's an example:

@kubernetes(memory=2000, cpu=1)
 @pypi(python="3.10.1", packages={"kubernetes" : ""})
 @step
 def setup_ray_cluster(self):
    import tempfile
    import json
    from kubernetes import client, config
    from kubernetes.client.rest import ApiException
        
    print("Read in KubeRay Cluster Resource Config ...")
    kuberay_config_file = tempfile.NamedTemporaryFile(mode="w")
    kuberay_config_file.write(self.kuberay_cluster_config)
    kuberay_config_file.seek(0)
    with open(kuberay_config_file.name) as f:
        kuberay_cluster_specs = json.load(f)
        
    config.load_kube_config()
    custom_api = client.CustomObjectsApi()

Running into an issue where the k8s pod cannot read the kubeconfig file (which I guess makes sense since the kubeconfig file is on a local machine), but I think we need to move it to the Metaflow K8s pod. Any insight on how to accomplish this?