Hi team, is it possible to get the batch job id of batch job launched by

@batch

decorator? My team is launching batch job in aws sagemaker studio's terminal

I believe it is available as an env var, so you can access like

os.getenv("AWS_BATCH_JOB_ID", None)

Thank you eddie! You are always so helpful🤗 I've got another question I'd love some insights on. My team is going to create a Cloudwatch dashboard to monitor the utilization of GPU launched by

@batch

. But we are now having a problem in linking the specific GPU to the user's batch job. The current plan is to get the

job_id

of the batch job, and then use the job id to get the instance ids of GPU launched by the job. I was wondering if metaflow has some functions that might help but I am missing? Any thoughts or ideas on this would be greatly appreciated.

I have used this 3rd party lib called ec2-metadata to fetch instance id in python before, but never tried with Metaflow (please share an example if you get it working and have the time!). The way to do something analogous without adding a dependency would be to follow this part of AWS EC2 docs. If you have more than one GPU on the instance, you might also want to use

nvidia-smi

to track the utilization of specific processor like how @gpu_profile does.

Thank you! These will definitely help, let me take a look into it

@fast-dog-88545 also this - https://outerbounds-community.slack.com/archives/C02116BBNTU/p1690488013956959?thread_ts=1677174406.722629&cid=C02116BBNTU