Hi there, is there any chance that the

mf_env-creation.lock

can cause deadlocks? Am using Metaflow 2.11.9+netflix-ext(1.1.2) and when running a flow locally that needs two separate containers (built with the

@pypi

decorator), I have had a lot of trouble with file locks. Despite deleting these if they exist, I always get a

Timeout occurred while acquiring lock /home/xxx/miniforge3/envs/mf_env-creation.lock

after 60 minutes When debugging, it seems like there are two threads stuck at the same point, which suggests a deadlock to me. Is this a possibility or is it more likely that I am doing something wrong?

Hey. It’s definitely a possibility (it’s been tested a lot but still). What exactly are you trying to do and what are you seeing? I can try to help out.

Actually, I think this is probably a case of not removing all the lock files which we hadn't noticed after moving to the netflix extensions. I.e. the ones in all of these directories.

So does it work now?

it does

Cool!!! Was the error message not clear? I can improve it. I hate all the lock stuff I have to do but without it there are some weird things that happen at times.

There was an error saying where the file was, but I thought I copied and pasted that for deletion and it still didn't work, but it may be that there were multiple, so think that one is on me. One thing that could potentially be clearer is how to change the timeouts for the message to reduce cycle time. I think it's

METAFLOW_CONDA_LOCK_TIMEOUT

but don't think it's in the docs?

I’ll add that. Thanks for the feedback.