Hi,
Anyone aware of where exactly are the metaflow...
# ask-metaflowh
happy-journalist-26770
08/20/2024, 4:48 PMHi,
Anyone aware of where exactly are the metaflow artifacts like dataframes in the intermediate steps are stored if we dont explicitly upload them to s3?
I guess it pickles them and stores on s3 itself?
d
dry-beach-38304
08/20/2024, 4:54 PMExactly that — they are stored in S3 in a content-addressed manner
h
happy-journalist-26770
08/20/2024, 4:56 PMThanks for the clarification @dry-beach-38304!
i
incalculable-xylophone-18959
08/21/2024, 2:12 PMIf I could just tag onto this... could you confirm that it is acceptable to use a single metadata service for multiple MF clusters?
a
ancient-application-36103
08/21/2024, 2:17 PMtotally!
🙌 1
f
fast-vr-44972
08/21/2024, 3:51 PMDo they get deleted automatically?
👀 1
a
ancient-application-36103
08/21/2024, 3:52 PMNo, given that they are content addressed, the objects are shared by multiple runs
thankyou 1
ancient-application-36103
08/21/2024, 3:53 PMAlso, it allows for fetching the contents of your run at any time in the future using the metaflow client