Hello everyone! I was wondering how we could run a "clean" "cleanup" of the database (let's say we don't need anymore all the history of the flows). Would you have some documentation around it?

@ripe-car-38698 the database contains pointers to the actual artifacts that are stored in s3 (think of the db as an index on top of the data in s3). do you want to do the pruning to save on storage costs (better to clean s3) or to reduce the number of runs visible in the UI (drops rows older than a specific timestamp in all tables)?

Thank you @square-wire-39606 for the quick reply: i think we need both actually, the motive was more about the UI (your second case), but in our dev env, we might want just to flush everything indeed.

the data that is stored in s3 is highly compressed and deduped while being stored - to ensure that your storage costs are minimal. you can consider dropping the entire prefix

/metaflow/<flow-name>

from S3 if you want to delete all the data related to all the runs of any flow. To clean from the DB, you can drop rows older than a specific timestamp from all the tables.

than kyou!