Hi, has there been any update on allocating memory resources dynamically since this?

no new features along those lines - the same guidance still applies

Hey Ville, the requirements for my steps would depend on the size of the dataframe I load at the start of the pipeline, which is variable. and the memory and cpu requirements for batch steps goes up or down based on the size of the dataframe.