Hey folks, is it a feasible approach to integrate ...
# ask-metaflowa
acoustic-battery-52116
12/09/2023, 3:12 AMHey folks, is it a feasible approach to integrate Metaflow workflows within the AWS Databricks environment?
s
straight-shampoo-11124
12/09/2023, 3:16 AMyep, there are a few ways to do it, depending what you want to achieve
straight-shampoo-11124
12/09/2023, 3:18 AMyou can take a look at this @pyspark decorator for one approach. Another approach is to use their usual Python libraries to submit SQL queries like with any other query engine, like Athena in this example
🙌 1
a
acoustic-battery-52116
12/09/2023, 3:22 AMGreat, @victorious-lawyer-58417, thanks! I’ll explore the material and eventually come-back here. I really want to make my DS team happy by using the super smart Metaflow abstracions :)
s
straight-shampoo-11124
12/09/2023, 3:23 AMgreat! We are happy to help you get it working 🤗
❤️ 1