the blog post about Metaflow @ CNN is currently to...
# dev-metaflown
narrow-lion-2703
05/11/2021, 11:32 PMthe blog post about Metaflow @ CNN is currently top of the second page at HN woohoo https://news.ycombinator.com/item?id=27124644
🎉 6
🙏 3
narrow-lion-2703
05/11/2021, 11:34 PMDo other folks find Terraform more appealing than Cloudformation in general?when Ville left this comment i told him "ha, no one will be in favor in CF" and look at that.. the first comment is in favor of CF
narrow-lion-2703
05/11/2021, 11:35 PMactually no wait apparently I can't read, it is actually in favor of Terraform
narrow-lion-2703
05/11/2021, 11:35 PMmy world hasn't been shattered
🤣 3
narrow-lion-2703
05/11/2021, 11:36 PMoh and the author of the comment is @eager-jelly-89580, didn't see that either :)
w
worried-mechanic-36312
05/11/2021, 11:44 PM@narrow-lion-2703 do you understand what databricks is doing in there? 😉
worried-mechanic-36312
05/11/2021, 11:47 PMoh I can ask the author! @eager-jelly-89580 are you still using databricks ? I'm curious to know how you run it with Snowflake!
e
eager-jelly-89580
05/12/2021, 12:15 AMyeah.. @narrow-lion-2703 that was my comment 😄 would be nice to hear from others
eager-jelly-89580
05/12/2021, 12:16 AM@worried-mechanic-36312 yes, we are still using databricks - can you let me know what specifically you are interested in with regards to connecting to snowflake?
w
worried-mechanic-36312
05/12/2021, 12:21 AMyeah, story on my side.... since moving to SF (and dbt blah blah) we kind of put to rest athena, Spark etc., since all distributed computing is offloaded to SF now, which is awesome as all the rest of the pipeline is containerized and runs locally and/or on MF (+horovord sometimes) (we live at terabyte scale, not google / uber etc scale). So im curious to talk to people that moved to SF but then still having some other layer before the actual training!
e
eager-jelly-89580
05/12/2021, 12:47 AMThanks, that context is very helpful... So we have basically moved in the same direction and within my group we have mostly abandoned databricks. We were using SF in conjunction with Databricks but it was mostly just pulling data from SF, processing it and then doing some training. At the moment, we are doing our processing in MF but we are beginning to use dbt for some data enrichment, cleaning, etc. That said, there are still folks in the greater organization that are using DB + SF, but I suspect they will go in the same direction.
w
worried-mechanic-36312
05/12/2021, 12:48 AMOh great, thx for confirming we are not crazy 😜
e
eager-jelly-89580
05/12/2021, 12:49 AMyep, life got a lot simpler without it
w
worried-mechanic-36312
05/12/2021, 12:50 AMin case you're curious , we shared a bit of our setup here
<https://academy.qusandbox.com/#/market/60994482611019305776aaf8>e
eager-jelly-89580
05/12/2021, 12:53 AMgreat - I'll check it out - was this from today?
w
worried-mechanic-36312
05/12/2021, 12:56 AMyes, it was recorded today, building on some previous OS repos / posts.... I have bunch of notes here and there for a written version of
serverless dataOps😁 1
c
calm-magician-1871
05/13/2021, 11:58 AMI was super excited to see this hacker news post! Congrats everyone!!! 🎉 🎉 🎉
🎉 3
e
eager-jelly-89580
05/17/2021, 1:22 PM@worried-mechanic-36312 I got a chance to watch your presentation over the weekend and I found it to be quite good and relevant to our work. Do you mind if I DM you some follow-up questions?
w
worried-mechanic-36312
05/17/2021, 1:24 PManytime @eager-jelly-89580 - always happy to chat
🙏 1