HI,
My last email about DataOps in Azure was one of the most popular among all in my previous email campaigns. I was happy to see that it sparked conversations, but also showcased that many (we are over 260 subscribers!!!) are interested in DataOps.
I see on daily basis how many engineers and analysts struggle in getting their heads around deployment, testing, integrations. It's a very complex topic afterwards.
Hence, inspired by the growing popularity of Data Build Tool (DBT), "an open-source DataOps solution", I decided to experiment with the tool and share my findings with you.
The first time I heard about dbt in a meeting with Snowflake technical presales. Apparently, dbt was used often in combination of Big Query or Snowflake.
Does it support Azure Data Platform stack? Yes - with open-source connectors.
I've tested dbt against Azure Synapse, Azure SQL and Azure Databricks. Also, I managed to get Azure DevOps pipeline deployment.
Is it better that Azure Data Factory? In my opinion - yes.
Simply, I love the tool because it's simple though powerful, there is no UI, and it is very engineering friendly. Did I say how simple it is?
Check out my two articles about it and YouTube video.
|