Setting up an end-to-end data platform deployment and rolling out incremental changes later is not trivial. Hence, here are two articles you might find interesting.
Valdas MaksimaviÄius IT Architect & Microsoft Data Platform MVP | Data Engineering, Data Science, Cloud Modernization 2021-01-17
If you work on setting up Azure Data Platform and deployment, this project might save you weeks of work.
You will find not only complete infrastructure deployment, but also a complete pipeline example with parking sensors. Oh, did I mention unit and integration tests? It's there too š
Next tool that I highly encourage to explore, is Data Build Tool (DBT). Initially, I've stumbled upon DBT while testing Snowflake. And I was curious how it works with Windows, Azure SQL and Azure Synapse.
Valdas MaksimaviÄius IT Architect & Microsoft Data Platform MVP | Data Engineering, Data Science, Cloud Modernization 2021-01-24
Inspired by the growing popularity of Data Build Tool (DBT), I decided to set it up on my PC and explore the tool against Azure SQL and Azure Synapse.
DBT Cloud version with a nice UI. However, why not to try the VSCode and DBT CLI command line experience? :)
The blog post is a documentation of all the steps taken to get DBT CLI up and running, with error messages, so you can find a workaround faster.
