CI/CD Databricks Asset Bundles, Azure DevOps and GitHub Actions
April 09, 2025Tutorial on how to do CI/CD for Databricks Assets like Notebooks, Files and Jobs and Cluster set up
A Blog by Clint Grove who lives in Cambridge and works for Microsoft. Building useful data resources to help bring data to life. Find me on LinkedIn
Databricks Delta Lake writing and reading to Dynamics365 Dataverse
December 02, 2024Using Dynamics365 API calls to the Dataverse from Databricks
Python based Web App with API endpoints writing to SQL database based on Service Bus Messages
July 20, 2024In this tutorial I take a batch of service bus messages, use an web app api endpoint and trigger off a write process to a sql database using passwordless authentication.
Data Factory Deployments using AVM and how to maintain lifecycle of Infra and operational code
May 17, 2024If you are using the new Azure Verified Modules from aka.ms/avm and you are using bicep, you may have asked yourself should I build my pipelines, linked services, managed private endpoints all in the IAC or should I do it in the Data Factory then read this
How to create your first Delta Lakehouse incremental flow
April 22, 2024Doing "delta" work in Databricks and Synapse using Delta Lake technology is so simple, follow along with me
Incremental data load with Data Factory and Azure SQL Server
January 24, 2024Using a high watermark and lookup table to see the last modified date and do incremental load to target table
Power BI - Running total for Fiscal Year starting April using DatesYTD
June 12, 2023How to get separate lines in a line chart, one for each financial year
How to promote your Dev Azure Synapse Workspace to your next higher environment
April 30, 2023If you have ever wondered how you use git and DevOps to promote your Dev Synapse Workspace to the next Synapse in Test, then read on...
Using Certificates for Service Principals instead of Secrets
February 07, 2023If you have had security recommendations on rather using certificates instead of secrets for your app registration in AAD then follow this
How to create a Common Data Model using Mapping Data Flows and connect it to Customer Insights
January 13, 2023How to build a Common Data Model in a Lake Store and use it as a data source for Customer Insights Dynamics 365
Power BI - Getting the Totals to calculate properly
November 19, 2022How to get the correct totals in a Power BI report
ARM Templates - Enable Synapse SQL Pool Diagnostics
June 15, 2022Enabling Diagnostics on Synapse SQL Pools requires you to use an "Extension Resource". Find out how I did it.
Use Power BI Rest API
May 31, 2022Using the Power BI API is not straight forward. Here is my journey
ARM Templates - Enable Synapse SQL Pool Auditing
May 28, 2022Enabling Auditing on Synapse is different to enabling it on Azure SQL Server. Find out how I did it in this weeks blog post
ARM Templates - Enable Azure SQL Server Auditing
May 27, 2022I was recently tasked with updating an ARM template to include SQL Auditing to be enabled and for the logs to go to a Log Analytic workspace. Read how I did it.
Use Power BI with summarised data
May 26, 2022Using Power BI on very granular data is not always a good idea. The user experience can be slow and users can get frustrated with the tool. Read more about the best way to use Power BI
Databricks Delta Load, Data Factory Copy Activity from SQL to Databricks and Staging with Gen2 Lake Store
May 24, 2022Recently we tried to copy a SQL table to a Delta Table in Azure Data Factory. Immediately we were hit with the error 'You cannot copy directly from SQL to Delta'. We had to stage the data first in the Storage Account. But it wasn't so straight forward. Our Databricks resource was not able to fetch the data from the Storage Account without the proper permissions