Azure Synapse and Databricks Implementation
Business Problem
Global Procurement is annually a multibillion-dollar transaction sector. The team wanted a modern data and analytics platform that supports their digital transformation journey.
To review and analyze prodcure ERP data and domains across a variety of dimensions.
Map data from ERP system to one taxonomy. Ability to drill down from summary scorecards into transaction level details.
Need a solution where global users can access 24/7 and it has to be scalable for future capabilities.
Develop a solution in Azure Synapse that handles concurrent customer requests and provide data with reduced time lag (less than 12 hours)
Solution
A scalable and 24/7 available solution with automated data pipelines to feed the front-end apps (Power BI dashboards).
Azure Synapse used as the major consumption layer for Dashboards and Reporting.
Azure Data Lake (Data in Delta format in ADLS Gen2) as consumption layer for Data Science (ML/AI).
Azure Data Factory (ADF) is the primary orchestration tool & Databricks is the key transformation tool
Outcome
Data is aggregated and fed into Power BI for spend and supplier performance dashboards for global procurement sector with adequate drill down capabilities.
Data is processed, cleansed and enriched in the data lake and ready to consume in Synapse.
Handled large amounts of global incoming data from many disparate formats and systems.
Silver layer data ready to be consumed for ML/AI use cases.
