Many sources? Various format? Unstructured data? Big Data? You might think that these only a buzz words. Not really. These days it’s a part of modern data flow architecture.
No matter what do you use – SQL Server, Cosmos DB, Azure SQL DW, Azure Data Factory, Data Lake… somewhere there you can find Databricks.
So, the question is: what the Azure Databricks is and which scenario it could be used in?
Use Databricks to analyse large DataSets at scale, write Python, Scala or SQL command in one notebook to ingest, process and push the data to the required target.
Use Databricks’ notebook as a part of Azure Data Factory pipeline. We also will try to answer whether Databricks would replace SSIS as a modern ETL/ELT process?
If you are wondering about all these things – you should join me in this session.