Welcome to the Azure Databricks Intermediate course designed for data engineers, PySpark developers, ETL professionals, and working professionals who want to build strong real-world Azure Databricks skills beyond beginner concepts. This course focuses on intermediate-level Azure Databricks concepts with practical hands-on examples, performance optimization techniques, Delta Lake implementations, and production-oriented ETL development scenarios used in real-world data engineering projects.
You will start by understanding advanced Spark concepts such as partitions, shuffle operations, repartition vs coalesce, caching, persistence, execution plans, Catalyst Optimizer, Tungsten Optimization, and Spark join algorithms including broadcast joins, sort merge joins, and shuffle hash joins. The course also covers processing complex file formats such as JSON, XML, and Excel using PySpark. You will learn advanced transformation techniques including flattening nested JSON structures using explode and arrays_zip functions.
