Ch 1: SQL SERVER INTRODUCTION

• Data, Databases and RDBMS Software
• Database Types : OLTP, DWH, OLAP
• Microsoft SQL Server Advantages, Use
• Versions and Editions of SQL Server
• SQL : Purpose, Real-time Usage Options
• SQL Versus Microsoft T-SQL [MSSQL]
• Microsoft SQL Server - Career Options

• SQL Server Components and Usage
• Database Engine Component and OLTP
• BI Components, Data Science Components
• ETL, Data Engineer and BI Components
• Course Plan, Concepts, Resume, Project
• 24 x 7 Online Lab for Remote DB Access
• Software Installation Pre-Requisites

Ch 5: SQL Basics - 3, T-SQL INTRO

• Database Objects : Tables and Schemas
• Schemas : Group Tables in Database
• Schemas : Security Management Object
• Creating Schemas & Batch Concept
• Using Schemas for Table Creation
• Data Storage in Tables with Schemas
• Data Retreival and Usage with Schemas
• Table Migrations across Schemas

• Import and Export Wizard in SSMS
• Data Imports with Excel File Data
• Performing Bulk Operations in SSMS
• Temporary Tables : Real-time Use
• Local and Global Temporary Tables
• # and ## Prefix, Scope of Usage

Ch 9: JOINS, T-SQL QUERIES Level 3

• GetDate, Year, Month, Day Functions
• Date & Time Styles, Data Formatting
• DateAdd and DateDiff Functions
• Cast and, Convert Functions in Queries
• String Functions: SubString, Relicate
• Len, Upper, Lower, Left and Right
• LTrim, RTrim, CharIndex Functions

• MERGE Statement - Comparing Tables
• WHEN MATCHED and NOT MATCHED
• Incremental Load with MERGE Statement
• IIF() Function for Value Compares
• CASE Statement : WHEN, ELSE, END
• ROW_NUMBER() and RANK() Queries
• Dense Rank and Partition By Queries

Ch 2: SQL SERVER INSTALLATIONS

• System Configuration Checker Tool
• Versions and Editions of SQL Server
• SQL Server and SSMS Installation Plan
• SQL Server Pre-requisites : S/W, H/W
• SQL Server 2016 / 2017 Installation
• SQL Server 2019 Installation
• Instance Name and Server Features
• Instances : Types and Properties

• Default Instance, Named Instances
• Port Numbers, Instance Differences
• Service and Service Account Use
• Authentication Modes and Logins
• Windows Logins and SQL Logins
• FileStream and Collation Properties

Ch 6 : CONSTRAINTS,INDEXES Basics

• Constraints and Keys - Data Integrity
• NULL, NOT NULL Property on Tables
• UNIQUE KEY Constraints: Importance
• PRIMARY KEY Constraint: Importance
• FOREIGN KEY Constraint: Importance
• REFERENCES, CHECK and DEFAULT
• Candidate Keys and Identity Property

• Database Diagrams and ER Models
• Relationships Verification and Links
• Indexes : Basic Types and Creation
• Index Sorting and Search Advantages
• Clustered and NonClustered Indexes
• Primary Key and Unique Key Indexes
• Need for Indexes - working with Keys

Ch 10: VIEW, SPs, Function Basics

• Views : Types, Usage in Real-time
• System Predefined Views and Audits
• Listing Databases, Tables, Schemas
• Functions : Types, Usage in Real-time
• Scalar, Inline and Multi-Line Functions
• System Predefined Functions, Audits
• DBId, DBName, ObjectID, ObjectName
• Variables & Parameters in SQL Server

• Procedures : Types, Usage in Real-time
• User & System Predefined Procedures
• Parameters and Dynamic SQL Queries
• Sp_help, Sp_helpdb and sp_helptext
• sp_pkeys, sp_rename and sp_help
• Important System Objects and Metadata

Ch 3: SSMS Tool, SQL BASICS - 1

• SQL Server Management Studio
• Local and Remote Connections
• System Databases: Master and Model
• MSDB, TempDB, Resource Databases
• Creating Databases : Files [MDF, LDF]
• Creating Tables in User Interface
• Data Insertion & Storage. Limitations

• SQL : Purpose and Real-time Usage
• SQL Versus T-SQL : Basic Differences
• DDL, DML, SELECT, DCL and TCL
• Creating Tables using SQL Scripts
• Data Storage, Inserts - Basic Level
• Table Data Verifications with Select
• SELECT Statement for Table Retrieval

Ch 7: JOINS, T-SQL Queries : Level 1

• JOINS - Table Comparisons Queries
• INNER JOINS For Matching Data
• OUTER JOINS For (non) Match Data
• Left Outer Joins with Example Queries
• Right Outer Joins with Example Queries
• FULL Outer Joins - Realtime Scenarios
• Join Queries with "ON" Conditions

• Join Unrelated Tables in SQL Server
• NULL, IS NULL Operators in Joins
• CROSS JOIN and CROSS APPLY
• CROSS JOIN Versus CROSS APPLY
• One-way & Two Way Data Comparisons
• Important Join Queries in T-SQL
• Join Options: Merge, Loop, Hash

Ch 11: Triggers & Transactions

• Triggers - Purpose, Real-world Usage
• FOR/AFTER Triggers - Real time Use
• INSTEAD OF Triggers - Real time Use
• INSERTED, DELETED Memory Tables
• Using Triggers for Data Replication
• Enable Triggers and Disable Triggers
• Database Level, Server Level Triggers

• Transactions : Types, ACID Properties
• Transaction Types and AutoCommit
• EXPLICIT & IMPLICIT Transactions
• COMMIT and ROLLBACK Statements
• Open Transaction Scenarios & Cause
• Query Blocking Scenarios @ Real-time
• NOLOCK and READPAST Lock Hints

Ch 4: SQL BASICS - 2

• Creating Databases & Tables in SSMS
• Single Row Inserts, Multi Row Inserts
• Rules for Data Insertion Statements
• SELECT Statement @ Data Retrieval
• SELECT with WHERE Conditions
• Batch Concept and Go Statement
• AND and OR Operators Usage
• IN Operator and NOT IN Operator

• Between, Not Between Operators
• LIKE and NOT LIKE Operators
• UPDATE Statement & Conditions
• DELETE & TRUNCATE Statements
• Logged and Non-Logged Operations
• ADD, ALTER and DROP Columns
• ALTER & DROP Table Statements

Ch 8: Group By, T-SQL Queries Level 2

• GROUP BY Queries and Aggregations
• Group By Queries with Having Clause
• Group By Queries with Where Clause
• Using WHERE and HAVING in T-SQL
• Rollup : Usage and T-SQL Queries
• Cube : Usage and T-SQL Queries
• UNION and UNION ALL Operator
• EXISTS Operator, Query Conditions

• Sub Queries and Alternatives to Joins
• Using Joins with Group By Queries
• Using Joins with Nested Sub Queries
• Sub Queries with Joins and Group By
• Using UNION and UNION ALL in Queries
• Nested Sub Queries with Group By, Joins
• Comparing WHERE, HAVING Conditions

Ch 12 : ER MODELS, NORMAL FORMS

• Normal Forms for Entity Relationships
• First Normal Form and Atomocity
• Second Normal Form, Candidate Keys
• 3rd Normal Form Multi Value Dependancy
• Boycee-Codd Normal Form : BNCF
• Fourth Normal Form Realtime Advantages
• Self Joins & Self Reference Keys
• 1:1, 1:M, M:1, M:M Relationship Types

• Joins with Group By Queries
• Joins with Sub Queries, Formating
• Office Data Connections, Excel Reports
• Excel Pivot Reports and Reports
• SQL Queries (Auto Generated) in BI Tools
• FETCH OFFSET, NEXT ROWS, Order By
• Data Refresh (Manual and Automated)

Chapter 1: Cloud Basics, Azure SQL DB

• Cloud Introduction and Azure Basics
• Azure Implementation: IaaS, PaaS, SaaS
• Benefits of Azure Cloud Environment
• Azure Data Engineer: Job Roles
• Azure Storage Components
• Azure ETL & Streaming Components
• Need for Azure Data Factory (ADF)
• Need for Azure Synapse Analytics

• Azure Resources and Resource Types
• Resource Groups in Azure Portal
• Azure SQL Server [Logical Server]
• Firewall Rules and Azure Services
• Connections with SSMS & ADS Tools
• Working with Azure Portal
• Resource Group Navigations, Options
• DP 203 Certification Guidance

Chapter 1: Azure Storage & Containers

• Storage Components in Microsoft Azure
• Azure Storage Services and Types - Uses
• High Availability, Durability & Scalability
• Blob: Binary Large Object Storage
• General Purpose: Gen 1 & Gen 2 Versions
• Blobs, File Share, Queues and Tables
• Data Lake Gen 2 Operations with Azure
• Azure Storage Account Creation

• Azure Storage Container: Usage
• Azure Data Explorer: Operations
• File Uploads, Edits and Access URLs
• Azure Storage Explorer Tool Usage
• Azure Account Options in Explorer
• Directory Creation, File Operations
• End User Access Options With Files
• Data Explorer Vs Storage Explorer Tool

Chapter 1: Azure Intro, Azure Databricks

• Azure Databricks : Purpose & Config
• Need for Azure Databricks (ADB)
• Azure Databricks Service Creation
• Azure Databricks Workspace & Usage
• Spark Cluster Configurations & Capacity
• Driver Nodes and Worker Nodes in Spark
• Master Node & Cluster Creation Process
• Cluster Types and Capacity Options

• Standard, High Concurrency Clusters
• Databricks Runtime Service & DBUs
• Databricks File System (DBFS) and Usage
• Azure Databricks Workspace Operations
• ETL and Data Storage Components
• Spark Concepts and Spark SQL
• Spark Context and Spark Session
• DataFrame, Dataset and Real-time Use

Chapter 2: Synapse SQL Pools (DWH)

• Dedicated SQL Pools in Azure
• Enterprise Data Warehouse with Synapse
• DWU: Data Warehouse Units, Resources
• Massively Parallel Processing (MPP)
• Control Nodes and Compute Nodes
• SQL Pool Access from SSMS Tool
• T-SQL Queries @ SQL Pools
• Start/Resume/Pause, Scaling Options

• Creating Tables in Azure SQL Pool
• Compression, MAX DOP & Indexes
• Distributions: Round Robin, Hash
• Distributions: Replicate and Usage
• Data Imports with COPY Table
• Dynamic Views (DMV) with PDW
• Data Loads Monitoring, Resource Class

Chapter 2: Azure Migration, BLOB Imports

• SQL Server (On-Premise) to Azure Migration
• Source Database Scripts & Validations
• BACPAC File Generation From SSMS Tool
• Azure Data Lake Storage and SSMS Access
• Azure Storage Container, BACPAC Files
• Azure SQL Server Creation From Portal
• Azure SQL DB Imports, Storage SAS Keys
• Azure SQL Database Migrations, Verification

• BLOB Data Access from On-Premise
• Data Imports From Excel and CSV Files
• BLOB Data Imports using T-SQL Queries
• SAS - Shared Access Signature Generation
• CSV File - Uploads, Downloads, Edits, Keys
• Master Keys, Credentials, External Sources
• BULK INSERT Statement and Data Imports
• T-SQL Imports : Practical Limitations

Chapter 2: SQL Notebooks & Python

• Notebooks: Concept, Usage Options
• Creating SQL Notebooks in Databricks
• Using DBFS Tables in SQL Notebooks
• Data Access and Analytics Options
• SparkSQL Queries: SELECT, GROUP BY
• SparkSQL Queries: Aggregates, Conditions
• Notebook Operations: Download, Clone
• Notebook Operations: Upload, Reuse

• SQL Notebooks with Python Code
• Using DBFS Sample Data Sources (CSV)
• Dataframes: Creation and Real-time Use
• Pandas Dataframe, Virtual Table Creation
• Dataframe Data Access, Caching Options
• Take() and Display() Functions in PySpark
• Temporary View Creation and Access
• SparkSQL Queries, Analytics, Chart Reports

Chapter 3: Azure Data Factory Concepts

• Azure Data Factory (ADF) Concepts
• Hybrid Data Integration at Scale
• ADF Pipeline Components & Usage
• Configure ADF Resource in Azure
• Understanding ADF Portal and IR
• Linked Services and Connections
• Datasets and Tables / Files for ETL
• ADF Pipelines: Design, Publish & Trigger

• ADF Pipeline with Copy Data Tool
• Creating Azure Storage Account
• Storage Container, BLOB File Uploads
• Data Loads with Azure BLOB Files
• DIU Allocations and Concurrency
• Creating Linked Services, Datasets
• Pipeline Trigger, Author and Monitor

Chapter 3: Azure Tables, Shares

• Azure Tables - Real-time Usage
• Schema-less Design and Access Options
• Structured and Relational Data Storage
• Tables, Entities and Properties Concepts
• Azure Tables: Creation and Data Inserts
• Azure Tables in Portal - GUI and Data Types
• Azure Tables: Data Imports in Explorer
• Data Edits, Queries & Delete Operations

• Azure Files - SMB Protocol, Creation, Usage
• Shared Access, Fully Managed & Resiliency
• Performance, Size Requirements for Shares
• Azure Storage Explorer Tool for File Shares
• Azure Queues: Message Queues, Limitations
• Adding Messages, Queuing and De-Queuing
• Data Access & Clear Queue from Explorer
• End Points for Azure Message Queues

Chapter 3: Python Notebooks

• Azure SQL Server Configurations
• Azure SQL Database Creation
• Azure Firewall Rules and IP Address
• Allow Azure Services, Remote Access
• Connection Tests with SSMS Tool
• Python Notebooks with Azure Databricks
• Data Imports and Table Creations (Code)
• Parquet Files and Usage in Databricks

• Using Dataframes for Data Operations
• SparkSQL Queries with SELECT, TOP
• Establishing Connections to Azure SQL DB
• JDBC Connection Strings, DataframeWriter
• JDBC Properties, Port Settings & Options
• Data Extraction, SQLContext & Dataframes
• Pandas Data Frame for Big Data Analytics
• JDBC URL Options & PySparkSQL Modules

Chapter 4: ADF Pipelines, Polybase

• Copy Data Tool For ETL Operations
• Azure SQL DB to Synapse Data Loads
• Working with Multi Tables Data Loads
• Query Options for Source Datasets
• Transformations with Copy Data Tool
• Rename, Rearrange & Remove Options
• Pipeline Execution: DTU & DOCP
• ADF Pipeline Monitoring Options

• ADF Pipelines: Execution Settings
• ADF Logging Options, Consistency Check
• Compression Option, DOP and DOCP
• ETL Staging Advantages & Performance
• Staging with Storage Account, Container
• ADF Pipeline Triggers and Monitoring
• Polybase For Azure Synapse, Advantages

Chapter 4: Azure Storage Security, Admin

• Azure Data Lake Storage Security Options
• Shared Access Keys - Primary, Secondary Keys
• SAS Key Generation: Container, Tables, Files
• SAS Key Permissions, Validation Options
• Access Keys: Account Level Permissions
• Azure Active Directory (AAD): Users, Groups
• Azure AD Security: RBAC with IAM, ACLs
• Owner Role, Contributor and Reader Role

• Azure Data Lake Storage Security Options
• ACL : Access Control Lists & Security
• Azure BLOB Storage Containers & ACLs
• Folder Level and File Level Security
• ACL Permissions: Read, Write & Execute
• Access Policy: Creation and Realtime Use
• Permissions: rwacdl; Azure Principals, CORS
• Comparing IAM and ACLs in Data Lake Store

Chapter 4: Open Data Sources, DeltaLakes

• Creating Python Notebooks with Databricks
• Spark Dataframes with Azure OpenDatasets
• Windows Azure Storage Blob [wasb] Sources
• Creating Dataframes & Temporary Views
• Using Print and Display Functions with ADB
• Big Data Analysis with BLOB Data & Charts
• Keys, Values, Aggregations, Display Type
• Databricks Notebooks, Jobs and Stages

• Azure DeltaLake Implementation
• ACID Properties and Upsert Advantages
• Delta Engine Optimizations & Uses
• Pipeline Creation with JSON Files in DBFS
• Delta Tables Creation, Data Loads
• Spark Cluster Settings: Auto Optimize
• Auto Compact and Delta Table Optimize
• Delta Locations; Data Retrieval, Versions

Chapter 5: OnPremise Data with ADF

• On-Premise Data Sources with Azure
• Self Hosted Integration Runtime (IR)
• Access Keys, Remote Linked Services
• Synapse SQL Pool (DW) with OnPremise
• Staged Data Copy and Performance
• Pipeline Executions and Monitoring
• Pipeline RunIDs and Audits / Tracing
• Incompatible Rows Skips, Fault Tolerance

• Incremental Loads with Files (BLOB)
• Pipeline Executions and Schedules
• Regular Schedules and Tumbling Window
• Execution Retry and Delay Options
• Binary Copy, Last Modified Date in Blob
• Automated Loops and Trigger Schedules
• Incremental Loads Verification Tests

Chapter 5: Azure Monitoring, Power BI

• Azure Monitor, Metrics & Logs
• Monitoring Azure Storage Namespaces
• Add KQL Metrics; Account, Blob and File
• Total Ingress and Egress Metrics: Charts
• Average Latency, Transaction Count
• Request Breakdowns, Signal Logic Options
• Azure Alerts and Conditions, Notifications
• Signal Logic Conditions and Emails

• Power BI Desktop Tool Installation
• Binary Data and Record Data Access
• Azure Data Lake Storage: Access Keys
• Azure Data Lake Storage with Power BI
• BLOB File Access with Power BI
• Azure Tables Creation and File Imports
• Azure Table Access with Power BI

Chapter 5: Databricks Security & Jobs

• Azure Databricks Security Operations
• Azure Active Directory (Azure AD)
• AD Users and RBAC with IAM
• Owner, Contributor & Reader Roles
• Workspace Admin Permissions
• Notebook Permissions and Share Options
• Shared Notebooks, User Access Options
• Notebook Operations: Clone & Export

• Databricks Jobs: Creation Options, Usage
• Job Limits, Workspace, Concurrency Limits
• Notebooks with and without Parameters
• Jobs with Default Parameters, Executions
• Interactive, Automated Clusters for Jobs
• Job Schedules and Manual Executions
• Active Jobs, Recently Run Jobs, Monitoring
• ADB Jobs with Azure OData Sources, BLOB

Chapter 6: ADF Data Flow - 1

• Limitations with Copy Data Tool
• Data Flow Task, Data Flow Activity
• Transformations with Data Flow
• Spark Cluster For Debugging
• Cluster Node Configurations
• Data Preview Options with DFT
• SELECT Transformation & Options
• JOIN Transformation and Usage

• Conditional Split Transformation
• Aggregate & Group By Transformations
• Synapse Sink Options with DFT
• DFT Optimization Techniques
• Pipeline Debug Runs and ETL Testing
• Spark Cluster For Pipeline Executions
• Pipeline Monitoring & Run IDs

Chapter 6: Azure Stream Analytics, IoT

• Azure Stream Analytics: Real-time Usage
• Real-time Data Processing, Event Tracking\
• Ingest, Deliver and Analysis Operations
• Azure Stream Analytics Jobs Concept
• Understanding Input & Output Options
• SAQL Queries for Stream Analytics Jobs
• IoT: Internet Of Things For Real-time Data
• Need for IoT Hubs and Event Hubs

• Creating IoT Device for Data Inputs
• Creating Azure Strean Analytics Resource
• Stream Analytics Jobs for Historical Data
• Azure SQL Database Options for ASA Jobs
• SAQL: Query Formatting and Validation
• Historical Data Uploads, ASA Job Execution
• Stream Analytics Job Monitoring Options

Chapter 6: Databricks @ BLOB, Power BI

• BLOB Data Access with Databricks
• Accessing Storage Account, Container
• Gerate, Use SAS: Shared Access Signature
• dbutils.fs.mount() with DBFS Store
• fs.azure.sas.container.strorageaccount
• spark.read() and DBFS Mounts
• Scala Transformations, Create Temp View
• Spark SQL Queries with Temp Views

• dataframe.write.jdbc() & JVM Properties
• spark.read.jdbc() with Azure SQL DB
• Power BI Integration with Databricks
• Server Host Name, Port and Http Path
• Cluster Configurations and JDBC
• User Access Token Generation, Usage
• Spark ClusterAccess, Power BI Analytics

Chapter 7: ADF Data Flow - 2

• ADF Pipelines For ETL Operations
• Data Flow Tasks and Activities in Synapse
• Pivot Transformation For Normalization
• Generating Pivot Column, Aggregations
• Pivot Transformation and Pivot Settings
• Pivot Key Selection, Value and Nulls
• Pivoted Columns and Column Pattern
• Column Prefix, Help Graphic & Metadata

• Window Functions & Usage in Data Flow
• Rank / DenseRank / Row Number
• Over Clause and Input Options
• Derived Column Transformations
• Exists & Lookup Transformations
• Reusing Data Flow Tasks in Synapse
• Pipeline Validations & Executions

Chapter 7: IoT Hubs & Event Hubs

• Azure Stream Analytics For API Data
• IoT Hubs & IoT Devices, Connection Strings
• Rasberry APP Connections with IoT Hub
• Azure Storage Account and Container
• Creating Azure Stream Analytics Job
• Configuring Input Aliases with IoT Hub
• Configuring Output Alias with ADLS Gen 2
• SAQL Query and Job Executions; Monitoring

• Azure Event Hubs and Event Instances
• Event Hub Namespaces, Partition Counts
• Access Policies, Permissions & Defaults
• RootManageSharedAccessKey & Options
• Connection Strings & Event Service Bus
• Telco App Installation, Executions. LIVE Data
• On-Premise App Integration with ASA Jobs

Chapter 7: Databricks Integrations

• Azure Databricks with Data Lake Storage
• Handling Unstructured Data in Azure
• Data Preparation and Staging Operations
• Azure App (Service Principal) Registration
• Azure Key Vault Creation & Key Usage
• Service Principal Permissions @ Data Loads
• Tenants and Authorization Settings
• Client Credentials, Token Provider Options

• Spark Notebooks For Dynamic Connections
• Parameterized Options & Blob Access
• Data Preparation & Big Data Ingestion
• Data Extraction and ADLS Storage
• show(), transformations, wasbs Options
• Azure SQL Server & Synapse Creations
• Data Loads with Incremental Changes

Chapter 8: Azure Synapse Analytics

• Azure Synapse Analytics Resource
• Azure Synapse Analytics Workspace
• Managed Resource Group, SQL Account
• SQL Admin Account and its Purpose
• Operations with Synapse Workspace
• ADLS Gen 2 Storage Account, Container
• Synapse Studio (Synapse Portal)
• Dedicated SQL Pools & Spark Pools

• Creating Dedicated SQL Pools
• Synapse Tables, Data Loads with T-SQL
• COPY INTO Statements with T-SQL
• Clustered Column Store Indexes
• Row Terminator and Compressions
• T-SQL Queries and Aggregations
• Aggregation Data Loads in Synapse

Chapter 8: Azure Stream Analytics Security

• Azure Key Vaults & ADLS [Data Lake] Security
• Azure Passwords, Keys and Certificates
• Azure Key Vaults - Name and Vault URI
• Inbuilt Managed Key and Azure Key Vault
• Standard Type, Premium Type Azure Key Vaults
• Secret Page, Key Backups and Key Restores
• Adding Keys to Azure Vaults. Key Type, Size
• Using Azure Key Vaults to secure Resources

• Azure Storage: Replications and DR Options
• LRS: Locally Redundant Storage
• GRS: Globally Redundant Storage
• ZRS: Zone Redundant Storage
• Replication Options and Advantages
• Replication Verification and Modifications
• Azure Storage Endpoints, Failover Partner

Real-Time Project

• ADF Integration, Real-time Project
• Azure Databricks Integrations with ADF
• Defining Scala Notebooks in ADB
• Using Notebooks in Azure Data Factory
• spark.conf.set & fs.azure.account.key
• spark.read.format, Option() and Head()
• Online Retail Database Data Source
• Azure Migrations and ETL Concepts
• Azure SQL Pool (Synapse DWH) Tables

• Apache Spark Pool : Databases, Tables
• Azure Data Lake Storage (ADLS Gen 2)
• Azure Stream Analytics Jobs with IoT
• Azure Data Bricks and DBFS, Notebooks
• Concept wise FAQs, Resume Guidance
• Project Requirement, Solution, FAQs
• DP 203 Certification Guidance

Chapter 9: Synapse Analytics with Spark

• Apache Spark Pool in Azure Synapse
• Spark Cluster Nodes: Vcores, Memory
• Creating Spark Clusters @ Synapse Studio
• Python Notebooks For Remote Access
• Creating Databases in Apache Spark Pool
• Data Loads from Dedicated SQL Pools
• Table Creations, Aggregation Operations
• PySpark Code for Data Operations, Writes

• Serverless Pool in Azure Synapse
• Connections, Usage with Serverless Pool
• Using Azure OpenDatasets in Synapse
• OPENROWSET and BULK Data Loads
• Azure Storage Account : Data Analysis
• Working with Parquet Files in Synapse
• Python Notebooks (Pyspark) in Synapse

Chapter 10: Incremental Loads @ Synapse

• Incremental Loads with Synapse Studio
• Multi Table Merge Operations
• On-Premise Data Sources & Timestamps
• Azure SQL DB Destinations, Watermarks
• Watermark Table Usage & Audits
• Stored Procedures for Timestamp Updates
• Table Data Type and Dynamic MERGE
• SQL Queries for Datasets and Fetch

• Lookup Activity and its Usage un Synapse
• Expressions in ADF Portal for Lookup
• Expressions in ADF Portal for Source
• Output Pipeline Expression, Data Window
• Concat Function, Run IDs Expressions
• JSON Parameters, Pipeline Scheduling
• Pipeline Validation, Trigger and Monitoring

Chapter 11: Optimizations, Power Query

• ADF ETL with GUI : Power Query
• Power Query Resoruce Creation, Use
• Source Data Configurations & Settings
• Rename, Remove, Pivot, Group By, Order
• Index, Filter, Remove Error Rows
• Using Power Query Activity, ADF Pipelines
• Spark Cluster Configurations for Pipelines
• Concurrency, Big Data Recommendations

• Storage Optimization Techniques
• ETL Optimization Techniques
• SQL Pool (Synapse) Optimizations
• Indexes, Partitions, Distributions, DOP
• Pipeline Optimization Techniques
• Partitions, DOCP, Compressions, DIU
• Staging, Polybase and Core Counts

Chapter 12: Pipeline Monitoring, Security

• Azure Monitor Resource and Usage
• Pipeline Monitoring Techniques
• ADF: Pipeline Monitoring and Alerts
• Synapse: Pipeline Monitoring and Alerts
• Synapse: Storage Monitoring and Alerts
• Conditions, Signal Rules and Metrics
• Email Notifications with Azure
• Concurrency, Big Data Recommendations

• Azure Active Directory (AAD) Users, Groups
• IAM: Identity & Access Management
• Synapse Workspace Security with RBAC
• ADF Security with RBAC: Owner, Contributor
• Azure Synapse SQL Pool Security: Logins
• Users, Roles and Resource Classes (RC)
• ADF V1 to V2 Migrations, Considerations