DP-203 Microsoft Azure Data Engineer Associate Training

LU1 Get started with data engineering on Azure

Topic 1 Introduction to data engineering on Azure

• What is data engineering
• Important data engineering concepts
• Data engineering in Microsoft Azure

Topic 2 Introduction to Azure Data Lake Storage Gen2

• Understand Azure Data Lake Storage Gen
• Enable Azure Data Lake Storage Gen2 in Azure Storage
• Compare Azure Data Lake Store to Azure Blob storage
• Understand the stages for processing big data
• Use Azure Data Lake Storage Gen2 in data analytics workloads

Topic 3 Introduction to Azure Synapse Analytics

• What is Azure Synapse Analytics
• How Azure Synapse Analytics works
• When to use Azure Synapse Analytics
• Exercise - Explore Azure Synapse Analytics

LU2 Build data analytics solutions using Azure Synapse serverless SQL pools

Topic 4 Use Azure Synapse serverless SQL pool to query files in a data lake

• Understand Azure Synapse serverless SQL pool capabilities and use cases
• Query files using a serverless SQL pool
• Create external database objects
• Exercise - Query files using a serverless SQL pool

Topic 5 Use Azure Synapse serverless SQL pools to transform data in a data lake

• Transform data files with the CREATE EXTERNAL TABLE AS SELECT statement
• Encapsulate data transformations in a stored procedure
• Include a data transformation stored procedure in a pipeline
• Exercise - Transform files using a serverless SQL pool

Topic 6 Create a lake database in Azure Synapse Analytics

• Understand lake database concepts
• Explore database templates
• Create a lake database
• Use a lake database
• Exercise - Analyze data in a lake database

Topic 7 Secure data and manage users in Azure Synapse serverless SQL pools

• Choose an authentication method in Azure Synapse serverless SQL pools
• Manage users in Azure Synapse serverless SQL pools
• Manage user permissions in Azure Synapse serverless SQL pools

LU3 Perform data engineering with Azure Synapse Apache Spark Pools

Topic 8 Analyze data with Apache Spark in Azure Synapse Analytics

• Get to know Apache Spark
• Use Spark in Azure Synapse Analytics
• Analyze data with Spark
• Visualize data with Spark
• Exercise - Analyze data with Spark

Topic 9 Transform data with Spark in Azure Synapse Analytics

• Modify and save dataframes
• Partition data files
• Transform data with SQL
• Exercise: Transform data with Spark in Azure Synapse Analytics

Topic 10 Use Delta Lake in Azure Synapse Analytics

• Understand Delta Lake
• Create Delta Lake tables
• Create catalog tables
• Use Delta Lake with streaming data
• Use Delta Lake in a SQL pool
• Exercise - Use Delta Lake in Azure Synapse Analytics

LU4 Work with Data Warehouses using Azure Synapse Analytics

Topic 11 Analyze data in a relational data warehouse

1. Design a data warehouse schema
2. Create data warehouse tables
3. Load data warehouse tables
4. Query a data warehouse
5. Exercise - Explore a data warehouse

Topic 12 Load data into a relational data warehouse

• Load staging tables
• Load dimension tables
• Load time dimension tables
• Load slowly changing dimensions
• Load fact tables
• Perform post load optimization
• Exercise - load data into a relational data warehouse

Topic 13 Manage and monitor data warehouse activities in Azure Synapse Analytics

• Scale compute resources in Azure Synapse Analytics
• Pause compute in Azure Synapse Analytics
• Manage workloads in Azure Synapse Analytics
• Use Azure Advisor to review recommendations
• Use dynamic management views to identify and troubleshoot query performance

Topic 14 Secure a data warehouse in Azure Synapse Analytics

• Understand network security options for Azure Synapse Analytics
• Configure Conditional Access
• Configure authentication
• Manage authorization through column and row level security
• Exercise - Manage authorization through column and row level security
• Manage sensitive data with Dynamic Data Masking
• Implement encryption in Azure Synapse Analytics

LU5 Transfer and transform data with Azure Synapse Analytics pipelines

Topic 15 Build a data pipeline in Azure Synapse Analytics

• Understand pipelines in Azure Synapse Analytics
• Create a pipeline in Azure Synapse Studio
• Define data flows
• Run a pipeline
• Exercise - Build a data pipeline in Azure Synapse Analytics

Topic 16 Use Spark Notebooks in an Azure Synapse Pipeline

• Understand Synapse Notebooks and Pipelines
• Use a Synapse notebook activity in a pipeline
• Use parameters in a notebook
• Exercise - Use an Apache Spark notebook in a pipeline

LU6 Work with Hybrid Transactional and Analytical Processing Solutions using Azure Synapse Analytics

Topic 17 Plan hybrid transactional and analytical processing using Azure Synapse Analytics

• Understand hybrid transactional and analytical processing patterns
• Describe Azure Synapse Link

Topic 18 Implement Azure Synapse Link with Azure Cosmos DB

• Enable Cosmos DB account to use Azure Synapse Link
• Create an analytical store enabled container
• Create a linked service for Cosmos DB
• Query Cosmos DB data with Spark
• Query Cosmos DB with Synapse SQL
• Exercise - Implement Azure Synapse Link for Cosmos DB

Topic 19 Implement Azure Synapse Link for SQL

• What is Azure Synapse Link for SQL?
• Configure Azure Synapse Link for Azure SQL Database
• Configure Azure Synapse Link for SQL Server
• Exercise - Implement Azure Synapse Link for SQL

LU7 Implement a Data Streaming Solution with Azure Stream Analytics

Topic 20 Get started with Azure Stream Analytics

• Understand data streams
• Understand event processing
• Understand window functions
• Exercise - Get started with Azure Stream Analytics

Topic 21 Ingest streaming data using Azure Stream Analytics and Azure Synapse Analytics

• Stream ingestion scenarios
• Configure inputs and outputs
• Define a query to select, filter, and aggregate data
• Run a job to ingest data
• Exercise - Ingest streaming data into Azure Synapse Analytics

Topic 22 Visualize real-time data with Azure Stream Analytics and Power BI

• Use a Power BI output in Azure Stream Analytics
• Create a query for real-time visualization
• Create real-time data visualizations in Power BI
• Exercise - Create a real-time data visualization

LU8 Govern data across an enterprise

Topic 23 Introduction to Microsoft Purview

• What is Microsoft Purview?
• How Microsoft Purview works
• When to use Microsoft Purview

Topic 24 Discover trusted data using Microsoft Purview

• Search for assets
• Browse assets
• Use assets with Power BI
• Integrate with Azure Synapse Analytics

Topic 25 Catalog data artifacts by using Microsoft Purview

• Register and scan data
• Classify and label data
• Search the data catalog

Topic 26 Manage Power BI assets by using Microsoft Purview

• Register and scan a Power BI tenant
• Search and browse Power BI assets
• View Power BI metadata and lineage

Topic 27 Integrate Microsoft Purview and Azure Synapse Analytics

• Catalog Azure Synapse Analytics data assets in Microsoft Purview
• Connect Microsoft Purview to an Azure Synapse Analytics workspace
• Search a Purview catalog in Synapse Studio
• Track data lineage in pipelines
• Exercise - Integrate Azure Synapse Analytics and Microsoft Purview

LU9 Data engineering with Azure Databricks

Topic 28 Explore Azure Databricks

• Get started with Azure Databricks
• Identify Azure Databricks workloads
• Understand key concepts
• Exercise - Explore Azure Databricks

Topic 29 Use Apache Spark in Azure Databricks

• Get to know Spark
• Create a Spark cluster
• Use Spark in notebooks
• Use Spark to work with data files
• Visualize data
• Exercise - Use Spark in Azure Databricks

Topic 30 Use Delta Lake in Azure Databricks

• Get Started with Delta Lake
• Create Delta Lake tables
• Create and query catalog tables
• Use Delta Lake for streaming data
• Exercise - Use Delta Lake in Azure Databricks

Topic 31 Use SQL Warehouses in Azure Databricks

• Get started with SQL Warehouses
• Create databases and tables
• Create queries and dashboards
• Exercise - Use a SQL Warehouse in Azure Databricks

Topic 32 Run Azure Databricks Notebooks with Azure Data Factory

• Understand Azure Databricks notebooks and pipelines
• Create a linked service for Azure Databricks
• Use a Notebook activity in a pipeline
• Use parameters in a notebook
• Exercise - Run an Azure Databricks Notebook with Azure Data Factory

Practice Exam

Promotion Code

Your will get 10% discount voucher for 2nd course onwards if you write us a Google review.

Minimum Entry Requirement

Knowledge and Skills

• Able to operate using computer functions
• Minimum 3 GCE ‘O’ Levels Passes including English or WPL Level 5 (Average of Reading, Listening, Speaking & Writing Scores)

Attitude

• Positive Learning Attitude
• Enthusiastic Learner

Experience

• Minimum of 1 year of working experience.

Target Age Group: 18-65 years old

Minimum Software/Hardware Requirement

Software:

• R https://cran.r-project.org/
• R Studio https://www.rstudio.com/products/rstudio/download/

Hardware: Window or Mac Laptops

• Data Engineer
• Azure Data Engineer
• Cloud Data Engineer
• Business Intelligence Developer
• Data Architect
• Data Scientist
• Machine Learning Engineer
• Big Data Engineer
• Data Analyst
• Database Administrator
• Cloud Solutions Architect
• DevOps Engineer
• Software Developer
• IT Professional
• Systems Analyst

Sanjiv Venkatram: Sanjiv i is an ACTA certified experienced leader with a proven track record in business / finance consulting and in developing i) business intelligence (BI) solutions ii) data analytics/analysis solutions and iii) IOT lead BI solutions. Sanjiv's goal through Prudentia Consulting, is to promote the simple joy and excitement of actively using the Microsoft Platform. He believes that the agility afforded by the Microsoft platform helps businesses get time back for deeper business thinking and to spend more time with their end customers

Sanjiv has rich experiences in diverse/complex high-tech businesses, turn around environments and strategic transformations. His functional expertise is in sales analytics, financial planning and analysis, engineering and program management. He has worked across discrete manufacturing, professional services and higher education verticals. He also has a working knowledge of equities portfolio management within the financial services domain.Sanjiv is the CEO of Prudentia Consulting, an organization committed to promoting the active usage of the Microsoft Platform. Prior to this, he has worked at Microsoft (US & APAC: 9.2 years), Cognizant Tech Solutions (3.3 years), Yazaki North America (8 years) and until recently at Oracle. Here are a few of his BI/analytics projects driven at scale: Built APAC wide BI dashboard using the Power BI umbrella tool set (Power BI online, Power BI desktop and Power Pivot) and a KPI lake (SQL DB), Helped develop key KPIs – identified key KPIs and helped land this in the DB, Developed a budget audit tool that captured budget inputs from a host of countries across the globe, Developed a business unit P&L reporting tool (functional architecture) in Business Objects for the world-wide financial planning and analysis team.

Alec Tan: Alec Tan is a ACTA certified trainer, He has a number of Comptia certifications. Since 2002, starting off from IT technical background to pre-sales, sales account manager, system integration, operate IT retail / repair shop business in Sim Lim Square 2008 ~ 2012, and back to IT industry employment, freelance IT Trainer till present.

Kishan Raaj: Kishan Raaj is a seasoned Data Science and SAP Program Manager with over 15 years of experience, currently leading all end-to-end training and projects for Data Science across Asia at LITHAN, Singapore. With expertise in machine learning, data analytics, and cloud technologies, he is well-versed in techniques such as regression, decision trees, clustering, and text mining. He has hands-on experience with tools like Python, R, Tableau, PowerBI, and Microsoft Azure Machine Learning Studio.
As an ACTA certified trainer, Kishan has designed and delivered comprehensive courses for SAP and Data Science programs, managing teams and training in multiple countries, including Singapore, Malaysia, China, India, and Vietnam. His background in SAP, particularly in S4/HANA and ABAP, combined with his expertise in machine learning libraries such as TensorFlow and Keras, positions him as a highly capable leader in technical education and project management across various industries.

Quah Chee Yong: Quah Chee Yong is a ACTA certified trainer. Quah Chee Yong Chee Yong is an experienced professional who has held various Technical, Operations and Commercial positions across several industriesA firm believer that AI can create a better world, he has equipped himself with the Knowledge and Skills in the fields of Data Science, Machine Learning, Deep Learning and Cloud Deployment

He has a deep passion for training & facilitating and is currently a Singapore WSQ certified Adult Educator. He particularly enjoys the interactive engagements with his fellow trainers and learners

Peter Cheong :  I am a ACLP certied trainer. Specialise In IT related knowlege and conduct IT Training which Include Microsoft Window Server Technology (Wintel) and Linux - Centos/Red Hat. Comptia ,ITIL , Motorola Solution Trunking System and Cisco Networking. I was worked in Motorola Solutions Conduct Motorola Astro 25 Trunking System For Police Force Malaysia (RMPnet),Taiwan Navy, Indonesia METRO POLDA (Police Force). After that I Join As IT Group Manager For W-Group which include 17 subsidiaries Companies in Real Estate Developer,Plantation, Building Management Services ,Contruction and also Fiber Opti Service Provider in Sabah,Malaysia.

Alan Tan: Alan Tan is a ACTA certified trainer. Alan Tan hold Masters in Data Science in Singapore University of Technology and Design. He is Microsoft Power Platform PL-900 certified and Microsoft Azure Data Fundamentals  Certified DP-900. He have this skills in Microsoft Power Platform development skills such as Powerapps, Power Automate,Power Portal, Power Virtual Agent, Power BI, Sharepoint, Dataverse, etc. For Programming language such as Python and Robotics Process Automation (RPA) development. He is working as a Senior Data Engineer in an MNC.

Bernard Peh: Bernard Peh is a Business Development Director and ACTA certified trainer with over 20 years of experience in the financial services industry. He has held key leadership roles and integrated technology, digital marketing, and data science to drive growth in sales, recruitment, and financial planning. As a mentor, Bernard has developed many successful financial professionals who have achieved top industry accolades like TOT, COT, and MDRT.

With deep expertise in data science, Bernard has advised firms and designed training programs for institutions like NTUC Learning Hub. He continues to apply data science to collective funds, achieving exceptional results such as a 400% increase in assets under management, while empowering financial professionals with technology-driven solutions.

Ken Hiong: Ken Hiong is a ACTA certified trainer. Ken has over 20 years of work experience in the healthcare and pharmaceutical industry, having assumed various functional and managerial roles in sales, marketing, business development, finance, business analysis and planning. With an interest and experience in scripting, Ken has worked on projects using HTML, CSS, PHP, MySQL, WordPress, MS Office, VBA, Power BI, etc. Notably, Ken is a proven expert Excel user at work who has made efficient many work processes, improved data analysis and enhanced the quality of business planning and reporting for organizations.

Ken graduated with a Pharmacy degree, holds a Master of Business Administration from the National University of Singapore and a Master of Applied Finance from the University of Adelaide. An ardent advocate of lifelong learning, Ken is ACTA trained and seeks to continually upgrade his IT skills through various channels of learning and Microsoft certifications. With MS Office Master qualification, Ken looks forward to assisting individuals and corporations improve their computer skills, productivity and business outcomes.

Raja Rohan: Raja Rohan is a certified specialist in data analytics, big data science, and business intelligence, with expertise in tools like Qlik and Microsoft reporting technologies. His strong technical foundation spans areas such as RPA (Robotic Process Automation), artificial intelligence, machine learning, deep learning, and natural language processing.

Raja has a proven track record of leading and managing projects in BI, data warehousing, and analytics, with a focus on data mining and statistical techniques. He has extensive experience in project delivery, from requirement analysis to final execution, ensuring quality and timely outcomes. As a team leader, Raja excels at driving performance through mentorship and effective communication, consistently delivering results in complex technical environments.

Audrey Lin : Audrey Lin is a ACTA certified trainer. She is a Business Development Director at Gill Technologies, an IT Consulting and Managed Services Provider focused at servicing SMEs. I'm responsible for the operations, sales, marketing, accounting, and finance functions of the organization. She have 13 years of experience in helping businesses and schools maximize their Return on Investment on IT through the use of the right technology, improving their business processes and increasing user adoption. My expertise is around Microsoft 365, MS Teams and Power BI. To help organizations improve employee productivity with tools from the MS 365 stack, she have delivered several classroom & online-styled end user training sessions to organizations from various industries - real estate, F&B, oil & gas, healthcare, schools, logistics. I have also created multiple customized & on-demand videos that end users can refer to for a quick review. Being certified as a Microsoft Data Analyst Associate, She is proficient at helping businesses gather business insights from their wide array of data. She have been involved in BI Projects with healthcare firms, logistics companies, FIs, etc., starting from the design phase, right through to the daily management of the BI system. Her area of knowledge in managing security and compliance in MS 365 is significant and ever-growing. She have good understanding of the Zero Trust Principle and uses that to improve the IT security posture of her clients. With the support of her engineering team at Gill Technologies, we use cloud-based management tools from MS 365, namely Intune, Endpoint Manager, Defender, etc. to manage our clients' endpoints.

Agus Salim : I am a professional with more than 10 years of experience in Project Management, IT Solutions Management, and Systems Integration both in waterfall and agile methodology. He started out his career as a Web Developer before moving on to Business Analyst/Project Manager. He has strong leadership and the capability of leading a team with a proven ability to deliver projects with tight timelines. Besides his experiences in managing projects, he has good knowledge in Cybersecurity and hands-on experience in Next Generation Firewall such as Check Point. During his free time, he likes to explore Cloud Technology, especially on Microsoft Azure.

Truman Ng: I am a ACLP certified trainer. With In-depth and diversified experiences from Project Management DevOps, Blockchain, Project Management, Education, IT and even Numerology, I am passionate to meet good people and generate new ideas, in pursuit of realization of better world!