Data Engineering on Google Cloud Platform (DEGCP)

Course Description Agenda Course Outline
 

Course Overview

This four-day instructor-led class provides participants a hands-on introduction to designing and building data processing systems on Google Cloud Platform. Through a combination of presentations, demos, and hand-on labs, participants will learn how to design data processing systems, build end-to-end data pipelines, analyze data and carry out machine learning. The course covers structured, unstructured, and streaming data.

A qui s'adresse cette formation

This class is intended for experienced developers who are responsible for managing big data transformations including:

  • Extracting, Loading, Transforming, cleaning, and validating data
  • Designing pipelines and architectures for data processing
  • Creating and maintaining machine learning and statistical models
  • Querying datasets, visualizing query results and creating reports

Pré-requis

To get the most of out of this course, participants should have:

  • Completed Google Cloud Fundamentals: Big Data and Machine Learning (GCF-BDM) course OR have equivalent experience
  • Basic proficiency with common query language such as SQL
  • Experience with data modeling, extract, transform, load activities Developing applications using a common programming language such Python
  • Familiarity with Machine Learning and/or statistics

Objectifs

This course teaches participants the following skills:

  • Design and build data processing systems on Google Cloud Platform
  • Process batch and streaming data by implementing autoscaling data pipelines on Cloud Dataflow
  • Derive business insights from extremely large datasets using Google BigQuery
  • Train, evaluate and predict using machine learning models using Tensorflow and Cloud ML
  • Leverage unstructured data using Spark and ML APIs on Cloud Dataproc
  • Enable instant insights from streaming data

Contenu

Module 1: Google Cloud Dataproc Overview
  • Creating and managing clusters.
  • Leveraging custom machine types and preemptible worker nodes.
  • Scaling and deleting Clusters.
  • Lab: Creating Hadoop Clusters with Google Cloud Dataproc.
Module 2: Running Dataproc Jobs
  • Running Pig and Hive jobs.
  • Separation of storage and compute.
  • Lab: Running Hadoop and Spark Jobs with Dataproc.
  • Lab: Submit and monitor jobs.
Module 3: Integrating Dataproc with Google Cloud Platform
  • Customize cluster with initialization actions.
  • BigQuery Support.
  • Lab: Leveraging Google Cloud Platform Services.
Module 4: Making Sense of Unstructured Data with Google’s Machine Learning APIs
  • Google’s Machine Learning APIs.
  • Common ML Use Cases.
  • Invoking ML APIs.
  • Lab: Adding Machine Learning Capabilities to Big Data Analysis.
Module 5: Serverless data analysis with BigQuery
  • What is BigQuery.
  • Queries and Functions.
  • Lab: Writing queries in BigQuery.
  • Loading data into BigQuery.
  • Exporting data from BigQuery.
  • Lab: Loading and exporting data.
  • Nested and repeated fields.
  • Querying multiple tables.
  • Lab: Complex queries.
  • Performance and pricing.
Module 6: Serverless, autoscaling data pipelines with Dataflow
  • The Beam programming model.
  • Data pipelines in Beam Python.
  • Data pipelines in Beam Java.
  • Lab: Writing a Dataflow pipeline.
  • Scalable Big Data processing using Beam.
  • Lab: MapReduce in Dataflow.
  • Incorporating additional data.
  • Lab: Side inputs.
  • Handling stream data.
  • GCP Reference architecture.
Module 7: Getting started with Machine Learning
  • What is machine learning (ML).
  • Effective ML: concepts, types.
  • ML datasets: generalization.
  • Lab: Explore and create ML datasets.
Module 8: Building ML models with Tensorflow
  • Getting started with TensorFlow.
  • Lab: Using tf.learn.
  • TensorFlow graphs and loops + lab.
  • Lab: Using low-level TensorFlow + early stopping.
  • Monitoring ML training.
  • Lab: Charts and graphs of TensorFlow training.
Module 9: Scaling ML models with CloudML
  • Why Cloud ML?
  • Packaging up a TensorFlow model.
  • End-to-end training.
  • Lab: Run a ML model locally and on cloud.
Module 10: Feature Engineering
  • Creating good features.
  • Transforming inputs.
  • Synthetic features.
  • Preprocessing with Cloud ML.
  • Lab: Feature engineering.
Module 11: Architecture of streaming analytics pipelines
  • Stream data processing: Challenges.
  • Handling variable data volumes.
  • Dealing with unordered/late data.
  • Lab: Designing streaming pipeline.
Module 12: Ingesting Variable Volumes
  • What is Cloud Pub/Sub?
  • How it works: Topics and Subscriptions.
  • Lab: Simulator.
Module 13: Implementing streaming pipelines
  • Challenges in stream processing.
  • Handle late data: watermarks, triggers, accumulation.
  • Lab: Stream data processing pipeline for live traffic data.
Module 14: Streaming analytics and dashboards
  • Streaming analytics: from data to decisions.
  • Querying streaming data with BigQuery.
  • What is Google Data Studio?
  • Lab: build a real-time dashboard to visualize processed data.
Module 15: High throughput and low-latency with Bigtable
  • What is Cloud Spanner?
  • Designing Bigtable schema.
  • Ingesting into Bigtable.
  • Lab: streaming into Bigtable.
Classroom training
Modality: C

Durée 4 jours

Prix (Hors Taxe)
  • France: 2 600,- €
Dates et Inscription
 
pointer une ville pour s'enregistrer Agenda
France
17.12. - 20.12.2019 Paris
FLEX Training

Please see below our alternative, English language, FLEX course option.

14.05. - 17.05.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Amsterdam
02.09. - 05.09.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Belgrade
03.09. - 06.09.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/London
14.10. - 17.10.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Sofia
19.11. - 22.11.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/London
02.12. - 05.12.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Bucharest

Fast Lane Flex™ Classroom If you can't find a suitable date, don't forget to check our world-wide FLEX training schedule.

Europe
Allemagne
14.05. - 17.05.2019 Berlin
03.06. - 06.06.2019 Düsseldorf
24.06. - 27.06.2019 Francfort
02.07. - 05.07.2019 Hambourg
16.07. - 19.07.2019 Münster
30.07. - 02.08.2019 Munich
13.08. - 16.08.2019 Berlin
27.08. - 30.08.2019 Düsseldorf
Autriche
30.07. - 02.08.2019 Wien (iTLS)
04.11. - 07.11.2019 Wien (iTLS)
Bulgarie
14.10. - 17.10.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Sofia
Espagne
07.05. - 10.05.2019 Madrid
16.07. - 19.07.2019 Madrid
22.10. - 25.10.2019 Madrid
Italie
21.05. - 24.05.2019 Milan
23.07. - 26.07.2019 Rome
03.09. - 06.09.2019 Milan
12.11. - 15.11.2019 Rome
10.12. - 13.12.2019 Milan
Pays-Bas
14.05. - 17.05.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Amsterdam
Portugal
14.05. - 17.05.2019 Lisbonne
23.07. - 26.07.2019 Lisbonne
05.11. - 08.11.2019 Lisbonne
Roumanie
02.12. - 05.12.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Bucharest
Royaume-Uni
03.09. - 06.09.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/London
19.11. - 22.11.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/London
Serbie
02.09. - 05.09.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Europe/Belgrade
Suisse
24.06. - 27.06.2019 Zürich
10.09. - 13.09.2019 Zürich
10.12. - 13.12.2019 Zürich
North America
Etats-Unis
18.06. - 21.06.2019 Formation en ligne Fuseau horaire: US/Pacific Langue: Anglais
20.08. - 23.08.2019 Formation en ligne Fuseau horaire: US/Eastern Langue: Anglais
22.10. - 25.10.2019 Formation en ligne Fuseau horaire: US/Central Langue: Anglais
17.12. - 20.12.2019 Formation en ligne Fuseau horaire: US/Eastern Langue: Anglais
Canada
20.08. - 23.08.2019 Formation en ligne Fuseau horaire: Canada/Eastern Langue: Anglais
22.10. - 25.10.2019 Formation en ligne Fuseau horaire: Canada/Central Langue: Anglais
Latin America
Argentine
10.12. - 13.12.2019 Formation en ligne Fuseau horaire: America/Buenos_Aires Langue: Español
Brésil
24.09. - 27.09.2019 Formation en ligne Fuseau horaire: America/Sao_Paulo Langue: Portuguais
03.12. - 06.12.2019 Formation en ligne Fuseau horaire: America/Sao_Paulo Langue: Portuguais
Chile
25.09. - 28.09.2019 Formation en ligne Fuseau horaire: America/Santiago 3 jours Langue: Español
Colombie
20.08. - 23.08.2019 Formation en ligne Fuseau horaire: America/Bogota Langue: Español
Costa Rica
14.05. - 17.05.2019 Formation en ligne Fuseau horaire: America/Costa_Rica Langue: Español
01.10. - 04.10.2019 Formation en ligne Fuseau horaire: America/Costa_Rica Langue: Español
Mexico
08.07. - 11.07.2019 Formation en ligne Fuseau horaire: America/Mexico_City Langue: Español
Peru
04.06. - 07.06.2019 Formation en ligne Fuseau horaire: America/Lima Langue: Español
05.11. - 08.11.2019 Formation en ligne Fuseau horaire: America/Lima Langue: Español
Asie Pacifique
Inde
28.05. - 31.05.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Asia/Calcutta
18.06. - 21.06.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Asia/Calcutta
Singapour
14.05. - 17.05.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Asia/Singapore
25.06. - 28.06.2019 FLEX training This is an Anglais language FLEX course.
Fuseau horaire: Asia/Singapore
Cours en ligne avec instructeur
Formation en mode FLEX™.