| 01. Course Overview.mp4 | 3.7 MB | ||
| 02. Prerequisites and Course Outline.mp4 | 2.9 MB | ||
| 03. Delta Lake.mp4 | 11.8 MB | ||
| 04. Data Ingestion-Definition Challenges and Best Practices.mp4 | 10.7 MB | ||
| 05. Auto Loader for Data Ingestion.mp4 | 4.3 MB | ||
| 06. Demo-Creating an External Cloud Storage Source for Ingestion of Files.mp4 | 10.7 MB | ||
| 07. Demo-Ingesting Streaming Data into Delta Lake.mp4 | 12.6 MB | ||
| 08. Demo-Tracking Processed Files using Auto Loader.mp4 | 9.6 MB | ||
| 09. Demo-Ingesting Batch Data into Delta Lake.mp4 | 5.6 MB | ||
| 10. Demo-Ingesting Data into Delta Lake Using SQL.mp4 | 6.5 MB | ||
| 11. Databricks Data Ingestion Network.mp4 | 5.5 MB | ||
| 12. Performance Issues in Spark.mp4 | 6.7 MB | ||
| 13. Performance Bottlenecks in Spark-Serialization and Skew.mp4 | 7.5 MB | ||
| 14. Performance Bottlenecks in Spark-Spill Shuffle and Memory.mp4 | 7 MB | ||
| 15. Memory Partitions and Disk Partitions.mp4 | 2.1 MB | ||
| 16. Demo-Disk Partitioning.mp4 | 16.4 MB | ||
| 17. Data Skipping and Z-order Clustering.mp4 | 4.9 MB | ||
| 18. Demo-Z-ordering on a Small Delta Table.mp4 | 7.9 MB | ||
| 19. Demo-Z-ordering on a Large Delta Table.mp4 | 6.9 MB | ||
| 20. Bucketing to Optimize Joins.mp4 | 3.3 MB | ||
| 21. Demo-Bucketed and Unbucketed Tables.mp4 | 8.8 MB | ||
| 22. Demo-Joining Bucketed and Unbucketed Tables.mp4 | 12.1 MB | ||
| 23. FIFO and Fair Schedulers.mp4 | 7.2 MB | ||
| 24. Demo-Default Pool FIFO Scheduling.mp4 | 10.8 MB | ||
| 25. Demo-Configuring Different Pools to Share Resources.mp4 | 8.3 MB | ||
| 26. Delta Cache.mp4 | 7.1 MB | ||
| 27. Demo-Configuring the Delta Cache on a Cluster.mp4 | 3.6 MB | ||
| 28. Demo-Running Queries on Cached Data.mp4 | 11.4 MB | ||
| 29. New Features in Apache Spark 3.0.mp4 | 4.5 MB | ||
| 30. Summary and Further Study.mp4 | 1.8 MB | ||
| Bonus Resources.txt | 409.6 B | ||
| DS_Store | 6 KB | ||
| Get Bonus Downloads Here.url | 204.8 B | ||
| bike_sharing.csv | 18.5 MB | ||
| demo-01-AutoLoaderWithStreamingData.html | 808.1 KB | ||
| demo-01-AutoLoaderWithStreamingData.ipynb | 66.6 KB | ||
| demo-02-AutoLoaderWithBatchData.html | 348.4 KB | ||
| demo-02-AutoLoaderWithBatchData.ipynb | 14 KB | ||
| demo-03-SQLForDataIngestion.html | 345.1 KB | ||
| demo-03-SQLForDataIngestion.ipynb | 25.1 KB | ||
| demo-04-DiskPartitioning.html | 1.5 MB | ||
| demo-04-DiskPartitioning.ipynb | 1.3 MB | ||
| demo-05-Z-Ordering.html | 1.3 MB | ||
| demo-05-Z-Ordering.ipynb | 1.2 MB | ||
| demo-06-Bucketing.html | 918.1 KB | ||
| demo-06-Bucketing.ipynb | 859.8 KB | ||
| demo-07-SchedulerPool.html | 610.5 KB | ||
| demo-07-SchedulerPool.ipynb | 33.2 KB | ||
| demo-08-DeltaCache.html | 756.9 KB | ||
| demo-08-DeltaCache.ipynb | 550.4 KB | ||
| diagnosing-and-mitigating-performance-problem-slides.pdf | 1.9 MB | ||
| exploring-and-mitigating-data-ingestion-problem-slides.pdf | 2.9 MB | ||
| fairscheduler.xml | 307.2 B | ||
| optimizing-spark-for-performance-slides.pdf | 1.3 MB | ||
| summer.csv | 2.5 MB | ||
| tips_01.csv | 2.5 KB | ||
| tips_02.csv | 2.6 KB | ||
| tips_03.csv | 3.1 KB | ||
| video_games_data_01.csv | 3.6 KB | ||
| video_games_data_02.csv | 3.7 KB | ||
| video_games_data_03.csv | 3.8 KB | ||
| video_games_data_04.csv | 3.9 KB | ||
| ▲ 123 total files | |||
Optimizing Apache Spark on Databricks
https://TutGee.com
Duration: 2h 19s | Video: .MP4, 1280x720 30 fps | Audio: AAC, 48 kHz, 2ch | Size: 291 MB
Genre: eLearning | Language: English
This course will teach you how to optimize the performance of Spark clusters on Azure Databricks by identifying and mitigating various issues such as data ingestion problems and performance bottlenecks
What you'll learn
The Apache Spark unified analytics engine is an extremely fast and performant framework for big data processing. However, you might find that your Apache Spark code running on Azure Databricks still suffers from a number of issues. These could be due to the difficulty in ingesting data in a reliable manner from a variety of sources or due to performance issues that you encounter because of disk I/O, network performance, or computation bottlenecks.
In this course, Optimizing Apache Spark on Databricks, you will first explore and understand the issues that you might encounter ingesting data into a centralized repository for data processing and insight extraction. Then, you will learn how Delta Lake on Azure Databricks allows you to store data for processing, insights, as well as machine learning on Delta tables and you will see how you can mitigate your data ingestion problems using Auto Loader on Databricks to ingest streaming data.
| torrent name | size | uploader | age | seed | leech |
|---|---|---|---|---|---|
| 347.3 MB | abidmail | 5 years | 0 | 0 | |
| 322.4 MB | freecoursewb | 5 years | 7 | 0 | |
|
[ FreeCourseWeb ] PluralSight - Optimizing Fields, Tags, and Event Types in Splunk Posted by
freecoursewb in Other
|
216.6 MB | freecoursewb | 6 years | 3 | 0 |
| 1.79 GB | NepsterJay | 11 years | 1 | 0 |
All Comments