Tag: Hadoop
-
Day 24
Data Warehouse Online Analytical Processing (OLAP). Store structured data, data source from Online Transactional Processing (OLTP), live database. Storage repository; not storing live data, only store pass/historical data. Mostly used for model training, analysis and pattern discovery. Example, student data stored in database but graduated student data stored in data warehouse. Data Lake Store both…
-
Day 6
Map Reduce refer to– https://informationit27.medium.com/hadoop-mapreduce-in-action-b7c723b604ba– https://www.slideshare.net/mudassarmulla/tutorial-hadoop-hdfsmapreduce– https://cwiki.apache.org/confluence/display/HADOOP2/JobTracker– https://www.youtube.com/watch?v=ULtOZqlZnCw Tools built on top of Map Reduce Hive – convert HSQL into Map Reduce, easier coding with SQL alike syntax without java programming. From Facebook. Spark Shortcoming of Map Reduce lacks of abstractions for leveraging distributed memory write result to external storage system inefficient for an important…
-
Day 5 (1)
Tutorial 4 Tutorial 5 Discuss and evaluate suitable techniques/methods being used in literature while performing the big data analytics on the following:a) Market Basket Analysis.b) Customer Churn Prediction Analysis.Please support your discussion based on a research paper. Example AnswerThe big data analytics on Market Basket Analysis could help to· Provide combo offers based on products…
-
Day 4
5 Daemon “Daemon” that sound like demon, is the background service that not initiated by user. HDFS (Slave)……………….. Data Node …………….(Master) Name Node …………….(Master) Secondary Name Node (Backup) Map Reduce (Slave)……………….. Task Tracker …………….(Master) Job Tracker Hadoop is distributed storage and processing. It only means the data node (storage & processing), not for name node;…
-
Day 3
What is the benefit of distributed? Using parallel concept, original task might complete in 11 hour, but if parallel in 4 machine, it would took only 3 hour. Challenges Division of data (got 4 machine but data is divided into 3) Distribution of data (2TB to M1, 1TB to M2, 500GB to M3, 500GB to…
