WELCOME TO RICHFIELD DISTANCE LEARNING

[Audio] WELCOME TO RICHFIELD DISTANCE LEARNING BIG DATA & INTERNET OF THINGS 600.

[Audio] PRESCRIBED OR RECOMMENDED BOOKS Prescribed Textbook: Geng, H. (2017).Internet of Things and Data Analytics Handbook. United States: Wiley. ISBN:9781119173649 Recommended Reading: Big Data, IOT and Machine Learning Tools and applications ISBN:9780367336745.

[Audio] BIG DATA & IOT 600 Chapter 1: Big Data Analysis and Extraction Techniques Chapter 2: IoT Architectures and Applications Chapter 3: IoT technologies and Standards Chapter 4: Big Data Storage and Security Chapter 5: Big Data ethics, standards and policies.

[Audio] Chapter 1 Big Data Analysis and Extraction Techniques.

[Audio] Big Data is the term used to describe the application of specialized techniques to process very large sets of data, which are often too complex to process using regular database management tools. The challenges that big data solves include capture, storage, search, sharing, transfer, analysis, and visualization..

[Audio] Main characteristics of big data Volume Velocity Variety Veracity Value.

[Audio] Volume (Scale) Data Volume 44x increase from 2009 to 2020 From 0.8 zettabytes to 35zb Data volume is increasing exponentially Exponential increase in collected/generated data.

[Audio] mobile-internet TrendsMontage 2+ billion people on the Web by end 2011 30 billion RFID tags today (1.3B in 2005) 4.6 billion camera phones world wide 100s of millions of GPS enabled devices sold annually 76 million smart meters in 2009… 200M by 2014 12+ TBs of tweet data every day ? TBs of data every day 25+ TBs of log data every day.

[Audio] Variety (Complexity) Relational Data (Tables/Transaction/Legacy Data) Text Data (Web) Semi-structured Data (XML) Graph Data Social Network, Semantic Web (RDF), … Streaming Data You can only scan the data once A single application can be generating/collecting many types of data Big Public Data (online, weather, finance, etc) To extract knowledge all these types of data need to linked together.

[Audio] Velocity (Speed) Data is being generated fast and need to be processed fast Online Data Analytics Late decisions  missing opportunities Examples E-Promotions: Based on your current location, your purchase history, what you like  send promotions right now for store next to you Healthcare monitoring: sensors monitoring your activities and body  any abnormal measurements require immediate reaction.

[Audio] Veracity Data veracity, in general, is how accurate or truthful a data set may be. In the context of big data, however, it takes on a bit more meaning. More specifically, when it comes to the accuracy of big data, it’s not just the quality of the data itself but how trustworthy the data source, type, and processing of it is..

[Audio] Value Insight discovery and pattern recognition that lead to more effective operations, stronger customer relationships and other clear and quantifiable business benefits..

[Audio] Sources of Big data Business Transaction Social Media Platforms Internet Of Things Web and digital activity Medical Records Scientific Research.

[Audio] Common tools and frameworks used for big data analysis. Hadoop Apache Spark Apache Kafka MongoDB Tableau Amazon Web Services (AWS) Apache Flink Power BI.

[Audio] Big Data Analysis Techniques: Descriptive analysis Predictive analysis Deep learning and Machine Learning (ML) Real-time analytics Sentiment analysis.

[Audio] Data Cleaning Finding and fixing mistakes, inconsistencies, and missing values in raw data is the goal of data cleaning..

[Audio] Data Normalization The goal is to standardize and scale data in order to facilitate efficient processing and comparison..

[Audio] Data Transformations The goal of data transformation is to transform unprocessed data into a format that may be used for analysis or model construction..

[Audio] Big Data Challenges Complexity: Data in different formats and structures (such as structured, semi-structured, and unstructured) must be handled by transformation pipelines. Real-Time Constraints: Workflows using streaming data require quick transformations. Resource Management: To reduce memory and computation consumption, large-scale transformations need to be optimized..

[Audio] Review questions What is Big Data, and what are its defining characteristics (e.g., the 3Vs or 5Vs)? Explain the difference between structured, unstructured, and semi-structured data. Describe how Big Data differs from traditional data management systems in terms of scale and processing. List at least three tools or frameworks commonly used for Big Data processing and describe their primary purposes. Compare Hadoop and Spark in terms of data processing capabilities and use cases. What is the role of NoSQL databases (e.g., MongoDB, Cassandra) in Big Data environments?.

[Audio] All the best with your studies!!!. All the best with your studies!!!.