Authors:
Anjan Kumar Reddy Ayyadapu
Addresses:
Department of Information Technology, Cloudera Inc., Ashburn, Virginia, United States of America.
Abstract:
Internet of Things (IoT) devices have generated a record volume of real-time data that demands scalable and effective processing frameworks. This research presents scalable machine learning (ML) methods for real-time large-scale data analytics in IoT networks. For real-time applications, IoT data is too fast and complex for standard analytics systems; therefore, models must be accurate and computationally efficient. We propose a hybrid ML framework with distributed learning, edge-cloud coordination, and stream processing pipelines. Federated learning ensures anonymity, and Apache Kafka-based communications handle real-time data processing and ingestion. We evaluate the model's latency, throughput, and accuracy on numerous IoT datasets. Our results show that hybrid online learning methods with parallel processing improve system responsiveness and resource utilisation. A bar chart and a multi-line graph illustrate model performance and scalability. Performance matrices and comparison Tables confirm the approach's efficacy. This paper explains how to utilise machine learning to scale vertically and horizontally in IoT contexts, thereby driving smart infrastructure. We conclude by considering energy utilisation, data heterogeneity, and future research directions such as federated transfer learning, light neural networks, and quantum-aided ML for IoT contexts.
Keywords: Parallel Processing; Real-Time Processing; Big Data; Scalable Machine Learning; Edge Computing; Analytics Systems; Online Learning; Smart Infrastructure; Energy Usage.
Received: 13/06/2024, Revised: 25/07/2024, Accepted: 10/09/2024, Published: 03/06/2025
DOI: 10.64091/ATICL.2025.000146
AVE Trends in Intelligent Computer Letters, 2025 Vol. 1 No. 2 , Pages: 51-61