Creating a Data Fabric for IoT

The move to data-driven decisions is dominating every industry, with significant implications for data processing. The emergence of IoT and edge computing promises to add a new dimension to data-driven insights, but has also added additional considerations for modern data processing. This has created the need for new approaches and technologies that can bring IoT data and edge computing into the data processing fold. Such an approach need to satisfy the following: Capture data across edge devices and intelligently transform it so that it can be used at the edge, in the cloud and in the data center Manage and facilitate the flow of data from the edge to the data center or cloud to support real-time processing Trigger data processing applications as soon as data arrives at the edge, in the data center and in the cloud Process data centrally and push intelligence to the edge Provide support to quickly react to insights to mitigate risk, reduce costs and ensure security Make it easy to deploy and manage in an increasingly complex IT environment In this talk, we will talk about how to build an IoT data fabric using Apache Pulsar and how to satisfy all these requirements that works both in the edge and data center. It provides a small footprint and It can run at the edge where it captures, filters and transforms data and then pushes data to data center or cloud. Another instance of Apache Pulsar can run in the data center captures this data from several edge devices, processes them and store them indefinitely for post analysis. It provides sophisticated analytical capabilities using Pulsar Functions and SQL to process the data as it flows.