Menu

IoT FEATURE NEWS

The Role of In-Memory Computing and HTAP in High Performance IoT Architectures

By Special Guest
Nikita Ivanov, CTO and Co-founder, GridGain
December 27, 2016

By 2020, Gartner expects over 20 billion devices to be connected to the Internet of Things (IoT). According to the “Cisco Global Cloud Index: Forecast and Methodology, 2015–2020”, the total amount of data created by devices, driven by IoT, will reach 600 ZB per year by 2020, up from 145 ZB per year in 2015. To deliver on the promise of IoT, successful IoT platforms will need a data architecture that can deliver the performance and scale to deliver real-time insights on vast amounts of data. One of the most promising technologies to address these challenges is in-memory computing-based Hybrid Transactional and Analytical Processing (HTAP).

IoT Use Cases
Consider just a handful of the IoT use cases that demand fast data processing and the resulting real-time insights to drive actions:

  • Manufacturers are using sensors to track production, streamline industrial processes, and monitor inventory in industries ranging from food to logistics
  • In logistics, self-driving vehicles will produce and consume massive amounts of data while determining optimal routes and minimizing fuel usage
  • In transportation, both ride-sharing companies and taxi drivers are using apps to broadcast anonymous information about their travel, becoming part of a broader effort to implement intelligent traffic control
  • Healthcare organizations are implementing both hospital and residential-based patient monitoring, as well as environmental monitoring
  • Connected home products include security devices, thermostats, refrigerators, robots, and garden sensors

In most of these use cases, the analysis of IoT data must occur in real or near-real time to drive appropriate action. If analytics are not processed in time to prevent a patient emergency or food spoilage, then the benefits of IoT can be minimal.

In-Memory Hybrid Transactional/Analytical Processing for IoT
An infrastructure capable of the speed and scale required for IoT can be built on hybrid transactional and analytical processing solutions that use a single data store for transactions and analytics. Until recently, in-memory HTAP was expensive. Users were also typically concerned about the impact slow running queries could have on the ability to process transactions in real-time. Today, however, in-memory HTAP solutions are increasingly affordable thanks to reductions in the cost of memory. In addition, analytical processing, often based on streaming technology, greatly reduced concerns about analytics impacting operations. The simplified architecture of HTAP solutions also offers considerable cost savings potential compared to two side-by-side solutions – one for OLTP and one for OLAP.

There are three current approaches to IoT platforms: proprietary, cloud-based and open source. The distinctions across these solutions mirror the distinctions across other solution categories.

Proprietary
These solutions may be purpose-built and address a specific use case well but may come with high costs and result in vendor lock-in. Examples include GE Predix, the Bosch IoT Suite, or PTC ThingWorx.

Cloud-based
These solutions can be cost-effective and easy to implement for certain IoT applications. However, they can have rigid application and device integration interfaces and are unable to provide real-time analytics so are more commonly used for non-real-time IoT applications. Examples include AWS IoT and Azure IoT Suite.

Open Source
These solutions can be very flexible because the source code can be customized. They can also be cost effective and result in no vendor lock-in. Some of these solutions offer the in-memory capabilities necessary for real-time IoT applications. Examples include the SMACK stack (Apache® Kafka™, Mesos™, Akka™, Cassandra™ and Spark™) and Apache® Ignite™.

In-Memory Computing Platforms for IoT
The basic elements required for an in-memory HTAP solution for Internet of Things applications include:

  • An in-memory data grid for distributed data storage and scalability
  • An in-memory compute grid for high performance parallel processing of queries
  • In-memory streaming processing to allow analysis of incoming data in real-time
  • An in-memory SQL grid that provides in-memory database capabilities including support for SQL communications through ODBC/JDBC and may provide geospatial support for location-based IoT use cases

These interconnected modules can form a single, easily managed layer running in memory for the type of speed and scalability needed to deal with the torrent of data flowing from IoT devices. By enabling these core pieces of the IoT architecture to be implemented in an integrated, scalable, high performant, in-memory computing platform, IoT applications can manage the fast data processing required for next-generation IoT solutions.




Edited by Ken Briodagh


SHARE THIS ARTICLE
Related Articles

Zentera's Lee Emphasizes Importance of Shared Responsibility

By: Paula Bernier    12/15/2017

Zentera leader, IoT Evolution Expo speaker, discusses enclaves, IoT security, and the company's Internet of Things work with GM.

Read More

IoT Time Podcast S.2 Ep. 57 Unisys

By: Ken Briodagh    12/13/2017

In this episode of IoT Time, Ken Briodagh sits down with Bill Searcy, VP, Global Justice, Law Enforcement, and Border Security, Unisys (unisys.com/saf…

Read More

Canadian Municipalities Get Funding for 72 Infrastructure Improvements

By: Ken Briodagh    12/13/2017

The Canadian Infrastructure and Communities ministry and the Federation of Canadian Municipalities have announced funding for 72 Smart City initiative…

Read More

Hardware-Based IoT Security: Consider Your Options

By: Special Guest    12/13/2017

Risk vs. Reward - a tradeoff that factors into every business decision.

Read More

IoT and Cleantech Join Forces on Clean Energy

By: Special Guest    12/13/2017

The Internet of Things plays an important role in the adoption of clean technology and transforming operations and processes adjusted to the new envir…

Read More