Menu

IoT FEATURE NEWS

The Role of In-Memory Computing and HTAP in High Performance IoT Architectures

By Special Guest
Nikita Ivanov, CTO and Co-founder, GridGain
December 27, 2016

By 2020, Gartner expects over 20 billion devices to be connected to the Internet of Things (IoT). According to the “Cisco Global Cloud Index: Forecast and Methodology, 2015–2020”, the total amount of data created by devices, driven by IoT, will reach 600 ZB per year by 2020, up from 145 ZB per year in 2015. To deliver on the promise of IoT, successful IoT platforms will need a data architecture that can deliver the performance and scale to deliver real-time insights on vast amounts of data. One of the most promising technologies to address these challenges is in-memory computing-based Hybrid Transactional and Analytical Processing (HTAP).

IoT Use Cases
Consider just a handful of the IoT use cases that demand fast data processing and the resulting real-time insights to drive actions:

  • Manufacturers are using sensors to track production, streamline industrial processes, and monitor inventory in industries ranging from food to logistics
  • In logistics, self-driving vehicles will produce and consume massive amounts of data while determining optimal routes and minimizing fuel usage
  • In transportation, both ride-sharing companies and taxi drivers are using apps to broadcast anonymous information about their travel, becoming part of a broader effort to implement intelligent traffic control
  • Healthcare organizations are implementing both hospital and residential-based patient monitoring, as well as environmental monitoring
  • Connected home products include security devices, thermostats, refrigerators, robots, and garden sensors

In most of these use cases, the analysis of IoT data must occur in real or near-real time to drive appropriate action. If analytics are not processed in time to prevent a patient emergency or food spoilage, then the benefits of IoT can be minimal.

In-Memory Hybrid Transactional/Analytical Processing for IoT
An infrastructure capable of the speed and scale required for IoT can be built on hybrid transactional and analytical processing solutions that use a single data store for transactions and analytics. Until recently, in-memory HTAP was expensive. Users were also typically concerned about the impact slow running queries could have on the ability to process transactions in real-time. Today, however, in-memory HTAP solutions are increasingly affordable thanks to reductions in the cost of memory. In addition, analytical processing, often based on streaming technology, greatly reduced concerns about analytics impacting operations. The simplified architecture of HTAP solutions also offers considerable cost savings potential compared to two side-by-side solutions – one for OLTP and one for OLAP.

There are three current approaches to IoT platforms: proprietary, cloud-based and open source. The distinctions across these solutions mirror the distinctions across other solution categories.

Proprietary
These solutions may be purpose-built and address a specific use case well but may come with high costs and result in vendor lock-in. Examples include GE Predix, the Bosch IoT Suite, or PTC ThingWorx.

Cloud-based
These solutions can be cost-effective and easy to implement for certain IoT applications. However, they can have rigid application and device integration interfaces and are unable to provide real-time analytics so are more commonly used for non-real-time IoT applications. Examples include AWS IoT and Azure IoT Suite.

Open Source
These solutions can be very flexible because the source code can be customized. They can also be cost effective and result in no vendor lock-in. Some of these solutions offer the in-memory capabilities necessary for real-time IoT applications. Examples include the SMACK stack (Apache® Kafka™, Mesos™, Akka™, Cassandra™ and Spark™) and Apache® Ignite™.

In-Memory Computing Platforms for IoT
The basic elements required for an in-memory HTAP solution for Internet of Things applications include:

  • An in-memory data grid for distributed data storage and scalability
  • An in-memory compute grid for high performance parallel processing of queries
  • In-memory streaming processing to allow analysis of incoming data in real-time
  • An in-memory SQL grid that provides in-memory database capabilities including support for SQL communications through ODBC/JDBC and may provide geospatial support for location-based IoT use cases

These interconnected modules can form a single, easily managed layer running in memory for the type of speed and scalability needed to deal with the torrent of data flowing from IoT devices. By enabling these core pieces of the IoT architecture to be implemented in an integrated, scalable, high performant, in-memory computing platform, IoT applications can manage the fast data processing required for next-generation IoT solutions.




Edited by Ken Briodagh


SHARE THIS ARTICLE
Related Articles

Automobility LA Announces Theme for 2017 Design & Developer Challenge Presented By Microsoft

By: Ken Briodagh    8/17/2017

The Competition Requires Participants to Think About the Future and Develop a Transport/Mobility Concept for the City of Los Angeles' Bid to Host the …

Read More

Centerbridge Completes $1.26 Billion Acquisition of Syncsort and Vision Solutions

By: Ken Briodagh    8/17/2017

Syncsort and Vision Solutions Combining to Create a Powerhouse 'Big Iron to Big Data' Platform; Syncsort CEO Josh Rogers to Lead Company Operating Und…

Read More

Deloitte Selects Miami Based Unified Technologies for Caribbean Cyber Security Alliance

By: Ken Briodagh    8/16/2017

Unified Technologies, an IT solutions provider with operations in the Caribbean and North America, has announced a newly established Cyber Security Al…

Read More

InfoSec Veteran Mike Ahmadi to join DigiCert as Global Director of IoT Security

By: Ken Briodagh    8/16/2017

Ahmadi brings decades of leadership and advocacy in critical infrastructure security, including active participation in several standards creation gro…

Read More

IoT Time Podcast S.2 Ep.43 Nokia

By: Ken Briodagh    8/16/2017

On this episode of the IoT Time Podcast, Ken Briodagh sits down with Khamis Abulgubein, PLM, Emerging IoT Applications at Nokia.

Read More