Menu

IoT FEATURE NEWS

The Role of In-Memory Computing and HTAP in High Performance IoT Architectures

By Special Guest
Nikita Ivanov, CTO and Co-founder, GridGain
December 27, 2016

By 2020, Gartner expects over 20 billion devices to be connected to the Internet of Things (IoT). According to the “Cisco Global Cloud Index: Forecast and Methodology, 2015–2020”, the total amount of data created by devices, driven by IoT, will reach 600 ZB per year by 2020, up from 145 ZB per year in 2015. To deliver on the promise of IoT, successful IoT platforms will need a data architecture that can deliver the performance and scale to deliver real-time insights on vast amounts of data. One of the most promising technologies to address these challenges is in-memory computing-based Hybrid Transactional and Analytical Processing (HTAP).

IoT Use Cases
Consider just a handful of the IoT use cases that demand fast data processing and the resulting real-time insights to drive actions:

  • Manufacturers are using sensors to track production, streamline industrial processes, and monitor inventory in industries ranging from food to logistics
  • In logistics, self-driving vehicles will produce and consume massive amounts of data while determining optimal routes and minimizing fuel usage
  • In transportation, both ride-sharing companies and taxi drivers are using apps to broadcast anonymous information about their travel, becoming part of a broader effort to implement intelligent traffic control
  • Healthcare organizations are implementing both hospital and residential-based patient monitoring, as well as environmental monitoring
  • Connected home products include security devices, thermostats, refrigerators, robots, and garden sensors

In most of these use cases, the analysis of IoT data must occur in real or near-real time to drive appropriate action. If analytics are not processed in time to prevent a patient emergency or food spoilage, then the benefits of IoT can be minimal.

In-Memory Hybrid Transactional/Analytical Processing for IoT
An infrastructure capable of the speed and scale required for IoT can be built on hybrid transactional and analytical processing solutions that use a single data store for transactions and analytics. Until recently, in-memory HTAP was expensive. Users were also typically concerned about the impact slow running queries could have on the ability to process transactions in real-time. Today, however, in-memory HTAP solutions are increasingly affordable thanks to reductions in the cost of memory. In addition, analytical processing, often based on streaming technology, greatly reduced concerns about analytics impacting operations. The simplified architecture of HTAP solutions also offers considerable cost savings potential compared to two side-by-side solutions – one for OLTP and one for OLAP.

There are three current approaches to IoT platforms: proprietary, cloud-based and open source. The distinctions across these solutions mirror the distinctions across other solution categories.

Proprietary
These solutions may be purpose-built and address a specific use case well but may come with high costs and result in vendor lock-in. Examples include GE Predix, the Bosch IoT Suite, or PTC ThingWorx.

Cloud-based
These solutions can be cost-effective and easy to implement for certain IoT applications. However, they can have rigid application and device integration interfaces and are unable to provide real-time analytics so are more commonly used for non-real-time IoT applications. Examples include AWS IoT and Azure IoT Suite.

Open Source
These solutions can be very flexible because the source code can be customized. They can also be cost effective and result in no vendor lock-in. Some of these solutions offer the in-memory capabilities necessary for real-time IoT applications. Examples include the SMACK stack (Apache® Kafka™, Mesos™, Akka™, Cassandra™ and Spark™) and Apache® Ignite™.

In-Memory Computing Platforms for IoT
The basic elements required for an in-memory HTAP solution for Internet of Things applications include:

  • An in-memory data grid for distributed data storage and scalability
  • An in-memory compute grid for high performance parallel processing of queries
  • In-memory streaming processing to allow analysis of incoming data in real-time
  • An in-memory SQL grid that provides in-memory database capabilities including support for SQL communications through ODBC/JDBC and may provide geospatial support for location-based IoT use cases

These interconnected modules can form a single, easily managed layer running in memory for the type of speed and scalability needed to deal with the torrent of data flowing from IoT devices. By enabling these core pieces of the IoT architecture to be implemented in an integrated, scalable, high performant, in-memory computing platform, IoT applications can manage the fast data processing required for next-generation IoT solutions.




Edited by Ken Briodagh


SHARE THIS ARTICLE
Related Articles

Qualcomm, China Mobile Research Institute and Mobike to Commence LTE IoT Multimode Field Trials

By: Ken Briodagh    5/23/2017

According to a recent announcement, Qualcomm plans to commence the first eMTC/NB-IoT/GSM (LTE Cat M1/NB1 and E-GPRS) multimode field trials in China, …

Read More

New Cisco Survey Indicates That Most IoT Projects Are Failing

By: Ken Briodagh    5/23/2017

A new study conducted by Cisco seems to indicate that 60 percent of IoT initiatives stall at the Proof of Concept (PoC) stage and only 26 percent of c…

Read More

IoT Time Podcast S.2 Ep.28 DMI

By: Ken Briodagh    5/23/2017

On this episode of the IoT Time Podcast, Ken Briodagh sits down with Magnus Jern, Chief Innovation Officer, DMI.

Read More

Qualcomm Offers $11.0 Billion in Senior Notes to Finance NXP Acquisition

By: Ken Briodagh    5/22/2017

Qualcomm Incorporated has announced that it has priced a public offering of senior unsecured notes in a combined aggregate principal amount of $11.0 b…

Read More

5G Might Not Be The Next Big Thing for IoT, Study Says

By: Ken Briodagh    5/22/2017

Mobile Experts releases network and handset cost estimates for 5G, illustrates the weaknesses of 5G, especially for IoT.

Read More