Why a NoSQL Database Strategy was the Right Choice for This IoT Business

By Special Guest
Gena Minevich, VP, Research and Development, Freshub
May 15, 2017

IoT solutions require sophisticated capabilities to gather data from myriad sensors and transmitters, and to efficiently utilize that data to empower users with unprecedented convenience. However, as vast, powerful, and impactful as IoT applications can be, they demand equally strong command over their boundless stores of data to follow through on that promise.

At issue is the need to effectively harness huge quantities of data in order to deliver a high performance user experience, while also meeting the availability and scalability needs inherent with IoT applications. Addressing this issue means thoughtfully selecting and configuring the database used to handle IoT solutions’ unique appetite for data.

This is an issue to which we can speak from experience. At Freshub, we set out to build an IoT Smart Kitchen Commerce solution linking grocery retailers and connected appliance manufacturers to the consumer’s kitchen. The product helps make the IoT-driven smart kitchen a reality, transforming any kitchen into a smart kitchen and any appliance into a shopping touch-point. This solution called for creating and maintaining a database of over 1 million grocery products gathered through a detailed, granular-level mapping of online grocery product catalogs. Overall, the capability of the database to harness this data in real-time, and give users a satisfying experience, was make-or-break for the product.

In our case, we initially used MySQL for our database and managed it internally, which put us off to a challenging start simply because MySQL was a chore to manage for our needs. For our team, performing MySQL replication was a rather difficult and unpleasant task, especially when it broke and needed to be recreated. This was the reason we decided to switch from MySQL to a NoSQL database (in order to make the system administration much easier), and selected MongoDB as the best fit for that. We then used mLab as our MongoDB Database-as-a-Service provider to ensure the transition would be much more efficient, leaving our internal team free to invest their limited time purely on product development.

I want to be clear, however: NoSQL isn’t necessarily a better fit than MySQL for all IoT solutions. Each technology is good for the purpose it was designed to serve. In fact, there are many projects where a combination of both might be optimal.

That said, NoSQL was the more appropriate choice for our circumstances. For businesses like Freshub that work with multiple data providers using different and unpredictable data schemas, NoSQL can be much easier to use. Whereas MySQL requires schemas to be defined before data is added, NoSQL allows for de-normalized documents that enable faster and simpler integration with data no matter its structure. At the same time, developing a data model without worrying about schema is a more pleasant experience (at least in my opinion). If developers need to insert a new field they can simply add it to the class, or remove a field via quick deletion – no large-scale efforts or thought is needed.

NoSQL also offers advantages in scaling when handling high volumes of IoT data is required, which can be accomplished simply and without performance bottlenecks. For example, NoSQL allows developers to have as many sharded database nodes as needed, so a single server can always be a limited, pre-defined size and scale can still be achieved with ease. These are appealing features in preparation for the full-fledged IoT that’s quickly becoming a reality. It’s especially so in our case where connected kitchens will be awash with information delivered by sensors and transmitters embedded in all kinds of products – big data that our database needs to be fully prepared to handle.

On a practical level, NoSQL leverages the fact that it’s easier to find good Java or C# programmers than good database programmers, and NoSQL moves a lot of logic from the database to the application. For example, any programmer knows how to write a “select” query, but few are experts at fine-tuning performance for complex queries. Experiences may vary depending on the strengths of particular teams, but NoSQL can certainly make it simpler to assemble a highly-capable team that’s right for the tasks at hand. It should also be acknowledged that the industry is quickly shifting towards NoSQL, which should mean that the tools, frameworks, and knowledge available for getting the most out of it will only become more available and efficient.

This isn’t to say NoSQL is completely issue-free, but it does offer a lot to like. As our case demonstrates, MongoDB’s non-relational database structure serves the needs of IoT applications well by delivering the performance, ability, and scalability these applications require. Those networks of sensors and transmitters that enrich IoT solutions with data must have high-speed read and write database performance in order to deliver any functionality that utilizes data without delays. The always-on nature of IoT applications also means that the high-availability of data – reliably provided by MongoDB via mLab – is a necessity, since any downtime in the database effectively makes the application non-operational. At the same time, MongoDB provides the scalability required to handle the ever-growing amount of data that the IoT collects.

As we’ve learned, selecting the right database is essential to the user experience and, ultimately, the success of any IoT application. For those dealing with smaller and more static datasets, MySQL may be the superior option. In our case (dealing with large, dynamic, non-uniform datasets), NoSQL MongoDB better enabled us to deliver our IoT solution – now and into the future.

Gena Minevich is VP of Research and Development at Freshub, an IoT-based Smart Kitchen Commerce solution.

Edited by Ken Briodagh

Related Articles

In Digital Transformation, Operational Excellence is the Real Prize

By: Special Guest    4/24/2018

Many companies approach Digital Transformation by 'sensorizing' all their equipment, collecting data and hoping it turns out insights on what to impro…

Read More

A Year into Solving Challenges at the Edge of the IoT, EdgeX Foundry Celebrates

By: Cynthia S. Artin    4/23/2018

We've been tracking the progress of The Linux Foundation's EdgeX Foundry since they launched the community in 2017 and are following them again to the…

Read More

Concept3D Announces OSIsoft PI System Integration

By: Ken Briodagh    4/19/2018

Concept3D has announced that it has become an OSIsoft partner with the new capability of integrating and displaying operational data from OSIsoft's PI…

Read More

Microsoft and ClearSky Join Forces for File Transport Technology

By: Chrissie Cluney    4/17/2018

Microsoft Corp. and ClearSky Data have entered into a licensing agreement to enable access to Microsoft's Server Message Block (SMB) file transport te…

Read More

Telit Enables Applications and US Expansion for Otodata

By: Ken Briodagh    4/16/2018

Telit, a global enabler of the Internet of Things (IoT), announced a strategic partnership with Otodata, Canada's largest IoT network operator, with m…

Read More