IoT solutions require sophisticated capabilities to gather data from myriad sensors and transmitters, and to efficiently utilize that data to empower users with unprecedented convenience. However, as vast, powerful, and impactful as IoT applications can be, they demand equally strong command over their boundless stores of data to follow through on that promise.
At issue is the need to effectively harness huge quantities of data in order to deliver a high performance user experience, while also meeting the availability and scalability needs inherent with IoT applications. Addressing this issue means thoughtfully selecting and configuring the database used to handle IoT solutions’ unique appetite for data.
This is an issue to which we can speak from experience. At Freshub, we set out to build an IoT Smart Kitchen Commerce solution linking grocery retailers and connected appliance manufacturers to the consumer’s kitchen. The product helps make the IoT-driven smart kitchen a reality, transforming any kitchen into a smart kitchen and any appliance into a shopping touch-point. This solution called for creating and maintaining a database of over 1 million grocery products gathered through a detailed, granular-level mapping of online grocery product catalogs. Overall, the capability of the database to harness this data in real-time, and give users a satisfying experience, was make-or-break for the product.
In our case, we initially used MySQL for our database and managed it internally, which put us off to a challenging start simply because MySQL was a chore to manage for our needs. For our team, performing MySQL replication was a rather difficult and unpleasant task, especially when it broke and needed to be recreated. This was the reason we decided to switch from MySQL to a NoSQL database (in order to make the system administration much easier), and selected MongoDB as the best fit for that. We then used mLab as our MongoDB Database-as-a-Service provider to ensure the transition would be much more efficient, leaving our internal team free to invest their limited time purely on product development.
I want to be clear, however: NoSQL isn’t necessarily a better fit than MySQL for all IoT solutions. Each technology is good for the purpose it was designed to serve. In fact, there are many projects where a combination of both might be optimal.
That said, NoSQL was the more appropriate choice for our circumstances. For businesses like Freshub that work with multiple data providers using different and unpredictable data schemas, NoSQL can be much easier to use. Whereas MySQL requires schemas to be defined before data is added, NoSQL allows for de-normalized documents that enable faster and simpler integration with data no matter its structure. At the same time, developing a data model without worrying about schema is a more pleasant experience (at least in my opinion). If developers need to insert a new field they can simply add it to the class, or remove a field via quick deletion – no large-scale efforts or thought is needed.
NoSQL also offers advantages in scaling when handling high volumes of IoT data is required, which can be accomplished simply and without performance bottlenecks. For example, NoSQL allows developers to have as many sharded database nodes as needed, so a single server can always be a limited, pre-defined size and scale can still be achieved with ease. These are appealing features in preparation for the full-fledged IoT that’s quickly becoming a reality. It’s especially so in our case where connected kitchens will be awash with information delivered by sensors and transmitters embedded in all kinds of products – big data that our database needs to be fully prepared to handle.
On a practical level, NoSQL leverages the fact that it’s easier to find good Java or C# programmers than good database programmers, and NoSQL moves a lot of logic from the database to the application. For example, any programmer knows how to write a “select” query, but few are experts at fine-tuning performance for complex queries. Experiences may vary depending on the strengths of particular teams, but NoSQL can certainly make it simpler to assemble a highly-capable team that’s right for the tasks at hand. It should also be acknowledged that the industry is quickly shifting towards NoSQL, which should mean that the tools, frameworks, and knowledge available for getting the most out of it will only become more available and efficient.
This isn’t to say NoSQL is completely issue-free, but it does offer a lot to like. As our case demonstrates, MongoDB’s non-relational database structure serves the needs of IoT applications well by delivering the performance, ability, and scalability these applications require. Those networks of sensors and transmitters that enrich IoT solutions with data must have high-speed read and write database performance in order to deliver any functionality that utilizes data without delays. The always-on nature of IoT applications also means that the high-availability of data – reliably provided by MongoDB via mLab – is a necessity, since any downtime in the database effectively makes the application non-operational. At the same time, MongoDB provides the scalability required to handle the ever-growing amount of data that the IoT collects.
As we’ve learned, selecting the right database is essential to the user experience and, ultimately, the success of any IoT application. For those dealing with smaller and more static datasets, MySQL may be the superior option. In our case (dealing with large, dynamic, non-uniform datasets), NoSQL MongoDB better enabled us to deliver our IoT solution – now and into the future.
Gena Minevich is VP of Research and Development at Freshub, an IoT-based Smart Kitchen Commerce solution.
Edited by
Ken Briodagh
