Getting the Message About Big Data

I recently had a conversation with Jerry Coumo and Gari Singh of IBM that I am still in the process of parsing and delivering as information to you all. It was a very good discussion about a variety of subjects, but one in particular that I would like to highlight was the issue of big messaging.

Now, these days we have a lot of people talking about big data and various database strategies that take the blobs we use,  add as an append at the end of a table and makes the entire database a dynamic resource. It’s pretty cool to listen to the discussion because in effect, the database folks joined the Internet with this discussion. Of course they call it the cloud, but it is the Internet, at its root. 

So when Jerry and Gari started talking about analytics, I asked the question about whether M2M was a big data candidate. After all, much of the sensor data feeds live comfortably on 3G. Gari asked me the question, “Where does big data come from?” The answer is big messaging and in particular it points to MQTT.

Now this brought us to the fact that dynamics of front servers and backend systems were using JavaScript and things like Node.js and other JS solutions. However, what it really brought home was the balance between the raw data from the sensors and the back end decision making from the analytics.

This is where the network IO begins to matter greatly because as the analytics improve the data goes from event driven post analysis into predictive systems that basically want to be in the stream of data.

Now the big data side of the equation has a different kind of front end experience where instead of being just about the sensor stream, the analytics are becoming distributed and may be the dynamic for 4G/LTE to become necessary for the M2M data streams.

It also brought up a general question as to where a convergence between M2M and WebRTC might happen. Just like HTML5 and JavaScript where any kid with a browser can make a website, WebRTC promises that every website can have a talking/chatting capability.   However, when dealing with the complexity of communicating, if WebRTC is going to be interactive amongst sites, it’s going to end up using tools such as MQTT to manage the data. 

So I came away from the conversation understanding that the signaling is probably going to jump from the big data world to M2M and eventually voice. 

We have lots of solutions for messaging now that manage real-time effectively. Kaazing, PubHub and many others focus on real time communication. In the end the Internet of the Things is a common messaging platform and big data is the result.