I have focused a lot – perhaps too much – on the connectivity side of IoT. But, it’s fair to say that, as we embrace the world of AIoT, connectivity is assumed to be there in one form or another. The cloud is a given and getting there is someone else’s problem (in most cases). I also should warn the readers that most of the article is full of direct or slightly revised text by the writers.
I naïvely thought serverless was going to mean a distributed system, and not the name of a dynamic server configuration capability for the hyperscalers. Note that, in this article, I quote heavily from two sources. One that celebrates the start of Function-as-a-Service (FaaS) and one that compares “traditional methodology to serverless compute.” It is not my intention to rain on anyone’s parade – merely to let you, the reader, have an understanding of the issues involved.
We also should not argue with success, as shown by the list below.
Amazon Web Services (AWS): AWS offers a wide array of serverless services that are foundational for AIoT. They include:
- AWS IoT Core: A managed cloud platform that lets connected devices easily and securely interact with cloud applications and other devices. It provides device connectivity, device management, and data management capabilities.
- AWS Lambda: A serverless compute service that lets you run code without provisioning or managing servers. It can be triggered by various AWS services, including AWS IoT Core, to process IoT data and execute AI algorithms.
- Amazon S3: A scalable object storage service for storing the large volumes of data generated by IoT devices.
- Amazon DynamoDB: A fully managed NoSQL database for high-performance storage and retrieval of IoT data.
- AWS IoT Analytics: A fully managed IoT analytics service that makes it easy to run sophisticated analytics on massive volumes of IoT data without having to build and manage your own analytics infrastructure.
- AWS SageMaker: A comprehensive machine learning service that provides tools for building, training, and deploying ML models. Its serverless inference options allow you to deploy AI models to process data from IoT devices without managing infrastructure.
- AWS Greengrass: Extends AWS cloud capabilities to edge devices so they can collect and analyze data locally while still leveraging the cloud for management, analytics, and durable storage. It enables local execution of Lambda functions and ML inference on edge devices.
Microsoft Azure: Azure provides a robust set of serverless services for building AIoT solutions, including:
- Azure IoT Hub: A managed service that enables reliable and secure bidirectional communication between IoT devices and the cloud. It offers device provisioning, management, and secure data ingestion.
- Azure Functions: A serverless compute service that allows you to run event-triggered code without managing infrastructure. It can be integrated with Azure IoT Hub to process data streams and execute AI logic.
- Azure Blob Storage: A scalable and secure object storage for storing IoT data.
- Azure Cosmos DB: A globally distributed, multi-model database service for managing IoT data at scale. Its serverless tier provides cost-effective storage for variable workloads.
- Azure IoT Analytics: A fully managed service for analyzing IoT data streams, allowing you to derive insights and build visualizations.
- Azure Machine Learning: A cloud-based environment for building, training, deploying, and managing machine learning models. Azure Machine Learning services can be used to develop AI models that run on data from IoT devices.
- Azure IoT Edge: Extends Azure IoT Hub and Azure Machine Learning services to edge devices, enabling local processing and AI inference. Azure Functions can also run on IoT Edge.
Google Cloud Platform (GCP): GCP offers a strong suite of serverless products for AIoT development:
- Google Cloud IoT Platform: A fully managed service that allows you to securely connect, manage, and ingest data from globally distributed IoT devices. It provides device management, data ingestion, and stream processing capabilities.
- Cloud Functions: A serverless execution environment for building and connecting cloud services. It can be triggered by events from Google Cloud IoT Platform and other GCP services to process IoT data and run AI algorithms.
- Google Cloud Storage: A scalable and durable object storage service for storing IoT data.
- Cloud Firestore: A NoSQL document database for storing and querying IoT data.
- Cloud IoT Core Analytics: Enables you to analyze IoT data streams in real time using SQL-like queries.
- Vertex AI: Google Cloud's unified AI platform, offering tools for the entire ML lifecycle, from data preparation to model deployment and monitoring. Serverless inference options are available for deploying AI models to process IoT data.
- Edge TPU: Google's purpose-built AI accelerator hardware for edge devices, enabling fast and efficient machine learning inference at the edge.
If I understand correctly, this all started with AWS Lambda back on November 13, 2014.
So, let’s do a little history, courtesy of AWS explaining how Lambda, which celebrated its ten-year anniversary in November, was developed.
AWS prides itself on getting primitives into the hands of their customers. In the hands of the customers, Function-as-a-Service (FaaS) gets used in ways they never expected. Almost always, the customers used Lambda to work around limitations of what’s available. Those workarounds led to new solutions to abstract away complexity.
AWS Lambda is a perfect example. We saw that customers had entire EC2 fleets sitting idle, waiting to run simple functions like writing to a database or processing a file. It was inefficient, and we knew we could do better. So, we challenged ourselves to build a service that allowed our customers to focus on their unique application logic, not on the undifferentiated heavy lifting of provisioning servers, autoscaling, patching software, and logging. And it started with a doc.
As an Amazonian, regardless of role or seniority, there is an expectation that when you have a good idea, you’ll put pen to paper and craft a compelling narrative. These docs take many forms: one-pagers, two-pagers, the infamous six-pager, and of course PR/FAQs – a press release followed by frequently asked questions we anticipate our customers will have, or that our internal stakeholders will (and often do) have. These press releases are internal tools, not shared broadly, designed to ensure we clearly define what we will deliver for customers. And while each of these docs has a different purpose, the rationale remains the same. Writing forces the author to be clear, precise, and detailed. To string sentences together, take a position, and support that position with data. It places the burden on the author to avoid anything confusing or that could be misinterpreted by the reader. It’s hard work. I’ve never seen someone get it right the first time. It takes collecting feedback and revising and then revising again. But a good idea backed by a crisp doc has proven it can produce wonderful products – it’s something we’ve seen over-and-over again from 1-click buying and Amazon Prime to the launch of AWS and Kindle. So today, on Lambda’s tenth anniversary, I’d like to share the PR/FAQ that launched one of AWS’s foundational services. It’s been slightly modified for length, repetition, and readability, but it provides a peek behind the curtain at the customer problems we were observing in the early 2010s, and our vision for serverless compute.
Lambda, driven by the needs of our customers, has evolved quite a bit in the last decade. When you read this PR/FAQ, you’ll notice some things that have changed (e.g., 1ms billing granularity and support for functions with 10 GB of memory), and a handful of ideas that eventually turned into full-blown features (which I’ve annotated in the text below). It’s a reminder that while we try very hard to be thorough, we are not omniscient, and the goal of writing is not perfection. The goal is to get products into the hands of our customers, watch how they use them, make adjustments, and innovate along the way.
Marc Brooker put it quite brilliantly on a recent episode of the AWS Developer Podcast. I’m paraphrasing here, but the idea is that customers have always come up with unexpected use cases for services like Lambda – things that didn’t make it into a PR/FAQ. Engineers love this, because it challenges them to innovate and think bigger and more broadly for customers. And it’s the driving force that has led to a range of innovations, such as SnapStart, FireCracker, and containers for Lambda functions – the constant pursuit to make building easier for anyone using AWS.
The Lambda PR/FAQ is available here with annotations. You can also view the PR/FAQ without annotations as a PDF.
So that’s the history of AWS Lambda but, after ten years, these serverless solutions are not without concerns, which developers need to be aware of when it comes to where and when of using serverless compute.
Kostis Kapelonis is a software engineer with 12 years of experience with a special focus on GitOps and deployment automation.
Kostis points out some of the issues he sees using serverless computing which, in his opinion has been hyped as the ultimate solution to scalability and elasticity. However, many who adopted it were quickly disappointed by rising costs and maintenance difficulties. There may be no servers, but we still need a major effort to deploy and test workloads.
Serverless is a specific tool, suitable only for very specific use cases.
The initial promises of serverless were undoubtedly enticing for any developer who deals with cloud-native applications. No more server patching, no more complicated deployments. Just deploy your application and see it running in minutes.
Over- (and under-) provisioning is now a thing of the past. Your serverless application will automatically scale up and down according to workload.
Serverless technology was also tightly integrated into CI/CD solutions. How can anybody say no to this new, magic technology? On the other hand, your costs will also go down, as you essentially pay only for what your application is using.
Serverless is the perfect paradigm. Developers can move fast and focus only on business logic code, while managers and chief financial officers can enjoy the lower costs compared to traditional cloud methodologies.
Of course, the reality is much different. Several teams jumped on the serverless bandwagon without ever understanding the implications of their choice.
There is no silver bullet. Every seasoned engineer knows that all choices come down to trade-offs and serverless is no different.
When Is Serverless Computing the Right Choice?
Kostis suggests you ask these questions before using serverless solutions.
- Is your application asynchronous, with many small individual components that can be tested in isolation?
- Does traffic come and go with non-predictable timings?
- Is your virtual machine just sitting there doing nothing on the weekends?
If these are true serverless compute is something to explore.
Serverless Only Solves a Few Problems for Developers
The main selling point of serverless for developers (after the scalability benefits) is the ease of deployment and lack of maintenance. Just wrap your business code in a function and deploy it in less than a minute. No need to create a server; install dependencies and deploy your code anymore.
This might be true, especially in the case of Function-as-a-Service providers, but suffers from a common fallacy. Server management is not what blocks developers from shipping fast. More precisely, it’s not the only thing that blocks them.
In any non-trivial application, teams have several other concerns apart from servers. Developers need to manage their database schemas, the migration changesets, the integration tests between several components, the security vulnerabilities of all dependencies, the communication protocols with other external services and so on.
So, what does serverless offer on these tasks? Absolutely nothing. You’re still on your own. All these concerns are still there even if you adopt serverless.
Seasoned developers know that what matters most is your data (and by extension your database/queue). Mutating your database schema and making sure that you have enough data for auditing is a constant struggle. Losing valuable data will instantly make your customers angry.
So yes, serverless might help you in some scenarios with your computing needs and your business logic, but everything else stays the same.
Serverless is like promising architects that this new paint will dry up in two minutes instead of two hours. Even if this is true, is waiting for paint to dry the main problem that architects have today?
Constraints Introduced by Adopting Serverless
Building a serverless application has a lot of limitations.
You need to use an approved programming language. You need to communicate with the other serverless components of your cloud provider. You need to account for the cold start issues. You must make sure that your code works within a specific time limit. You need to manage context and save data in a different way.
All these limitations can make application development difficult and cumbersome. Ideally, developer teams that adopt serverless should already have experience with microservices and know how to write asynchronous services with proper network calls. Writing a serverless application changes completely what developers know about the running context and how to handle network failures or network errors.
Adopting serverless implies that your team needs training on how to write asynchronous components.
Debugging serverless applications is a nightmare compared to traditional methods. Running a serverless application locally (all of its components) is next to impossible. Running integration tests against a group of components locally is challenging and time consuming. Quickly locating an issue in a serverless application is more difficult than a monolith. You’ll spend the time you saved from not managing servers on debugging your serverless applications.
When a production application has an issue, developer teams always follow a familiar sequence of steps. First, the team needs to find where the problem is. Then, they need to code a solution that fixes the problem. Finally, they need to deploy the fix.
Serverless applications make the third step easier, but the first two steps are much more difficult. Knowing that you can deploy your code quickly is not a big benefit if it takes too much time to understand exactly where the problem is in all your asynchronous components and network call stacks.
Do You Really Save Costs with Serverless?
If you read most of the tutorials about serverless applications, you might get the impression that all services in the world perform just two tasks: image resizing and video/audio encoding. There’s a reason serverless proponents always use these examples. The benefits of serverless are only evident if your application follows some very specific patterns.
Several real-world applications don’t follow this kind of behavior. And for those, having a traditional virtual machine or container is a better solution. In fact, there’s the danger of actually paying more money if you port this kind of application to a serverless architecture.
Instant scalability comes with great responsibility. Several cloud vendors don’t offer by default any hard limits on how much capability your application uses. The benefits of serverless scalability are only true provided you have the budget to actually cover them.
Some of Us Can Go Back to Traditional Methods
So, now that the hype is over, what should you use instead? This is a tricky question, because the answer assumes that developers need to use only one development methodology. This could not be further from the truth: developers can mix and match according to the type of the application.
Serverless can be just another application architecture, you can still use the others. Developers should add serverless in their toolchest like any other tool. Understanding when to use serverless is equally important to understanding how to use serverless.
If your application matches certain requirements (bursty traffic, downtime between requests, asynchronous components), then serverless might be a perfect solution. But traditional development methods will be much easier (and more cost effective) in several other cases.
The big advantage of traditional applications is predictability in all fronts. A monolithic application is very easy to launch locally on a developer laptop. You can monitor it and test it as a whole without any specialized tools. Function calls within a monolith are perfect. They have zero latency, 100 percent reliability and easy traceability. Debugging a monolithic application is much faster than a serverless one and there are already several specialized debugging tools that allow you to understand what, where and how an issue appeared.
Developers are always more comfortable with synchronous applications. It is very easy to find the order of events and explain why something happened if there is a predictable order in all function calls and memory requests.
On the matter of cost, monoliths are better when you have a constant and predictable load. More importantly you can reason about your costs by simply multiplying the time a virtual machine is up with cost per minute of your cloud provider. It is very easy to lock this down (do not use VM autoscaling) or even change it on demand (preload some VM’s in advance of a big event). You have total control of the costs.
Conclusion: As AIoT continues to be deployed in Edge Servers, we may see different pricing models that will level the issues of cost. It may also lead to better diagnostics for serverless functions. Time will tell.
Edited by
Erik Linask
