Is Serverless a Good Fit for AI Applications Today?

Serverless functions allow developers to consume AI models on demand at a relatively lower cost without the complexities of setting up and managing the underlying infrastructure. But you may ask: Is Serverless a Good Fit for AI Applications? This blog post will decode that.

Why Serverless again ?

In a traditional server-based architecture, organizations bear the responsibility of provisioning, scaling, and maintaining the servers necessary to host their applications. This approach often requires significant upfront investment in hardware, ongoing management, and careful planning to handle traffic spikes and prevent downtime. As applications grow and demand fluctuates, the complexity of managing these servers can become a significant burden on development and operations teams.

Cloud-based Serverless functions

Serverless computing offers a transformative shift from this model by abstracting away the underlying infrastructure. Instead of managing servers, organizations can focus on writing code and deploying applications, while public cloud providers handle administrative tasks such as scaling, maintenance, and provisioning resources. Major cloud providers like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) offer robust serverless services like AWS Lambda, Azure Functions, and Google Cloud Functions, respectively. These services automatically scale with demand and charge only for the compute time used, eliminating the need for over-provisioning and reducing costs.

The rise of serverless computing is particularly significant in the context of modern AI and machine learning applications. Today, many pre-trained AI models like OpenAI and Google’s Gemini Pro run through APIs. Serverless architectures deploy these models with high availability and scalability, eliminating the complexity of managing infrastructure.

Furthermore, organizations can bring their own trained models and expose them using AI Gateways, which provide abstraction and simplify backend management. This approach enhances failure isolation and decentralizes the application architecture, making it more resilient and scalable.

Open-source Serverless functions

Beyond the major cloud providers, the serverless landscape includes powerful open-source options like Fission, OpenFaaS, and Knative. These open-source serverless frameworks can be deployed on Kubernetes clusters, leveraging Kubernetes’ inherent capabilities in container orchestration, auto-scaling, and resource management. By using these tools, organizations can achieve the flexibility of serverless computing while maintaining control over their infrastructure, a critical consideration for those with specific compliance or data sovereignty requirements.

OpenFaaS, for instance, simplifies the deployment of functions on Kubernetes, offering a rich ecosystem of integrations and an easy-to-use UI. Knative extends Kubernetes with built-in support for managing serverless workloads, providing features like automatic scaling based on demand, routing, and eventing. Fission offers a fast function as a service (FaaS) platform on Kubernetes, focusing on developer productivity by enabling quick deployments and a smooth development experience.

Apart from public cloud providers, you’ve got open-source choices like Fission, OpenFaaS, and Knative for serverless functions. These open-source serverless functions can be deployed on Kubernetes and take full advantage of Kubernetes’ inherent capabilities alongside their own prowess in auto-scaling and resource management.

The truth behind serverless and cost-saving

Serverless computing can be a cost-effective solution for businesses of all sizes, thanks to its “pay-as-you-go” (PAYG) model. Imagine if your organization is tired of dealing with fixed monthly fees for physical server upkeep or managing numerous virtual machines (VMs) in the cloud. With serverless functions, you only pay for the computing resources used when your code runs. This approach works well with microservices and event-driven architecture.

However, while serverless computing offers potential cost savings, it doesn’t always mean lower IT costs for every workload. It’s important to carefully evaluate your application’s specific needs and usage patterns to see if serverless will actually save you money. In some cases, certain workloads might end up costing more in a serverless environment.

Amazon Prime Video and Audio switched back to a monolithic architecture from microservices because the actual costs were much higher than expected. These costs included things like the computing power needed for the application, memory or storage for data, and data transfer fees for moving data in and out. By understanding these factors, you can better decide if a serverless approach is right for your needs and budget.

In their case, they restructured the workflow to reduce the high costs associated with data transfer within the process memory. Instead of using microservices, they switched to a container-based deployment where everything runs within the same instance, allowing them to scale vertically. This change led to a 90% reduction in infrastructure costs.

What workloads align best with serverless ?

Serverless architectures work well for workloads with varying usage patterns, where there are periods of high activity followed by times of low traffic. For these types of workloads, keeping a constantly running server may not be cost-effective.

Here’s a look at the types of workloads that align best with serverless computing:

Low and variable workloads

Serverless is ideal for applications with irregular traffic or low user demand. It’s cost-effective because you don’t pay for idle server time—serverless automatically scales down to zero when there’s no activity.

For example, a weather forecasting app might see traffic spikes during severe weather but stay quiet on regular days. With serverless, you can operate efficiently during low-traffic periods without incurring costs for unused server time.

High burst traffic

Serverless is excellent for handling sudden spikes in traffic. Traditional setups often require extra resources to handle peak loads, leading to higher costs even during normal usage. In contrast, serverless scales automatically to meet demand. An example of this is a ticketing website (e.g. billet reduct ). You see, a website like this one experiences a rush when event tickets go on sale. Serverless manages these peaks without the need for overprovisioning, saving money, and ensuring a smooth user experience.

Predictable workloads

For applications with steady and predictable usage, it may be more cost-effective to use provisioned infrastructure with reserved capacity, instead of serverless. For example, an enterprise HR system that sees consistent usage throughout the workday might benefit more from a traditional setup. This approach ensures consistent performance without the variability of serverless scaling.

Short-lived tasks 

For tasks that are quick and require minimal resources, serverless is the most cost-efficient choice. Provisioned servers may be more expensive due to minimum capacity requirements or specific billing structures. For instance, an image or file processing service that runs intermittently throughout the day benefits from serverless, as you only pay for the time the task is actually running, unlike provisioned servers that might have a fixed cost regardless of usage.

Understanding your workload type is key to making informed decisions. It’s not just about saving money but also about getting the best performance that meets your specific needs.

Understanding your workload type is important for making informed decisions. It’s not just about saving money but also about getting the best performance that suits exactly what you need.

Serverless has major drawbacks

While serverless computing has its benefits, there are also some downsides.

For tasks that need to run for long periods, serverless can become more expensive over time, making traditional or reserved infrastructure a cheaper option. Security and privacy are also important considerations. When using serverless, you’re handing over some of your data to another company, and the level of protection can vary. Plus, you’ll be sharing cloud resources with others, which can raise concerns. In fact, 60 percent of companies that haven’t used serverless yet worry about security and the unknowns of the cloud.

Monitoring is another challenge with serverless. If something goes wrong, it can be hard to figure out the problem. Fortunately, there are tools like Datadog and Dynatrace are designed to help with this.

A complete end-to-end monitoring solution quickly identifies and fixes issues. Long-running tasks may find serverless more expensive over time, making provisioned or reserved infrastructure a cheaper option.

Security and privacy are also things to consider. When you go serverless, you’re giving some of your data to another company, and how well it’s protected can vary. Also, you’ll be sharing resources in the cloud data center with others. Among companies that haven’t tried serverless, 60 percent worry about security and the unknown parts of the cloud.

To learn more about Serverless functions, I have summarized in the following video ( plus a hidden detail :)), so check it out if you’re interested :

The Truth about cost-saving

Serverless computing has many benefits for AI applications, especially because it removes the hassle of managing servers. This allows developers to focus on building and deploying AI models. One of the main advantages of serverless AI is its ability to automatically scale resources as needed. This is particularly useful when dealing with large or complex data, as serverless platforms can easily handle increased workloads without any manual adjustments.

Another benefit of serverless for AI is its cost-effectiveness. In traditional setups, you often pay for idle resources even when your AI application isn’t actively using them. With serverless, you only pay for the compute power and storage used during the execution of your AI tasks, which can save money, especially for businesses with varying workloads. Here are the full list of benefits and my thoughts about AI apps on Serverless Kubernetes.

However, it’s important to be aware of potential hidden costs when using serverless for AI. While serverless is generally cost-efficient, some AI tasks, like training large machine learning models, can be resource-intensive and may lead to higher costs than expected. In these cases, using dedicated or reserved infrastructure might be more economical.

AI applications often experience unpredictable usage patterns, leading to unexpected expenses in a serverless environment. Carefully assess your AI workloads and usage patterns to determine if serverless fits your needs. Sometimes, a hybrid approach combining serverless with other infrastructure options balances cost, performance, and scalability best.

Looking forward

Serverless computing can be a powerful and cost-efficient solution for AI applications. But I think it’s important to consider the specific needs and potential challenges of your AI projects. If you like to read stories like this, follow me (melonyqin here on Medium) and subscribe to my newsletter. See you in the next one! Thanks for your continued support, let’s stay tuned!