The newly introduced cloud.HttpServer in Pulumi makes it easy to serve a standard Node.js HTTP server as a serverless API on any cloud platform. This new API brings together the flexibility and rich ecosystem of Node.js HTTP servers, the cost and operational simplicity of serverless APIs, and the multi-cloud authoring and deployment of Pulumi. In this post, we walk through some of the background on why we introduced this new API and how it fits into the Node.js HTTP ecosystem.
Just over a month ago we publicly launched Pulumi, a new cloud native development platform. The response has been overwhelming and we’ve been hard at work responding to your feedback ever since.
Today, we are excited to release Pulumi 0.15 and make it available to download. This release includes improvements across the entire Pulumi development experience. Pulumi supports more platforms (Kubernetes and OpenStack), is faster (Parallelism), simpler (native TypeScript support), richer (serverless frameworks for Azure and GCP), and is more deeply integrated into the application lifecycle (GitHub App for CI/CD integration).
In this post, we’ll look at 12 “pearls” – bite-sized code snippets – that demonstrate some fun ways you can program the cloud using Pulumi. In my introductory post, I mentioned a few of my “favorite things”. Now let’s dive into a few specifics, from multi-cloud to cloud-specific, spanning containers, serverless, and infrastructure, and generally highlighting why using real languages is so empowering for cloud scenarios. Since Pulumi lets you do infrastructure-as-code from the lowest-level to the highest, we will cover a lot of interesting ground in short order.
At Pulumi, we do all of our development on GitHub, with a workflow built around topic branches. When a developer wants to make a change, they push a branch to GitHub, open a pull request and (in theory) once it’s merged, delete the branch. In practice, we’ll often forget to delete the topic branch (I’m probably the worst offender), which means we end up having topic branches linger on our main repository until they are explicitly cleaned up. While it’s a lot of fun to go a click through the GitHub UI from time to time, deleting merged branches, it’s even more fun to build automation to do this for us. Since GitHub has a rich set of webhooks and Pulumi makes it easy to write serverless functions, it felt like it would be natural to use Pulumi to write a hook that would clean up branches after a pull request got merged. In addition, Pulumi lets us leverage real programming languages to build abstractions, which means we can build a simple framework that hides much of the ceremony behind defining a hook and lets us focus on the core logic of our hook, without worrying about how it is deployed and managed.
Two weeks ago Amazon added Simple Queue Service (SQS) as a supported event source for Lambda. SQS is one of AWS’s oldest services, providing access to a powerful message queue that can do things like guarantee messages will be delivered at least once, or messages that will be processed in the same order they were received in. Adding SQS as a supported event source for Lambda means that now it’s possible to use SQS in a serverless computing infrastructure, where Lambdas are triggered in response to messages added to your SQS queue. Now, instead of needing some sort of Service dedicated to polling your SQS queue, or creating Simple Notification Service (SNS) notifications from your messages, you can instead just directly trigger whatever Lambda you want.
- write code just like an Express app... but end up with a fully deployable serverless app
- lambdas are... just lambdas
- no YAML required... freedom from indentation
- all the features of the V8 runtime... async await ahoy
- all the behaviors of immutable infrastructure as code tools... but we really mean 'as code'
Pulumi also supports containers (including Kubernetes), managed services, infrastructure and everything else in between that you might need for building cloud applications. Better than that, you can even combine them all in the same program.
Here at Pulumi we are (perhaps unsurprisingly!) huge fans of using Pulumi to manage our cloud infrastructure and services. We author our infrastructure in strongly-typed programming languages, which allows us to to benefit from rich tooling - documenting and factoring our infrastructure using the same software engineering practices we apply to our application code. This also allows us to create reusable abstractions which accelerate our ability to deliver new features and services, and our ability to standardize and refactor infrastructure patterns across our services with relative ease.
Pulumi makes it easy to build cloud applications that use a combination of containers, lambdas, and connected data services and infrastructure: Colada apps.
An example of a Colada app is extracting a thumbnail from a video. A serverless function can only run for 5 minutes, so we'll run a container in AWS Fargate to do the video processing.
In this app, a Lambda function is triggered whenever a new video is uploaded to S3. This function launches a task in Fargate that uses FFmpeg to extract a video thumbnail. A second Lambda function is triggered when a new thumbnail has been created.