Why the Amex Armegeddon is a Blessing in Disguise during Digital Transformation.
By Jeroen Mulder - mei 10, 2022
(And a fantastic driver for implementing a #FinOps practice)
Most cloud consultants and architects will recognize this. Well, I hope they recognize this. We all have our own accounts in one or more public clouds such as AWS, Azure and Google Cloud (GCP for short). I run small demo environments in these clouds and I use them to try out new features. These clouds offer free tier services, but after a while they will start charging for whatever you use on the platforms. Rule of thumb: the smaller the unit price, the more units you will probably need.
I use an American Express card to pay for the cloud services. It helped me a lot in understanding how cloud services are billed and how I can stay in control of costs I’m generating in any cloud. Because I had to pay for them myself. I will try to explain how this works.
I learned the hard way.
Doing a demo. Tried to spin some instances in a regio close to my location (West-Europe). Didn’t work. Then tried the alternative and use a region in the US. That worked. Did the demo and showed how autoscaling works, whilst making sure that every single instance had the same security posture as the original instance. Needed quite some services to do that. All worked fine. Answered questions from the audience – and totally forgot to scale down again or even completely delete the demo. Realized that after two days, when the whole thing was already charged to my Amex.
I learned through Amex Armegeddon. It got me to adopt and fanatically, almost frantically evangelize financial principles in operating cloud environments.
In my first book I wrote some substantial parts about these principles, commonly known as FinOps or cloud cost management. At the time I wasn’t aware of an organization that actively promoted FinOps: the FinOps Foundation. I learned about this organization in 2021, when I came across the FinOps bible written by founder J.R. Storment. There was this one golden rule that I immediately appreciated: Everyone takes ownership of their cloud usage.
Whenever I talk about adopting FinOps practices, I start with that rule. Every developer has to learn that every single choice he/she makes, comes with a consequence. Obviously, there might be consequences in terms of security (and teams must be 100 percent aware of this too and take mitigating actions into consideration using threat modeling), but also in terms of finances.
Here’s the challenge. With shift-left developers get tools in their hands that they can freely use to build environments. They can spin up a P3.16xlarge instance with 64 vCPUs and half a terabyte of memory in AWS. This instance alone costs around 25 USD per hour (on demand).
If it generates business value: go for it – since FinOps is not about saving costs, but about making money using cloud services (tip: you will learn if you follow the courses that are offered by the Foundation). But question yourself: do I need this machine? If yes: how often do I need this and when do I need this? Should I get it on-demand or is it wise to reserve it for a longer period and safe costs by having it as a reserved instance (saving up to tens of percentages compared to on-demand)? What are the conditions – yes, there’s small print! – to have this instance reserved for a longer period? All these questions relate to FinOps. Developers should be aware of this.
How do you make them aware? What would they do if they actually owned the business they are working for? Would they still make the same choices if they had to pay for the services themselves? It’s good to have some built-in ‘protection’ in the digital enterprise.
Work with budgets and forecasting based on business cases. Have thresholds set and monitoring in place that creates alerts if budgets are overspend. Think about rightsizing and optimizing environments – continuously, since financial management isn’t a one-off. It never is and also not in cloud.
Most important: have your teams aware of cloud spend and trained in FinOps.
Some closing words. I finally had time to take the FinOps exam last week, so I can proudly say that I’m FinOps certified. At last. I highly recommend you do the same. Check out FinOps.org. And with that, I’m closing off on the ambassadorship for the FinOps Foundation. Per July 1st I will be leaving Philips and join Fujitsu again as principal consultant. Unfortunately, consultants are not entitled to be ambassadors for the Foundation. Which doesn’t mean that I won’t be evangelizing FinOps anymore… That’s something different.