Some Turbot Guardrails customers must deal with massive scale in their environment (we work with teams that have millions of managed cloud resources across thousands of AWS accounts, GCP projects and Azure subscriptions) and previous generation product dealt with this scale with large fleets of auto-scaling ec2 instances.
One thing that has always stuck in our craw was lack of an elegant (and cost effective) way to run large numbers of servers that needed small high-performance disks. While our application required less than 100 GiB of disk space, we would frequently burst over 2,500 IOPS for periods of time, exhausting any available gp2 burst balance.
AWS EBS volumes using the gp2 volume type have a baseline performance of 3 IOPS per GiB, so a 100 GiB volume would only get a guaranteed 300 IOPS with some capacity to burst to 3,000. The cost of that volume would be $10/month. To guarantee 3,000 IOPS we had only 2 choices:
- Use a provisioned IOPS volume type (io1): The cost of running a 100 GiB volume with 3,000 IOPS would be $208/month (us-east-1).
- Overprovision capacity: If we provision a 1,000 GiB gp2 volume then we would also get a guaranteed baseline of 3,000 IOPS at the cost of $100/month.
While the obvious answer is (b), it feels wasteful to have so much unused storage and if your volumes require backups the additional cost of storage for snapshotting a terabyte volume vs 100 GiB eats away the 50% cost savings.
Given all of that, we were delighted to see the announcements from AWS during Tuesday's re:Invent keynote session of the availability of the new gp3 EBS instance type. EBS gp3 volumes have a baseline performance of 3,000 IOPS and can scale to 16,000 IOPS, regardless of volume size!
That means that in our use case a 100 GiB gp3 volume with 3,000 IOPS would only cost $8/month. One of our hero customers typically runs (on average) ~32 ephemeral instances; this one change to their configuration will save them over $35,000 a year.
The gp3 cost advantage really shines with small volumes needing high IOPS and large volumes needing moderate IOPS, but the really crazy news though is that gp3 is 10-20% less costly than gp2 in every head-to-head use case. You can see this for yourself in the following graph with shows comparative cost of gp2, gp3 and io1/io2 EBS volume types:
Given the available cost savings, we are recommending to all customers that they implement automated controls to discover any EBS volumes that are wasting money in their environment. This can be accomplished by setting the following policies:
First, we enable the policy that will check if volumes are approved:
Then build a list of approved volume types:
Finally, we build a calculated policy to check if there are any io1/io2 volumes that are using less than 16,000 IOPS (which would be a huge cost savings to move to gp3):
We are very excited to work with our customers to find and implement these cost savings on EBS, but there were also interesting announcements of new features for AWS Lambda and S3 that will result in cost savings for our customers running Turbot Guardrails as well. Keep an eye out for our next post on those use cases and subscribe to our newsletter so you dont't miss out!