Cost Optimized Cloud Development Environments at Siemens

At the heart of AWS compute is Amazon Elastic Compute Cloud (Amazon EC2). Many services will run in a container on an EC2 instance. However, not all instances are created equally. You pay for these resources by the hour or by the second (see EC2 pricing page); they can be tracked in the AWS billing console.

It is important to understand that compute in the cloud can be purchased in different ways with different cost considerations.

On-demand resources are the most common, and when your development team is just getting started, this is often the first thing developers will try. An On-Demand resource is available on request and remains "yours" until you shut it down or terminate it. They are the most flexible and a good choice for running something for a limited time with guaranteed availability. However, it's important to consider if your development instance has an up-time requirement? What would happen if the environment were unavailable for a short while? Would all work come to a halt? This is not usually the case, although it is possible that there are sometimes mission critical development activities, typically the development process can tolerate a failure.

Your development environment may be an isolated deployment, then the impact if a node was unavailable would be minimal. In some cases where proper CICD is used, there may be a window of time in which the software is under test and that time window maybe small but may reoccur multiple times per day. In this case, a cost optimization to consider is "spot instances".

Spot Instances are transient resources that can be purchased at a significantly reduced price (at up to 90% discount compared to On-Demand prices), but there is a caveat. If your instance is outbid on the marketplace, you will receive 2-minute notification before you lose your instance. When considering "I might lose my instance" many people worry that they would have significant problems if an instance was ever to loose. However, if you are designing stateless services and applications that are intended to run in a distributed, high availability, load balanced configuration you are designing services that can tolerate such incidents. EC2 instances should always be considered as ephemeral. They could fail, reboot or get lost even when using on-demand or reserved instances. Something can and will go wrong, which is why you design for resilience, redundancy and failover. Designing for these failures is essential in the cloud and your spot instance usage can benefit from this best practice design concept. Learn best practices for handling EC2 Spot Instance interruptions from this blog.

Reserved instances (RIs) are purchased as a commitment to run that instance 24x7 for a period of time (1 or 3-year term) with the highest savings available on a 3-year term. However, if you are not running the instance, you still pay the hourly price for that instance. Meaning you pay whether it's running or idle. Reserved Instances provide you with a discount (up to 72%) compared to On-Demand instance pricing.

RIs are shared within an account. What if I have a reserved m4.8xlarge instance, but I only need to run it 8 hours per day and 5 days a week? It may still make sense for me to purchase such an instance if my team is 24x7 because when I am not using the reserved instance, someone else can. If I have 3 m4.8xlarge and each runs 8 hours with no overlap, I could reserve one instance and share it because the m4.8xlarge that is running is billed at the RI price.

The break-even point for an RI is calculated by comparing the on-demand price of total required hours versus the RI price of 24x7. RIs are not always the right choice. It depends on what the instance type is, how long you plan to run it, and whether the RI can be shared.

Here is an example calculation to assist in determining when to use an RI. Let's take US East (N. Virginia) as example for t3.xlarge standard 1 year with no upfront:

• No Upfront total cost: $0.104 per hour * 24 hours * 365 days = $911.04
• On-Demand cost: $0.1664 per hour
• Number of hours for On-Demand to reach No Upfront total cost: $911.04 / $0.1664 = 5,475 hours = 228.125 days = 62.5% of one year
  

Using RIs in development is not always the most cost-effective option. Development is not production because instances do not have to handle production capacity. A smaller instance is often acceptable for the majority of development and test unless you are executing scalability, boundary, load or performance tests. However, even these do not have to be run continuously in development.

Development involves experimentation and testing; some testing may include trying different instance types in order to find the optimal resource. This is known on AWS as "right-sizing". This is the process of identifying the most cost-optimized compute resource needed to execute your tasks. If your code is very well written, highly optimized, memory and CPU efficient you may be able to use very inexpensive resources. By having a good understanding of your compute resource needs and creating a scalability calculation based on load characteristics you can gain a better understanding of the required resources at various scale points and under various conditions. You may be able to scale-back your development resources to lower costs or even run tasks with limited High Availability (HA) configurations except when running HA tests. AWS Cost Explorer Rightsizing Recommendations in your Billing and Cost Management console will help identify rightsizing opportunities for your idle and underutilized instances.

Keep your environment configuration separate from your infrastructure provisioning code. Automate provisioning of your compute and deployment of your code. This allows you to quickly and easily change the configuration, re-provision and redeploy your services to test different scenarios without locking yourself into an expensive compute contract. For example, you may be able to spend pennies per hour on T2 instances in development. T2 instances are a family of very low-cost AWS EC2 compute resources.

You can use a hybrid model of these approaches. Consider reserving a small pool of lower compute resources where you need an always-on environment and use spot resources for specialized test scenarios involving larger compute instances or auto-scaling. Using RIs or on-demand instances for HA configuration and spot instances for the elastic auto-scaling may also be an option if your code is well designed to failover or your infrastructure is self-healing.

The cloud is different. There is no computer. There is no server. There is a configuration and script that creates servers and puts all my favorite tools and code there. It may go away at any time and I don't care because with a click of a button I can make it come back with minimal effort. This is the key to happiness in the cloud.

With the cloud, I can easily "turn on" the development environment. Consider an IDE such as eclipse. With it you can write your code, but in order to test your code you build and start a runtime. With the runtime you can execute a unit test, launch an app server and perform interactive debugging. The cloud development "environment" is the same concept. If you have a service and you want to test it, you build -> deploy -> test. The issue is that in the deploy step you expect that you will deploy to something. What if the something you are deploying to doesn't exist or is simply not running. Automate the process and include the ability to provision your resources on deploy so you can turn them on only when you need them.

I often hear arguments such as: "We are a global team, operating 24x5 therefore someone always needs my environment running." What about the other 2 days a week? These two days account for about 28% of total time over a year. Imagine your infrastructure cost for development is 1 million/year and you can save 28%, you could potentially realize a savings of 280k / year.

I once observed a team experience a scalability issue in production. While performing a root cause analysis, a few hypotheses were established which needed further testing to verify. As a result, the team decided to execute a test on their scalability configuration in development. After 1 week an alert was sent from the cost monitoring tool to indicate a sudden spike in development costs for that team's AWS account.

The team's solution was to increase system capacity and execute a test to reproduce the production issue. This resulted in many more, larger nodes running in development than is normally expected. This is OK if you are shutting off the resources when they are not in use and only provisioning such an environment for test duration of the test. It can become very expensive to replicate a production-like environment in development. Establish a test scenario, provision for the test, execute, collect results and return to the low-cost deployment.

A little thought and effort in the beginning will provide significant long-term benefits. You just need to take the first step to start realizing these benefits.

Learn other cost optimization best practices here.