{"id":3597,"date":"2018-12-22T17:02:19","date_gmt":"2018-12-22T23:02:19","guid":{"rendered":"https:\/\/dev.iachieved.it\/iachievedit\/?p=3597"},"modified":"2018-12-22T17:07:53","modified_gmt":"2018-12-22T23:07:53","slug":"leveraging-instance-size-flexibility-with-ec2-reserved-instances","status":"publish","type":"post","link":"https:\/\/dev.iachieved.it\/iachievedit\/leveraging-instance-size-flexibility-with-ec2-reserved-instances\/","title":{"rendered":"Leveraging Instance Size Flexibility with EC2 Reserved Instances"},"content":{"rendered":"<p>Determining which <b>EC2 reserved instances<\/b> to purchase in AWS can be a daunting task, especially given the fact that you&#8217;re signing up for a long(ish)-term commitment that costs you (or your employer) real money.  It wasn&#8217;t until after several months of working with reserved instances and reading up that I became comfortable with their concepts and learning about a quite useful feature known as <b>Instance Size Flexibility<\/b>.<\/p>\n<p>But first, we need to cover what this post is not about, and that is how to choose what type of instance you need to run a given application (web server, continuous integration build server, database, etc.).  There are <a href=\"https:\/\/duckduckgo.com\/?q=determining+best+aws+instance+type\">plenty of tutorials out there<\/a>.  Once you&#8217;ve become comfortable with your choice of instance types (I gravitate towards the <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/burstable-performance-instances.html\">T<\/a>, <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/general-purpose-instances.html\">M<\/a>, and <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/memory-optimized-instances.html\">R<\/a> types), you can begin thinking about saving on your EC2 compute costs by purchasing reserved instances.<\/p>\n<p>I will admit to being a bit confused the first time I began purchasing reserved instances, and I attribute that to the fact that, well, they are a bit confusing.  Standard reserved instances.  Convertible reserved instances.  Zonal reserved instances.  No upfront payment.  Partial upfront payment.  Reserved instance marketplace.  There&#8217;s a lot to take in, and on top of that, it is a bit nerve-wracking making a choice that you might have to live with (and pay) for a while.  In fact, even after spending quite some time reading through everything, it <i>still<\/i> took me a few billing cycles to realize how reserved instances <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/concepts-reserved-instances-application.html\">really worked<\/a>.<\/p>\n<p>While I can&#8217;t help you get over that initial intimidation factor, what I can do is share a bit of wisdom I gathered from <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/apply_ri.html\">How Reserved Instances Are Applied<\/a>, with specific attention paid to <b>How Regional Reserved Instances Are Applied<\/b>.<\/p>\n<p>With some exceptions, you can purchase a number of <i>nano<\/i> (or other size) reserved instances for a given instance type, and those reservations <b>can be applied<\/b> to larger (or smaller) instances in that same family.  Note that there <i>are<\/i> exceptions (I told you it was confusing), as this feature does not apply to:<\/p>\n<ul>\n<li>Reserved Instances that are purchased for a specific Availability Zone<\/li>\n<li>bare metal instances<\/li>\n<li>Reserved Instances with dedicated tenancy<\/li>\n<li>Reserved Instances for Windows, Windows with SQL Standard, Windows with SQL Server Enterprise, Windows with SQL Server Web, RHEL, and SLES<\/li>\n<\/ul>\n<p>But that&#8217;s okay, because my favorite type of machine, a shared tenancy instance running Ubuntu 16.04 or 18.04 LTS, <i>is<\/i> supported.<\/p>\n<p><b>Instance Size Flexibility<\/b> works like this.  Each instance size is assigned a <i>normalization factor<\/i>, with the <code>small<\/code> size being given the unit factor of 1.  A <code>nano<\/code> instance has a normalization factor of 0.25.  That is, for the purposes of instance size flexibility and reserved instances, a single reservation for a <code>small<\/code> instance is the equivalent of 4 <code>nano<\/code> instances, and vice versa, 4 <code>nano<\/code> reserved instances are the equivalent of a single <code>small<\/code> reserved instance.<\/p>\n<p>AWS publishes the normalization factors in the <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/apply_ri.html\">How Reserved Instances Are Applied<\/a> documentation, but we&#8217;ll provide it here as well:<\/p>\n<table>\n<tr>\n<th>Instance size<\/th>\n<th>Normalization factor<\/th>\n<\/tr>\n<tr>\n<td>nano<\/td>\n<td>0.25<\/td>\n<\/tr>\n<tr>\n<td>micro<\/td>\n<td>0.5<\/td>\n<\/tr>\n<tr>\n<td>small<\/td>\n<td>1<\/td>\n<\/tr>\n<tr>\n<td>medium<\/td>\n<td>2<\/td>\n<\/tr>\n<tr>\n<td>large<\/td>\n<td>4<\/td>\n<\/tr>\n<tr>\n<td>xlarge<\/td>\n<td>8<\/td>\n<\/tr>\n<tr>\n<td>2xlarge<\/td>\n<td>16<\/td>\n<\/tr>\n<tr>\n<td>4xlarge<\/td>\n<td>32<\/td>\n<\/tr>\n<tr>\n<td>8xlarge<\/td>\n<td>64<\/td>\n<\/tr>\n<tr>\n<td>9xlarge<\/td>\n<td>72<\/td>\n<\/tr>\n<tr>\n<td>10xlarge<\/td>\n<td>80<\/td>\n<\/tr>\n<tr>\n<td>12xlarge<\/td>\n<td>96<\/td>\n<\/tr>\n<tr>\n<td>16xlarge<\/td>\n<td>128<\/td>\n<\/tr>\n<tr>\n<td>18xlarge<\/td>\n<td>144<\/td>\n<\/tr>\n<tr>\n<td>24xlarge<\/td>\n<td>192<\/td>\n<\/tr>\n<tr>\n<td>32xlarge<\/td>\n<td>256<\/td>\n<\/tr>\n<\/table>\n<h2>Using Instance Size Flexibility In Your Account<\/h2>\n<p>Now let&#8217;s take advantage of our knowledge about normalization factors and see how we can apply them to our account (and our bill).  We&#8217;re going to leverage the <a href=\"https:\/\/www.ruby-lang.org\/en\/\">Ruby programming language<\/a> and the <a href=\"https:\/\/aws.amazon.com\/sdk-for-ruby\/\">AWS SDK for Ruby<\/a>.  If you&#8217;ve never used Ruby before, do yourself a favor and invest some time with it.  You&#8217;ll be glad you did.<\/p>\n<p>Let&#8217;s get started.<\/p>\n<p>We&#8217;re going to be applying the instance size flexibility normalization factors, so let&#8217;s declare a <code>Hash<\/code> of their values.<\/p>\n<pre class=\"theme:tomorrow-night lang:default decode:true \">\nnormalizationFactors = {\n'nano':     0.25,\n'micro':    0.5,\n'small':    1,\n'medium':   2,\n'large':    4,\n'xlarge':   8,\n'2xlarge':  16,\n'4xlarge':  32,\n'8xlarge':  64,\n'9xlarge':  72,\n'10xlarge': 80,\n'12xlarge': 96,\n'16xlarge': 128,\n'18xlarge': 144,\n'24xlarge': 192,\n'32xlarge': 256\n}\n<\/pre>\n<p>Using <a href=\"https:\/\/bundler.io\">Bundler<\/a> to pull in our AWS SDK gem, we will retrieve all of our instances in a given region (remember that this feature is scoped to the zones in a given region).  I am using <code>us-east-2<\/code> in this example, also known as US East Ohio.<\/p>\n<pre class=\"theme:tomorrow-night lang:default decode:true \">\nrequire 'bundler\/setup'\nrequire 'aws-sdk-ec2'\n\nec2 = Aws::EC2::Resource.new(region: 'us-east-2')\n<\/pre>\n<p>Note that the above uses <code>~\/.aws\/credentials<\/code>.  If you do not have this file you will need to configure <a href=\"https:\/\/docs.aws.amazon.com\/sdk-for-ruby\/v3\/developer-guide\/setup-config.html\">your access key ID and secret access key<\/a>.<\/p>\n<p>Let&#8217;s iterate over our instances (filtering out Windows instances since they are not eligible for <b>Instance Size Flexibility<\/b>) and create a hash of the various classes.  In the end we want our hash to contain, as its keys, all of the <i>classes<\/i> (types) of instances we have, and the values to be a list of the sizes of those classes.<\/p>\n<pre class=\"theme:tomorrow-night lang:default decode:true \">\ninstances = Hash.new\nec2.instances.each do |i|\n  next if i.platform == 'windows' # Windows instances are not eligibile\n  class_, size = \"#{i.instance_type}\".split('.')\n  if not instances.has_key? class_ then\n    instances[class_] = []\n  end\n  instances[class_].push(size)\nend\n<\/pre>\n<p>For example, if we had 4 t2.nano, 3 t2.small instances, 1 t2.large, 4 m4.xlarge instances, and 2 m4.2xlarge instances, our hash would look like this:  <code>{\"t2\"=&gt;[\"nano\", \"nano\", \"nano\", \"nano\", \"small\", \"small\", \"small\", \"large\"], \"m4\"=&gt;[\"large\", \"large\", \"large\", \"large\", \"2xlarge\", \"2xlarge\"]}<\/code>.<\/p>\n<p>Now we&#8217;re going to determine how many <i>equivalent small instances<\/i> we have.  This is done by adding our normalization factors for each of the instance sizes.<\/p>\n<pre class=\"theme:tomorrow-night lang:default decode:true \">\nallocation = Hash.new\ninstances.keys.each do |k|\n  instances[k].each do |i|\n    if not allocation.has_key? k then\n      allocation[k] = 0.0\n    end\n    allocation[k] += normalizationFactors[i.to_sym].to_f\n  end\nend\n<\/pre>\n<p>Using our previous example of 4 t2.nano, 3 t2.small instances, 1 t2.large, 4 m4.xlarge instances, and 2 m4.2xlarge instances, we&#8217;re walking through the math of <code>0.25 + 0.25 + 0.25 + 0.25 + 1 + 1 + 1 + 4<\/code> for our t2 instances and <code>8 + 8 + 8 + 8 + 16 + 16<\/code> for the m4 instances.  This results in a <code>Hash<\/code> that looks like this:  <code>{\"t2\"=&gt;8, \"m4\"=&gt;64}<\/code>.  To be clear, the interpretation of this is that we have, for the purposes of <b>Instance Size Flexibility<\/b> with reserved instances, the equivalent of 8 t2.small and 64 m4.small instances in <code>us-east-2<\/code>.  Put another way, if we purchased 8 t2.small reserved instances and 64 m4.small instances in <code>us-east-2<\/code>, we would have 100% coverage of our EC2 costs with a reserved instance.<\/p>\n<p>Now, let&#8217;s take it a step further and see what the equivalence would be for the other sizes.  In other words, we know we have the equivalent of 8 t2.small and 64 m4.small instances, but what if we wanted to know how many equivalent nano instances we had?  This loop will create a row for each class and size:<\/p>\n<pre class=\"theme:tomorrow-night lang:default decode:true \">\nputs \"Class,Size,Count\"\nnormalizationFactors.each do |sk, sv|\n  allocation.each do |ak, av|\n    n = (av * (1.0 \/ sv)).floor\n    puts \"#{ak},#{sk},#{n}\"\n  end\nend \n<\/pre>\n<p>Again, taking our previous example, we would expect to see 32 t2.nano instances and 256 m4.nano instances.  That&#8217;s right.  If we purchased 32 t2.nano and 256 m4.nano instances we would have the equivalent of our 4 t2.nano, 3 t2.small instances, 1 t2.large, 4 m4.xlarge instances, and 2 m4.2xlarge instances.  Now, there doesn&#8217;t happen to be such a thing as an m4.nano instance, and we&#8217;ve corrected for this in our published <a href=\"https:\/\/github.com\/iachievedit\/ec2InstanceSizeFlexibility\">example code<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Determining which EC2 reserved instances to purchase in AWS can be a daunting task, especially given the fact that you&#8217;re signing up for a long(ish)-term commitment that costs you (or your employer) real money. It wasn&#8217;t until after several months of working with reserved instances and reading up that I became comfortable with their concepts [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3610,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[74,21,75],"tags":[77,78,80,79],"class_list":["post-3597","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-aws","category-devops","category-ec2","tag-aws","tag-ec2","tag-ec2-reserved-instances","tag-reserved-instances"],"_links":{"self":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts\/3597"}],"collection":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/comments?post=3597"}],"version-history":[{"count":15,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts\/3597\/revisions"}],"predecessor-version":[{"id":3614,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts\/3597\/revisions\/3614"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/media\/3610"}],"wp:attachment":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/media?parent=3597"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/categories?post=3597"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/tags?post=3597"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}