{"id":3341,"date":"2018-05-11T07:30:47","date_gmt":"2018-05-11T12:30:47","guid":{"rendered":"https:\/\/dev.iachieved.it\/iachievedit\/?p=3341"},"modified":"2018-05-11T07:30:47","modified_gmt":"2018-05-11T12:30:47","slug":"ansible-and-aws-part-2","status":"publish","type":"post","link":"https:\/\/dev.iachieved.it\/iachievedit\/ansible-and-aws-part-2\/","title":{"rendered":"Ansible and AWS &#8211; Part 2"},"content":{"rendered":"<p>In this post we&#8217;re going to look at <a href=\"https:\/\/en.wikipedia.org\/wiki\/Ansible_(software)\">Ansible<\/a> <i>variables<\/i> and <i>facts<\/i>, but mostly at variables (because I haven&#8217;t worked with facts much to be honest).  This is the second part of our series titled <b>Ansible and AWS<\/b> and adds to the first, so if you get lost make sure and have a look at <a href=\"https:\/\/dev.iachieved.it\/iachievedit\/ansible-and-aws-part-1\/\">Ansible and AWS &#8211; Part 1<\/a>.<\/p>\n<p>At some point you&#8217;re going to get to a point where you have two machines that <i>mostly<\/i> look like one another except for their environment, size, web URL, etc.  I&#8217;ve come across this when having two servers in separate environments, say, development, staging, and production.  They will have different URLs, different amounts of CPU or RAM (which can drive certain configuration values).  Or, let&#8217;s say each machine backs up data to a given S3 bucket, and that backup script you wrote needs the bucket name.  A perfect usecase for an Ansible variable.<\/p>\n<p>So let&#8217;s quickly look at the four primary locations for a variable, and then I&#8217;ll share several rules of thumb I use as to where to put one:<\/p>\n<ul>\n<li>in <code>ansible_hosts<\/code><\/li>\n<li>in <code>group_vars<\/code><\/li>\n<li>in <code>host_vars<\/code><\/li>\n<li>in the playbook<\/li>\n<\/ul>\n<p>Now I did say <i>primary<\/i> locations because there are other places; for now we&#8217;re ignoring variables and defaults that are included in roles or provided on the commandline.  For the canonical reference on variables, see the official <a href=\"http:\/\/docs.ansible.com\/ansible\/latest\/user_guide\/playbooks_variables.html\">documentation<\/a>.<\/p>\n<h2>ansible_hosts<\/h2>\n<p>Don&#8217;t put variables here.  Moving on.<\/p>\n<p>I kid, somewhat.  When first starting out you may see <code>ansible_hosts<\/code> files that look like this:<\/p>\n<p>[code lang=text]<br \/>\n[all]<br \/>\n18.188.72.168 HOSTNAME=ansible-helloworld OPERATING_ENVIRONMENT=staging<br \/>\n[\/code]<\/p>\n<p>Terrible form, but let&#8217;s try it out in our playbook (see <a href=\"https:\/\/dev.iachieved.it\/iachievedit\/ansible-and-aws-part-1\/\">Ansible and AWS &#8211; Part 1<\/a>) and <a href=\"https:\/\/github.com\/iachievedit\/ansible-helloworld\">the sample Github repository<\/a>.  We&#8217;re going to add another <i>task<\/i> to our playbook that creates a file on the server based upon a template.  The templating language is <a href=\"http:\/\/jinja.pocoo.org\">Jinja2<\/a> (don&#8217;t worry, subsequent posts will go into Jinja and Ansible templates in much greater detail).  First, create a new directory (inside your <code>ansible-helloworld<\/code> directory) called <code>templates<\/code>.  This is a specific name for Ansible, so don&#8217;t name it <code>template<\/code> or something else:<\/p>\n<p>[code lang=text]<br \/>\n# cd ansible-helloworld<br \/>\n# mkdir templates<br \/>\n[\/code]<\/p>\n<p>Inside of <code>templates<\/code> create a file called <code>environment.j2<\/code> (<code>.j2<\/code> is the extension used by Jinja2 templates) and populate it with the following content:<\/p>\n<p>[code lang=text]<br \/>\n# Created by Ansible<\/p>\n<p>OPERATING_ENVIRONMENT=&#039;{{ OPERATING_ENVIRONMENT }}&#039;<br \/>\n[\/code]<\/p>\n<p>Note!  I personally prefer any file on a server that was created by Ansible to say as much right at the beginning.  So many times people have gone on to a server and edited a file without realizing that it was generated and stands a good chance to be overwritten.  We could do an entire article on what a good header for a such a file might look like.  Hell, I might just do that!<\/p>\n<p>Then, in your playbook add the following task to the end:<\/p>\n<pre class=\"lang:yaml\">\n  # Create \/etc\/environment\n  - name:  Create \/etc\/environment\n    template:\n      src:  environment.j2\n      dest: \/etc\/environment\n<\/pre>\n<p>Remember that YAML is indentation sensitive, so if you paste this into your playbook, make sure it is properly aligned with the rest of your tasks.<\/p>\n<p>One last step!  Locate your <code>hostname<\/code> task and change the hardcoded <code>ansible-helloworld<\/code> (that&#8217;s our hostname), to <code>\"{{ HOSTNAME }}\"<\/code>, like this:<\/p>\n<pre class=\"lang:yaml\">\n  # Set our hostname\n  - name:  Set our hostname\n    hostname:\n      name:  \"{{ HOSTNAME }}\"\n<\/pre>\n<p>If you&#8217;re quick on the uptake, you should already know what is going to happen when this playbook is executed.  The variable <code>HOSTNAME<\/code> in the <code>ansible_hosts<\/code> file is going to be applied in the <code>hostname<\/code> task, and the <code>OPERATING_ENVIRONMENT<\/code> variable will be applied in the <code>template<\/code> task.<\/p>\n<p>Go ahead and run the playbook:<\/p>\n<p>[code lang=text]<br \/>\nPLAY [all] *********************************************************************<\/p>\n<p>TASK [Gathering Facts] *********************************************************<br \/>\nok: [18.188.72.168]<\/p>\n<p>TASK [Set our hostname] ********************************************************<br \/>\nok: [18.188.72.168]<\/p>\n<p>TASK [Install base packages] ***************************************************<br \/>\nok: [18.188.72.168] =&gt; (item=[u&#039;htop&#039;, u&#039;zsh&#039;])<\/p>\n<p>TASK [Create \/etc\/environment] *************************************************<br \/>\nchanged: [18.188.72.168]<\/p>\n<p>PLAY RECAP *********************************************************************<br \/>\n18.188.72.168              : ok=4    changed=1    unreachable=0    failed=0<br \/>\n[\/code]<\/p>\n<p>Because there is a new task (the <code>template<\/code> task), and our hostname didn&#8217;t change, we see <code>changed=1<\/code>.<\/p>\n<p>If you look at <code>\/etc\/environment<\/code> on the server, you should see:<\/p>\n<p>[code lang=text]<br \/>\ncat \/etc\/environment<br \/>\n# Created by Ansible<\/p>\n<p>OPERATING_ENVIRONMENT=&#039;staging&#039;<br \/>\n[\/code]<\/p>\n<p>Nice.<\/p>\n<p>Let&#8217;s change our hostname in <code>ansible_hosts<\/code> to simply <code>helloworld<\/code>:<\/p>\n<p>[code lang=text]<br \/>\n[all]<br \/>\n18.188.72.168 HOSTNAME=helloworld OPERATING_ENVIRONMENT=staging<br \/>\n[\/code]<\/p>\n<p>Rerunning the playbook will change the hostname to <code>helloworld<\/code>, since that is the new value of the <code>HOSTNAME<\/code> variable.<\/p>\n<h2>Group Variables<\/h2>\n<p>Notice in our <code>ansible_hosts<\/code> file there is the <code>[all]<\/code> tag?  Well, we can change to that split hosts up.  Let&#8217;s rewrite our <code>ansible_hosts<\/code> file to look like this:<\/p>\n<p>[code lang=text]<br \/>\n[staging]<br \/>\n18.188.72.168 HOSTNAME=helloworld<br \/>\n[\/code]<\/p>\n<p>and then create a directory (inside your <code>ansible-helloworld<\/code> directory) called <code>group_vars<\/code>.  Then in <code>group_vars<\/code> create a file called <code>staging.yml<\/code> and in it put:<\/p>\n<p>[code lang=text]<br \/>\n&#8212;<br \/>\nOPERATING_ENVIRONMENT:  staging<br \/>\n[\/code]<\/p>\n<p>and run your playbook.  If you&#8217;re following along verbatim nothing should happen.  All we&#8217;ve done is extracted variables common to staging servers (like our <code>OPERATING_ENVIRONMENT<\/code> variable) into a single file that will apply to all of the hosts in the <code>staging<\/code> group.<\/p>\n<p>Try renaming <code>staging.yml<\/code> to <code>somethingelse.yml<\/code> and rerunning.  You should get an error regarding an undefined variable, since Ansible wasn&#8217;t able to find <code>OPERATING_ENVIRONMENT<\/code>.  When you supplied a properly named <code>group_vars<\/code> file (<code>staging.yml<\/code>) it is able to look it up.<\/p>\n<p>Finally, for our group variables, notice the syntax changed from the <code>ansible_hosts<\/code> &#8220;ini-style&#8221; syntax to a YAML file.  This is important to note!<\/p>\n<h2>Host Variables<\/h2>\n<p>Now let&#8217;s take a look at <i>host variables<\/i> and how to use them.  Create a directory called <code>host_vars<\/code>, again, inside the <code>ansible-helloworld<\/code> directory.  In it create a directory named the same as how your host is defined in <code>ansible_hosts<\/code>.  My server is being referenced as 18.188.72.168, but since AWS provides an FQDN, I&#8217;ll switch to that to demonstrate how what is in <code>ansible_hosts<\/code> can be an IP address, alias in <code>\/etc\/hosts<\/code>, or an FQDN resolvable by DNS.  I&#8217;m going to change my <code>ansible_hosts<\/code> to this:<\/p>\n<p>[code lang=text]<br \/>\n[staging]<br \/>\nec2-18-188-72-168.us-east-2.compute.amazonaws.com<br \/>\n[\/code]<\/p>\n<p>and then create a file called <code>vars.yml<\/code> in <code>host_vars\/ec2-18-188-72-168.us-east-2.compute.amazonaws.com<\/code> and place the following content:<\/p>\n<p>[code lang=text]<br \/>\n&#8212;<br \/>\nHOSTNAME:  helloworld<br \/>\n[\/code]<\/p>\n<p>Try the playbook out!<\/p>\n<h2>Take Note<\/h2>\n<p>Have you noticed that we&#8217;ve eliminated the variables from our <code>ansible_hosts<\/code> file?  You may feel otherwise, but I&#8217;m a strong proponent of <code>ansible_hosts<\/code> files that consist of nothing more than groups of hostnames.  You might think at first that you&#8217;ll have just a couple of variables per host, but odds are this will grow to a dozen or more quickly!  Think about the various things that are configurable on an environment or host basis:<\/p>\n<ul>\n<li>what monitoring environment will this host report to?<\/li>\n<li>where are syslogs being sent?<\/li>\n<li>what S3 bucket is used for backing up configuration files (that aren&#8217;t in Ansible)?<\/li>\n<li>what are the credentials to that bucket?<\/li>\n<li>does the host (or group) have custom nameservers?<\/li>\n<li>how is application-specific configuration handled?<\/li>\n<\/ul>\n<p>And so on.  Trust me.  Get into the good habit of creating a <code>group_vars<\/code> and <code>host_vars<\/code> directories and start putting your variables there.<\/p>\n<h2>Variables In Playbooks (and Other Places)<\/h2>\n<p>You can also put variables in your playbooks, but I rarely do.  If it is in the playbook it&#8217;s really less of a variable and more of a strict setting, since it will override anything from your <code>ansible_hosts<\/code> file, <code>group_vars<\/code> or <code>host_vars<\/code>.  If you&#8217;re set on it, try this out.  In your playbook, add <code>vars:<\/code> like this:<\/p>\n<pre class=\"lang:yaml\">\n  become_method: sudo\n  vars:\n    HOSTNAME:  from_my_playbook\n    OPERATING_ENVIRONMENT:  from_my_playbook\n\n  tasks:\n<\/pre>\n<p>That is, nestle it in between the <code>become_method<\/code> and <code>tasks<\/code>.  Run it and you&#8217;ll see that both your hostname and <code>\/etc\/environment<\/code> file changes.<\/p>\n<h2>Facts<\/h2>\n<p>Ansible also provides the ability to reference <i>facts<\/i> that it has gathered.  To be sure, I don&#8217;t use this feature as often, but sometimes I&#8217;ll need the host&#8217;s IP address (perhaps to put it in a configuration file), or how many CPUs it has.  Seriously, check out the <a href=\"http:\/\/docs.ansible.com\/ansible\/latest\/user_guide\/playbooks_variables.html#information-discovered-from-systems-facts\">documentation<\/a> of all of the awesome facts you can reference in your playbook.  Here are some that you may find yourself using, <i>especially<\/i> if your playbooks require subtle tweaks to support different distributions, amount of memory, CPU, etc.:<\/p>\n<ul>\n<li><code>ansible_distribution_name<\/code><\/li>\n<li><code>ansible_distribution_release<\/code><\/li>\n<li><code>ansible_distribution_version<\/code><\/li>\n<li><code>ansible_eth0<\/code> (and properties of it)<\/li>\n<li><code>ansible_memtotal_mb<\/code><\/li>\n<li><code>ansible_processor_cores<\/code><\/li>\n<li><code>ansible_processor_count<\/code><\/li>\n<\/ul>\n<p>You can use these in your playbook in the same manner as variables:<\/p>\n<pre class=\"lang:yaml\">\n  - name:  Total Server RAM (MB)\n    debug:\n      msg:  \"Total Server RAM:  {{ ansible_memtotal_mb }} MB\"\n<\/pre>\n<p>This is a <code>debug<\/code> task will just print out a message:<\/p>\n<p>[code lang=text]<br \/>\nTASK [Total Server RAM (MB)] ***************************************************<br \/>\nok: [ec2-18-188-72-168.us-east-2.compute.amazonaws.com] =&gt; {<br \/>\n    &quot;msg&quot;: &quot;Total Server RAM:  990 MB&quot;<br \/>\n}<br \/>\n[\/code]<\/p>\n<p>Let&#8217;s make use of facts in our <code>\/etc\/environment<\/code> template:<\/p>\n<p><code>templates\/environment.j2<\/code>:<\/p>\n<p>[code lang=text]<br \/>\n# Created by Ansible ({{ ansible_date_time.date }} {{ ansible_date_time.time }} {{ ansible_date_time.tz }})<\/p>\n<p>OPERATING_ENVIRONMENT=&#039;{{ OPERATING_ENVIRONMENT }}&#039;<br \/>\n[\/code]<\/p>\n<p>Notice here we&#8217;re using the <code>ansible_date_time<\/code> fact to include in our generated <code>\/etc\/environment<\/code> file when the file was created.  After running:<\/p>\n<p>[code lang=text]<br \/>\nubuntu@helloworld:~$ cat \/etc\/environment<br \/>\n# Created by Ansible (2018-05-11 11:56:48 UTC)<\/p>\n<p>OPERATING_ENVIRONMENT=&#039;staging&#039;<br \/>\n[\/code]<\/p>\n<h2>Final Remarks<\/h2>\n<p>This post might seem at first glance to be a little less &#8220;meaty&#8221; than <a href=\"https:\/\/dev.iachieved.it\/iachievedit\/ansible-and-aws-part-1\/\">Part 1<\/a>, but variables and facts will comprise a large part of your future playbooks (we didn&#8217;t even talk about how they are used in roles).  As promised here are some guidelines I use when deciding where to put variable definitions:<\/p>\n<p><i>If<\/i> the hosts for a given Ansible playbook can be organized into logical groups (such as staging and production), and there are a set of variables that will be common to all of the staging servers, and likewise common to all of the production servers, these variables are a good candidate to put into <code>group_vars<\/code>.  Examples here might be:<\/p>\n<ul>\n<li>the endpoints for log shipping<\/li>\n<li>the IP addresses for name servers<\/li>\n<li>the IP of a monitoring server<\/li>\n<li>AWS region information (for example, if staging is in one region and production is in another)<\/li>\n<\/ul>\n<p>Or, let&#8217;s say you run some type of monitor traps that send e-mails on certain events.  You might want to send staging alerts to staging-alerts@iachieved.it vs. production-alerts@iachieved.it.  Ansible group variables might come in handy here.<\/p>\n<p><i>If<\/i> the variable is specific to a host, then obviously you&#8217;d put the information in <code>host_vars<\/code>.  I prefer to explicitly set server hostnames, and so <code>HOSTNAME<\/code> goes into the <code>vars.yml<\/code> for the host.  Application-specific information for that specific host, put it into <code>host_vars<\/code>.<\/p>\n<p>I&#8217;m hard pressed to think of when I&#8217;d want to explicitly put a variable in either <code>ansible_hosts<\/code> or the <code>playbook<\/code> itself; in <code>ansible_hosts<\/code> it just clutters up the file and in the <code>playbook<\/code> it&#8217;s effectively a constant.<\/p>\n<p>Now, make no mistake:  you will, over time, refactor your playbooks and the locations and groupings of variables.  If you&#8217;re a perfectionist and lose sleep over whether you&#8217;ve spent enough time in discussions over how to architect your playbooks, well, I feel sad for you.  It&#8217;ll never be right the first time, so make the best decision you can and sleep well knowing it can be refactored as your environment or needs change.<\/p>\n<h2>Getting the Code<\/h2>\n<p>You can find the finished playbook for this article on the <code>part2<\/code> branch of <a href=\"https:\/\/github.com\/iachievedit\/ansible-helloworld\">this Github repository<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post we&#8217;re going to look at Ansible variables and facts, but mostly at variables (because I haven&#8217;t worked with facts much to be honest). This is the second part of our series titled Ansible and AWS and adds to the first, so if you get lost make sure and have a look at [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3318,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-3341","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts\/3341"}],"collection":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/comments?post=3341"}],"version-history":[{"count":10,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts\/3341\/revisions"}],"predecessor-version":[{"id":3351,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/posts\/3341\/revisions\/3351"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/media\/3318"}],"wp:attachment":[{"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/media?parent=3341"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/categories?post=3341"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dev.iachieved.it\/iachievedit\/wp-json\/wp\/v2\/tags?post=3341"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}