It’s been some time since I’ve posted to this blog, which is a shame, because I do indeed enjoy writing as well as sharing what I’ve learned with the hope it helps someone else. The truth is, however, that writing quality articles takes a lot of time. I also suffer from that thought that surely someone else out there has written on a given topic and done a much better job that I could do. And the reality is, someone probably has, but that shouldn’t stop me from doing something I enjoy.<\/p>\n
So with that, here’s the first of what I hope to be several more articles on how to harness the power of Ansible<\/a> with AWS<\/a>. I firmly believe in learning by doing<\/i> and I also believe that at first<\/i>, you should take few shortcuts. What I mean by that is this: are you the type of person that’s saddened by the fact that some schools allow calculators in grade school? Or, even better, not bothering teaching someone how to use a library? And by that I mean how to get off of your ass, go to the library, navigate the stacks, and find a wealth of information you just might not find on Google? To be fair, I’m not really the Get off my lawn!<\/a> type, but it does irk me somewhat when I see people automating things they don’t have a fundamental understanding of in the first place.<\/p>\n With that, my approach will be to start off with doing some things manually<\/i>. Then I’ll illustrate how to take the manual steps and automate them. Sometimes it isn’t worth<\/i> automating something; usually it’s something you do infrequently enough and the energy required to automate it outweighs the benefit. I usually use the DevOps equivalent of the three times rule<\/a>. If I’ve done something by hand three times in a row, it’s a good candidate for automation, but not necessarily before then.<\/p>\n Ansible<\/a> is just so wonderful. If you’ve never heard of it, it falls into the general class of automation tools that allow you to codify a set of instructions<\/i> for doing or building something<\/i>. I tried to make that as abstract as possible, because the something<\/i> could be:<\/p>\n and so on. It’s almost as if these sets of instructions are like playbooks<\/i> or recipes<\/i> or cookbooks<\/i>. So much so many of the popular tools call their files variations of these phrases. Ansible uses tasks<\/i>, roles<\/i>, and playbooks<\/i>.<\/p>\n To use a tool you’ll have to install it. I develop on a Mac and can tell you that Ansible is<\/i> easy to use, though at first it may not seem like it. I know you want<\/i> to cut to the chase and just figure out how to automate that thing and you want it now<\/i>. Why do I have to do all this stuff just to do this thing I think should be easy! I can hear you say it because I say it all the time. That initial learning curve is always a pain in the ass.<\/p>\n Here are some simple hints to get started as quickly as possible. First, create a folder called [code lang=text] [ssh_connection] Now, for Like Ansible, AWS<\/a> is awesome. I have on my bookshelf a copy of The Cloud at Your Service<\/a>, where I was first introduced to AWS. This is awesome<\/i>, I thought. Rather than hosting a webserver from this old Sony Vaio (whatever happened to that thing anyway?), now I can create a VM in the cloud<\/i>. Forgive those that are middle-aged and still look back at where computing was, and where it is today, and think wow<\/i>.<\/p>\n Since AWS was my first love, I’ve tended to stick with it, and to be honest, for good reason. The product has become cheaper over the years even while adding more services such as S3, Route 53, VPCs (there was a time VPCs didn’t exist), and now things like RDS and much more. Yes, there are other cloud computing platforms out there (Azure, Google, Linode, Digital Ocean, Scaleway, and many more), and given the need I could<\/i> go and probably master them all, but for now I’m quite content with AWS. If you find yourself using another platform, Ansible will still work<\/i>, you just may need to tweak some of the techniques and steps presented here.<\/p>\n Finally, this isn’t an AWS tutorial by any means, though we will get into some handy Ansible modules that take some of the drudgery out of creating instances and databases (in which case if you’re on another platform you will definitely need to find some other module that does the equivalent). So if you don’t know how to create an EC2 instance and set some basic security groups, you might start here<\/a>.<\/p>\n One last note before we get back to Ansible, and that\u2019s I\u2019ll be using Ubuntu Server 16.04, also known as Xenial Xerus. You may prefer to work with CentOS, Red Hat Enterprise, or plain Debian. While Ansible is generally distribution agnostic, some things may not work exactly the same.<\/p>\n Alright, I hope you created an EC2 instance in AWS. I created one in the Ohio region (us-east-2), and as promised used Ubuntu 16.04 LTS (ami-916f59f4). For basic tutorials go ahead and use the t2.micro instance, it\u2019s a lot cheaper. You should also have ensured that you have access to this instance, that is, your public SSH key was installed on this new instance and your security groups permit you to access it (protecting your EC2 infrastructure with security groups is another great topic that I hope to get to one day).<\/p>\n My instance was given a public IP of 18.188.72.168 and my key was installed, so [code lang=text] Hot damn.<\/p>\n One of the unfortunate things about the default AMI for Ubuntu 16.04 is that it doesn\u2019t include [code lang=text] Of course we entered Y<\/em>.<\/p>\n Now for the fun part, some Ansible!<\/p>\n In your [code lang=text] Now, that’s the public IP address of my<\/i> instance, and you aren’t going to use that, you’ll use your instance’s IP. The point is your You’re ready for your first playbook. Call it whatever your like, we’ll just use That’s it! See you in Part 2!<\/p>\n Just kidding. We’ll add a bit more to our first playbook, but first, let’s take a look at the syntax of the file. The extension For a brief moment we’ll ignore the Now, don’t confuse the Notice how the If you’re confused, don’t worry. YAML (and any serialization syntax) can be baffling and bewildering. But if you follow some basic conventions and keep playbooks simple (which will involve breaking things down over time into roles and other tasks), it’ll be easy to read.<\/p>\n Now, what you aren’t supposed to do is put the Much better. Now, let’s run it!<\/p>\n [code lang=text] TASK [Gathering Facts] ********************************************************* TASK [Set our hostname] ******************************************************** TASK [Install htop] ************************************************************ PLAY RECAP ********************************************************************* In general, an operation is idempotent<\/i> if it produces the same result<\/i> even after being executed multiple times. This is an important property to strive for in Ansible playbooks, though my experience is that it isn’t always achievable. It may appear on the surface that our above playbook is idempotent though it isn’t. For argument’s sake let’s look at what we would desire from an idempotent playbook:<\/p>\n On the first pass of the playbook we see [code lang=text] Now, nothing has changed. Run it again!<\/p>\n [code lang=text] Success! An idempotent playbook.<\/p>\n Until We didn’t cover the We could create a separate Ansible<\/h2>\n
\n
Installing Ansible<\/h3>\n
brew install ansible<\/code> works great (if you don’t have Brew<\/a> installed on your Mac you should be ashamed of yourself). As of this writing I’m using Ansible version 2.5.2, as that is what brew<\/code> installed.<\/p>\nUsing Ansible<\/h3>\n
ansible-helloworld<\/code> and use it as home base. Then, you’re going to create two files: ansible.cfg<\/code> and ansible_hosts<\/code>.<\/p>\nansible.cfg<\/code>:<\/p>\n
\n[defaults]
\ninventory=ansible_hosts
\nhost_key_checking=False<\/p>\n
\npipelining=True
\n[\/code]<\/p>\nansible_hosts<\/code>, this is where you put the names of your servers that you’re going to be applying your tasks to. We don’t have any hosts yet, so we need to create one. You can, of course, apply Ansible tasks to your own Mac, but we won’t do that. You could also install VirtualBox and create a virtual machine to play with, but we’ll skip that too and go straight to AWS.<\/p>\nAmazon Web Services<\/h2>\n
Back to Ansible<\/h2>\n
ssh ubuntu@18.188.72.168<\/code> logged me right in:<\/p>\n
\nssh ubuntu@18.188.72.168
\nWelcome to Ubuntu 16.04.4 LTS (GNU\/Linux 4.4.0-1052-aws x86_64)
\n[\/code]<\/p>\npython<\/code> on it. Now granted, Python has only been around since 1991, but still, you\u2019d think it was worth including as a default package in an Ubuntu instance. This is important because Ansible requires python<\/code> to be on the target instance (we’re talking specifically about Linux instances here). So, until we show you how to create your own AMI, we need to install Python on this VM. Easy enough:<\/p>\n
\nubuntu@ip-172-31-22-141:~$ sudo apt-get install python
\nReading package lists… Done
\nBuilding dependency tree
\nReading state information… Done
\nThe following additional packages will be installed:
\n libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python-minimal
\n python2.7 python2.7-minimal
\nSuggested packages:
\n python-doc python-tk python2.7-doc binutils binfmt-support
\nThe following NEW packages will be installed:
\n libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python
\n python-minimal python2.7 python2.7-minimal
\n0 upgraded, 7 newly installed, 0 to remove and 0 not upgraded.
\nNeed to get 3,877 kB of archives.
\nAfter this operation, 16.6 MB of additional disk space will be used.
\nDo you want to continue? [Y\/n]
\n[\/code]<\/p>\nYour `ansible_hosts` File<\/h3>\n
ansible_hosts<\/code> file just put this:<\/p>\n
\n[all]
\n18.188.72.168
\n[\/code]<\/p>\nansible_hosts<\/code> file needs either an IP or FQDN of the machine(s) that you’re going to be working with. It’s that simple really.<\/p>\nYour First Playbook<\/h3>\n
playbook.yml<\/code> for now. And here we go:<\/p>\nplaybook.yml<\/code>:<\/p>\n\n---\n- hosts: all\n remote_user: ubuntu\n become: true\n become_user: root\n become_method: sudo\n tasks:\n # Set our hostname\n - name: Set our hostname\n hostname:\n name: ansible-helloworld\n<\/pre>\n
Ansible Modules<\/h3>\n
.yml<\/code> is on purpose as Ansible roles, tasks, variable configurations, playbooks, etc. are all written using YAML<\/a>, a “human friendly data serialization standard for all programming languages.” YAML’s syntax is indentation-driven (like Python), so you need to make sure everything is properly indented or you’ll get incomprehensible errors. For a quick check you can install something like yamllint<\/code><\/a> (availabe on the Mac with brew install yamllint<\/code>).<\/p>\nhosts<\/code> and remote_user<\/code> tags and focus instead on the tasks<\/code> section. We have one task listed here: hostname<\/code>. Ansible provides great reference documentation<\/a> to look up all of the things you can do with this module. This is a simple one as it only takes one parameter, name<\/code>.<\/p>\nname<\/code> parameter of the hostname<\/code> module with that of the name<\/code> parameter of the task<\/code>. This used to trip me up. It’s easier to see task entries when there are more than one, say let’s do that. I’m going to be very specific on how this is written (and then tell you not to do it):<\/p>\n\n---\n- hosts: all\n remote_user: ubuntu\n become: true\n become_user: root\n become_method: sudo\n tasks:\n # Set our hostname\n - name: Set our hostname\n hostname:\n name: ansible-helloworld\n\n - apt:\n name: htop\n state: present\n update_cache: yes\n name: Install htop\n<\/pre>\n
apt<\/code> module (which also has great reference documentation<\/a>) is the second list entry under tasks<\/code>; we know this because of the dash. But then there is some indentation (two spaces) and three key-value pairs (name<\/code>, state<\/code>, and update_cache<\/code>). Then<\/i> the indentation is backed out and we have name<\/code>. The second name<\/code> here (which has the value Install htop<\/code>) is associated with the task<\/i> at hand (in our case, apt<\/code>).<\/p>\nname<\/code> parameter for the task at the bottom. Clean that up!<\/p>\n\n---\n- hosts: all\n remote_user: ubuntu\n become: true\n become_user: root\n become_method: sudo\n tasks:\n # Set our hostname\n - name: Set our hostname\n hostname:\n name: ansible-helloworld\n\n - name: Install htop\n apt:\n name: htop\n state: present\n update_cache: yes\n<\/pre>\n
ansible-playbook playbook.yml<\/code><\/p>\n
\nPLAY [all] *********************************************************************<\/p>\n
\nok: [18.188.72.168]<\/p>\n
\nchanged: [18.188.72.168]<\/p>\n
\nchanged: [18.188.72.168]<\/p>\n
\n18.188.72.168 : ok=3 changed=2 unreachable=0 failed=0
\n[\/code]<\/p>\nIdempotency<\/h3>\n
\n
ansible-helloworld<\/code>, set it to ansible-helloworld<\/code><\/li>\nhtop<\/code> isn’t installed, install it, otherwise do nothing<\/li>\n<\/ul>\nchanged=2<\/code>, indicating that two tasks changed something<\/i> on the server. Indeed, we set the hostname and then installed htop<\/code>. Let’s run it again:<\/p>\n
\nPLAY RECAP *********************************************************************
\n18.188.72.168 : ok=3 changed=0 unreachable=0 failed=0
\n[\/code]<\/p>\n
\nPLAY RECAP *********************************************************************
\n18.188.72.168 : ok=3 changed=0 unreachable=0 failed=0
\n[\/code]<\/p>\nupdate_cache: true<\/code> triggers an upgrade of the htop<\/code> application from the underlying repository. Strictly speaking this one line prevents the playbook from being idempotent, and if it were critical that the server didn’t receive any<\/i> apt-get<\/code> updates, removing update_cache<\/code> can help but would likely be insufficient.<\/p>\nFinal Thoughts<\/h2>\n
remote_user<\/code>, become<\/code>, etc. directives in our playbook, but that’s okay. If you remove them the playbook won’t work at all. Let’s add one last interesting twist to the playbook, and that’s installing more than htop<\/code> to the server. I happen to like Z Shell<\/a> (and even more, Oh My Zsh<\/a>) installed on my servers. So let’s install it.<\/p>\napt<\/code> task (separate from the one installing htop<\/code>) to install zsh<\/code>, but that seems a bit silly. Let’s use the with_items<\/code> capability like this:<\/p>\n\n - name: Install htop\n apt:\n name: \"{{ item }}\"\n state: present\n update_cache: yes\n with_items:\n - htop\n - zsh\n<\/pre>\n