神刀安全网

Packing the way to immutable infrastructure

Immutable infrastructure is much more than just another buzzword. In a world where configuration drift and non-reproducability is your enemy, immutability is your saviour.

In this post I explain the difference between mutable and immutable infrastructure, the challenge of mutable infrastructure and how I’ve transitioned my private servers to be completely immutable.

What is immutable infrastructure?

Mutability is the ability to change. Immutability is the inverse; the inability to change. Given those definitions we can derive that mutable infrastructure is one where you can change the system in-place and that immutable infrastructure is one where you cannot. However, this is all too abstract. Let’s take a look at a concrete example.

Why immutable infrastructure?

You’re responsible for the maintenance and uptime of a high traffic website. The website utilizes a typical N-tier architecture in which you have a number of load balancers, frontend web servers, application servers and database servers.

A CVE (Computer Vulnerabilities and Exposures) has just been disclosed for the haproxy load balancer, your load balancer of choice, and enables an active adversary to eavesdrop on TLS handshakes via specially crafted packets. The haproxy team have released a patch which fixes the vulnerability and you need to roll out this hotfix across all of your load balancers.

The mutable v.s. the immutable

With a mutable infrastructure you’d need upgrade haproxy on each load balancer in turn, whether that be manually via SSH or via some configuration management system such as Chef or Puppet.

Should you choose to upgrade haproxy manually you run the risk of human error and missing out critical steps; after all, you have a considerable number of load balancers to upgrade.

Similarly, should you choose to upgrade haproxy via a configuration management system such as Chef or Puppet you have zero guarantee that all of your load balancers are in the same known state. A hotfix-incompatible configuration file absent one server could be present on all others. Furthermore, the presence of a bug in the DSL you wrote for the hotfix could result in your load balancers being unable to converge and even cause them to fork into many different final states. What should have been a seemingly simple upgrade has devolved into a fully blown forest fire.

Mutable infrastructure is one where you can change the system in-place. Immutable infrastructure is one where you cannot.

With immutable infrastructure you never modify servers in-place since every change is a possible cause of configuration drift and non-reproducability. However, if our servers do not change, how can we apply the hotfix?

The answer is to produce a single machine image for all of your load balancers. Each machine image can be versioned; but all must be adequtely tested, pre-configured and fully deployable. Therefore, rather than upgrade haproxy in place, we can instead build a fresh image with the hotfix baked in.

The process for rolling out the hotfix is equally simple: destroy the virtual machine(s) for each load balancer and create new ones from the hotfix image. You may want to deploy the hotfix image prior to destroying the existing virtual machines for availability.

Furthermore, should anything happen to be wrong with the hotfix image you can simply rollback the hotfix by re-imaging your load balancer instances with the previous machine image. It is considerably harder, if not impossible, to achieve this with mutable infrastructure.

Building images with Packer

Packer , built by Hashicorp , is a tool for creating configuration management, platform and cloud indepedent machine images. When combined with Terraform it becomes possible to automate the entire process of provisioning, building and deploying infrastructure from base image (i.e. Debian 8) to cloud provider (i.e. Amazon Web Services) and everything else from auto-scaling to DNS and email configuration along the way.

By deploying all instances of a server from the same machine image, whether they be a load balancer, frontend, application or database server, you are guaranteed that at every deploy all servers start with the same, well known initial state. The same cannot be said for mutable infrastructure as server configurations can drift over time.

Furthermore, whilst Packer works great with both Chef and Puppet, immutable infrastructure you rids you of the need for complex client-server configuration management systems, enabling you to provision your images with simpler tools such as Chef Solo, Ansible or Salt.

An example Packer file

The following figure shows the entire packer file that I use to build the machine image(s) for my website.

First I define an API token variable, necessary for packer to authenticate with my DigitalOcean account. Next I declare a single builder which builds a fully configured Debian machine image on DigitalOcean (I could also build images for Amazon Web Services, Google Cloud, Vagrant, etc with additional builders) followed by a sequence of provisioners that tell Packer how to provision the base image.

{     "variables": {         "do_api_token": "insert-api-token-here"     },     "builders": [{         "type": "digitalocean",         "api_token": "{{user `do_api_token`}}",         "image": "debian-8-x64",         "region": "lon1",         "size": "512mb",         "snapshot_name": "grobinson.net-{{timestamp}}"     }],     "provisioners": [{         "type": "shell",         "inline": [             "sleep 30"         ]     }, {         "type": "ansible",         "playbook_file": "../ansible/grobinson.net.yml",         "groups": "webservers"     }] }

All I have to do is run packer build . Packer then uses the DigitalOcean API to create a fresh Debian 8 instance with my SSH key, runs the grobinson.net.yml Ansible playbook to provision the machine, the API to stop the instance, create a snapshot and finally terminate the instance.

Packing the way to immutable infrastructure

The result is a machine image that I can deploy to any DigitalOcean datacenter on an instance of any size at the click of a button. I look forward to automating this final step with Terraform.

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » Packing the way to immutable infrastructure

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
分享按钮