Gabe Ortiz

Systems Engineer. Charcutier.

Read this first

Software Ate My Infrastructure: Two Years on AWS with Ansible, Terraform and Packer part 1

Previously: read about our first year

Agari Data has made significant investment into infrastructure as code. Almost two years into this project, we’ve learned some lessons. Our efforts have already yielded dividends by increasing engineering velocity while maintaining infrastructure reliability. Additionally, our new toolset enables more experimentation, assured in the knowledge that we’re never more than a few commands away from being able to roll back changes or re-deploy with zero data loss.

Our automation toolset makes managing cloud-based infrastructure orders of magnitude faster and more repeatable, but with more powerful tools at our disposal comes complexity, technical debt, and raised barriers to entry for simple changes. Managing state, deploying secrets, working with distributed data stores, and using tools that are themselves in a state of rapid development (e.g. Packer...

Continue reading →

Ansible and Terraform at Agari: An Automation Journey

Snowflakes and Early Automation Efforts

At Agari Data, as part of our mission to solve phishing we deal with data at scale. We’ve chosen AWS to help us move quickly, making sure our infrastructure is as agile as we are.

Unfortunately, in the beginning, we treated AWS instances much the same way as physical servers, each configuration was lovingly hand-crafted, packages were installed at the command line with artisanal care. We were in short, server-huggers.

That had to change. The only way we were going to truly leverage the strengths of a cloud infrastructure was to embrace the world of infrastructure-as-code and ephemeral servers that could be created and destroyed at will. Our first attempt at this involved an external contractor, Chef, berkshelf and many tears. The project was never entirely completed and left us in a worse state than pre-automation, particularly because there...

Continue reading →

Preserving source IP — Postfix behind Amazon ELB

AWS_ELB_Postfix.png Why run Postfix behind an ELB instead of doing DNS round-robin? Lots of reasons. It’s a lot easier to autoscale, for one. You get increased control over load distribution, you don’t have to wait for DNS propagation. But wait, won’t I lose the source IP since we’ll have to do TCP load balancing? No, of course not — that was a rhetorical question. Here’s how it’s done.

Amazon ELB supports Proxy Protocol. This will write a TCP header that can later be read by applications which support it. Luckily for us, Postfix supports Proxy Protocol in versions 2.10 and above.

Unfortunately, the AWS web console doesn’t have an easy way to enable Proxy Protocol, but it’s pretty easy to accomplish with the AWS CLI tools. First, we need to create a policy:

aws elb create-load-balancer-policy --load-balancer-name YOUR-ELB-HERE --policy-name EnableProxyProtocol  --policy-type-name ProxyProtocolPolicyType

Continue reading →