Iván Rodríguez

Why do software deployments turn into nail-biting situations? It’s because deployments are treated as do-or-die scenarios. Once you’ve released something, there’s no going back. But most times, there’s little need for so much panic. For a recent project, we made deployments fun again. How? By using the Blue/Green deployment strategy: [1] We created two identical prod environments. One was named the blue and the other green. [2] The blue environment represents the current version. The green had the new changes. [3] Since we were on AWS, this meant deploying the application on separate EC2 instances backed by different ELB load balancers. [4] Initially, Route 53 routed all the traffic through the blue ELB. [5] To direct the traffic to the new version (green), we switched Route 53 to point to the green ELB instance. This is where we hit a problem. There was a bug in the green version. Like all good bugs, it was discovered only after we switched. But there was no panic. We immediately switched back to the blue version and a fix was made on the green version. This time, we decided to move ahead gradually. Using the weighted distribution in Route 53, we just shifted 10% of traffic to the green version. The rest of the traffic continued to the blue version. This approach is also called Canary Testing. When we found no issues, we ramped up the traffic to the green version - first to 25%, then 50% and then 100%. Finally, the blue version was decommissioned and green became the new blue. There were a couple of considerations: 👉 The change didn’t involve the database. Otherwise, we would have included that as part of the replicated environment. 👉 Yes, it was costlier to maintain two parallel environments. But the workload was critical and we needed a quick way to move to the previous environment. So - have you used Blue/Green deployments? And what has been your experience like?