“Meetup, seasoned engineers and architects from GovTech’s cloud computing teams will attempt to de-mistify the Government’s work on Cloud Templates and Infrastructure-as-Code (IAC), and take you through our experiences thus far in implementing IAC across different government agencies and services. “
Details from GovTech on their meetup page
GovTech held their third iteration of their STACK-X meetup on 3rd October 2019 at their office in Mapletree Business City, 10 Pasir Panjang Road.
I have been following the GovTech’s STACK events for some time but have not really found the right event to attend. Being infrastructure-focused, developer-centric events did not really pique my interest. At least not until I caught wind of the third STACK-X meetup title.
“Cloud Templates & Infrastructure-As-Code”
Cloud-related? Check. Infrastructure-related? Check. It was an easy decision to make time for this event with topics I am very interested in. I have been dipping my toes into Cloud computing recently and was curious to learn more about infrastructure-as-code.
The meetup started at 6.40pm with an opening speech by Deputy Chief Executive, Chan Cheow Hoe. This was followed by a sharing from Hunter Nield, Samuel Loh and Tang Bing Wan.
You can view the sharing from Hunter and Samuel here:
The recording from Bing Wan’s session was not shared to the public as he had touched on a number of “secret sauce” that GovTech utilizes for Government on Commercial Cloud (GCC), and the sharing might be deemed too sensitive to be out in public.
What is Infrastructure as Code (IaC)? What problems do IaC solve? What are the benefits of IaC?
IaC “codifies” managing and provisioning of infrastructure by using API calls to replace interaction with hardware and GUI configuration.
Deployments across different environments tend to drift due to many reasons such as human error, unique requirements and differing architects. In addition to inconsistent deployments, changes to infrastructure can be undocumented at times (maybe all the time). It can also be extremely time consuming to migrate large number of workloads into a new environment.
IaC to the rescue! By defining configuration, deployment and management within templates of API calls, engineers can achieve consistency across multiple environments by enforcing the same template.
By making incremental changes to these templates, version control and documentation can be automated with common versioning tools (e.g. Git).
Takeaway 1: HashiCorp’s Terraform as preferred IaC tool
GCC provides access to Amazon Web Services, Azure and Google Cloud. It is not surprising that GovTech have gone with HashiCorp’s Terraform as their preferred IaC tool for GCC instead of tools like AWS CloudFormation or Azure Resource Manager.
Terraform is cloud-agnostic and ideal for GCC’s multi-cloud design.
Takeaway 2: Advocacy of Immutable Infrastructure
Immutable Infrastructure is an approach in which servers are never modified after they’re deployed. Any changes to servers are made to a common base image and relaunched.
IaC enables extremely fast infra configuration and deployment. By patching the golden image that the IaC template uses, there is no longer a need to patch existing servers. If your entire server have been “codified” into an IaC template, utilising the updated golden image only requires you to spin up a new instance of it and destroying the outdated one.
Realising immutable infrastructure should make security folks extremely happy. Monitoring for ANY changes to an infrastructure is way easier than monitoring SPECIFIC malicious changes.
Takeaway 3: Potential of Policy as Code
Since IaC “configures” your infrastructure from the beginning before deployment, there is an opportunity to inspect and audit for policy violations up-front.
Enter Policy as Code (PaC). PaC tools can be used as an initial validation step to prevent security incidents before they happen. Additionally, they can also audit existing environments periodically or in real-time.
GovTech is exploring tools such as CloudCustodian, Open Policy Agent and InSpec.
Takeaway 4: GitOps to replace change management meetings
GovTech advocates using GitOps to record discussions around pull requests and trigger webhooks. With this functionally, GitOps can trigger automation to run tests, build artifacts and create IaC deployment plans.
All discussions, test results, decisions and changes are automatically documented in Git pull request discussion threads. All past and present infrastructure configuration are versioned in Git.
Takeaway 5: Sys Admins need to adopt Developers mind-set
With IaC, PaC and GitOps, a lot of the system administrator functions when implementing configuration changes should be “shifted left”, done at script/code level, and not manually through Jumpboxes/Bastian Hosts/Jumphosts.
Final Thoughts
STACK-X Meetup #3 is truly an eye-opening experience with GovTech experts sharing their experiences and thought processes going through the journey to become a cloud-native organisation.
Huge thanks to Hunter, Samuel, Bing Wan and GovTech for this learning opportunity.