Environments are all created via AWS Cloudformation and Chef Cookbooks, and control the standing up and maintenance of VPCs, VPNs, monitoring stacks, and Kubernetes clusters.
Applications are written and checked into Version Control. Once changes are detected, Jenkins on Kubernetes starts the build process, which creates a Docker image and ships it to the EC2 Container Registry. Developers can run their applications within the Regional Sandbox, which is free for all to use, with built in Guard Rails. Additionally, a User Acceptance Testing environment runs a Kubernetes install and creates a version of the application for Quality Assurance. Once tests pass and the team is satisfied, the Helm updates the application running in production. Regressions are captured in the pipeline and stopped at the door prior to getting to production.
Monitoring the applications allows engineers the ability to instantly detect issues with their deployments and alert the team. Binary monitoring is through Sensu, with metrics being handled by Prometheus. Logs are aggregated with Fluentd and shipped to Elasticsearch and S3.
The engineering team is able to Build, Test, Deploy, and Monitor their work without having to be involved in the infrastructure at all. This is because the tooling allows them to access the items they need to ensure the applications are behaving correctly and consistently performant.
Standing up new regions is a process that takes less than an hour, and delivers a new VPC, VPN if needed and Kubernetes cluster with various items already stood up inside of it, ready to be used. Teams with access can then start using it immediately, because it mirrors the existing environments.