Container-based application deployment is at its peak, as is the popularity of orchestration platforms like Kubernetes that form the underlying infrastructure for containerized applications. Because of its ability to orchestrate and automate application deployment, scaling, and management, Kubernetes has become the de-facto deployment and orchestration tool for cloud-native applications.
We cannot talk about securing container-based applications without talking about DevSecOps.
This methodology has risen to take advantage of the agility between operations and development teams to create end-to-end security mindfulness. It is an approach that aims to integrate security throughout the application lifecycle to distribute responsibility for security between DevOps and security teams. Traditional security methods don’t quite cut it for cloud-native technologies as they happen outside of development processes and primarily take into account the runtime state of resources. That’s why automated, built-in security must become a priority within containerized hybrid cloud environments.
Building a DevSecOps strategy requires not only the right tooling but also the right culture. In this post, we’ll introduce you to several principles to keep in mind when developing a Kubernetes infrastructure strategy or improving an existing one.
Key DevSecOps Principles for Kubernetes Infrastructure
When we talk about Kubernetes as a container orchestration platform, we are referring to the loosely coupled set of building blocks that provide mechanisms to deploy, maintain and scale container applications. Generally speaking, Kubernetes is composed of a set of clusters made up of nodes. There are one or more pods within each node, and each pod consists of one or more containers. The containers are used to host containerized application images.
When we talk about Kubernetes security, we are talking about system-level security mindfulness, combining the security of the application code with the security of the image composition and dependencies, and finally, the configuration of the surrounding Kubernetes infrastructure from the Kubernetes manifests for the deployment to the core components of the cluster itself. This system truly spans development, security and ops.
The goal of Kubernetes-based DevSecOps strategies should be to align those teams to implement consistent guardrails and automate them across the Kubernetes development lifecycle.
Infrastructure Automation With IaC
Infrastructure as code (IaC) is key to embracing DevSecOps, especially when Kubernetes is involved. Functionally, IaC makes it easier to systematically operate cloud and Kubernetes infrastructure through machine-readable, version controllable templates. In that way, it allows you to manage your cloud infrastructure in the same way that you manage your apps, services and other code and eliminates the need for manual setup as well as the creation of one-time scripts that you would otherwise need to perform infrastructure changes.
From a security standpoint, IaC templates such as Kubernetes manifests (or more modular components such as Helm charts and Kustomize files) allow for early and automated detection of security misconfigurations. Being able to enforce security best practices such as Kubernetes CIS benchmarks as early as possible is crucial and is enabled in large part by IaC.
Immutability
An important principle for Kubernetes, especially when leveraging IaC, is to understand and strive for immutability. In the realm of infrastructure provisioning, immutability means that system components from container images that have been deployed previously can only be modified at the source and not at runtime.
Functionally, this allows for faster iterations and more frequent updates during which these components are ripped down, updated, tested, verified and then re-deployed. This approach is also required to incorporate DevSecOps fully and to persist security best practices baked into build-time all the way to runtime. Kubernetes environments can be used to impose security regulations such as limiting the system-level actions that an application is allowed to execute, requiring CPU and memory limits, and blocking the launch of containers.
Integration With CI/CD Pipelines
CI/CD pipelines are the heartbeat of agile product organizations, allowing for the automated integration and delivery of software from code development to production deployment.
Continuous integration (CI) is the process of automating building and packaging software. Modern CI tools spin up ephemeral instances to create build artifacts and then spin them down once it’s complete. Those ephemeral environments allow for functional testing as well as security scanning—from checking Kubernetes manifests for misconfigurations to identifying known vulnerabilities.
Continuous delivery (CD) is also required to safely and quickly roll out changes to your running Kubernetes workloads by standing up test infrastructure and, in some instances, creating parallel deployment instances for easier switching in case of downtime.
When leveraging IaC and Kubernetes, CI/CD allows for not only greater efficiency and productivity but also a higher level of security when it comes to deploying containerized applications.
Key Kubernetes DevSecOps Culture Principles
Technology is core to implementing a Kubernetes-based DevSecOps strategy, but without the right people, processes, culture, and even KPIs, it can actually create friction and bottlenecks that it’s trying to avoid. Aligning development, security and operations teams, however, is not an easy feat, as these teams’ goals are often at odds with one another. Engineers want to work on high-impact projects. Security wants to avoid incidents at all costs, which often slows down other teams. Operations exist to deploy features and improvements as fast as possible.
Breaking those silos and fostering collaboration in the name of shared security responsibility is key to success. Here are some considerations to consider to build a DevSecOps culture:
People
Since DevSecOps demands breaking down silos between teams, people are the foundation. Security training and fostering security champions has been touted as the go-to solution for making security matter, but you can’t stop there. DevSecOps is a two-way street that requires bi-directional knowledge sharing in order to build true shared accountability for security. For cloud-native applications wherein technologies and software supply chains are constantly growing and changing, this is especially vital.
Whether you already have the right building blocks or are looking to add to your existing product and IT teams, these are some of the skills you need on your DevSecOps team:
- A knack for efficiency: Regardless of department, efficiency and automation are key to DevSecOps success. When manual work inevitably crops up, teammates with productivity mindsets will invest the time to make that repeatable in the future despite the temptation to just complete the task at hand.
- Balance individual focus and greater goals: DevOps aims to break down the development process into smaller components and processes, isolating individual outcomes at each phase. DevSecOps requires striking the right balance between security and efficiency. To do that in practice, priorities need to be set, recognized and constantly evaluated from the organization level to the individual contributors.
- Continuous learning: Although Kubernetes has been around for a while now, it’s valuable for everyone to be constantly learning new things when it comes to building the most performant and innovative products. The same goes for security. Staying on top of the latest vulnerabilities and policies is essential to keep your applications secure. Having natural curiosity is ideal, but with consistent processes for training and education, you can achieve the same outcome.
Building your team based on formal titles isn’t necessary for building the right culture; looking for individuals with these qualities will ensure that security becomes a mindset rather than a barrier.
Processes
The DevSecOps paradigm necessitates new processes or perhaps improvements to existing ones that prioritize security at each step.
- Development: As code is being written and updated, encourage individual contributors to incorporate security feedback via IDE extensions or CLI tools. By surfacing security best practices earlier, developers are able to address issues with the right context and quickly to prevent issues from progressing further. This is also a great way to foster continuous security education.
- Build and deploy: As you add checks to your CI/CD pipelines, ensure that all teams are aware of blocking criteria so that friction doesn’t arise when a build fails, or a deployment is blocked due to a critical misconfiguration or vulnerability. When issues do arise, make sure you have individuals responsible and on-call to help things keep running smoothly.
- Runtime: Even with the most mature proactive security guardrails in place, the work doesn’t stop at deployment. Having the right visibility and developing processes for when security issues are exposed in runtime is a big part of embracing DevSecOps.
- Feedback and Planning: Here, it’s important for all stakeholders to understand the security impact new features and updates may have. Security training and awareness are also crucial at this phase, as work done in this phase will determine the security coverage throughout the rest of the development lifecycle.
Setting the right processes in place ensures that everyone is on the same page and sets the foundation for security consistency and cohesiveness.
Key Performance Indicators (KPIs)
One way to integrate DevSecOps into teams’ day-to-day is to hold each accountable via shared KPIs. Metrics should take into consideration not only how secure applications are but also how quickly deployments occur and how reliable applications are. Here are some sample KPIs that touch all development, operations and security teams:
- Volume of production issues over time and by severity: Ideally, the number of misconfigurations in runtime should go down over time if issues are addressed earlier. By having end-to-end visibility, it should also be easier to prioritize higher severity issues, leading to fewer alerts and hardening infrastructure over time.
- Mean time to remediation (MTTR): Related, as the volume of issues goes down over time, identified vulnerabilities and misconfigurations should be resolved faster over time. A shorter MTTR also indicates a stronger CI/CD pipeline and institutional knowledge when it comes to infrastructure being deployed and its security expectations.
- Deployment speed and frequency: As you bake security measures into your DevOps lifecycle, be sure to monitor how frequently and quickly you’re deploying. At the end of the day, security checks are only valuable if you’re able to deliver updates, so striking the right balance by tweaking levels of control is key.
Because Kubernetes is such a dynamic and complex system, it’s even more crucial to implement a solid set of KPIs to help you assess your organization’s success internally and externally. DevSecOps is getting more popular as a means to avoid costly (both in resources and reputation) breaches. Bringing the right technologies, people and processes together to establish baselines and measure success over time are all necessary for any mature Kubernetes-based DevSecOps strategy.
Interested in taking a deeper dive into DevSecOps for Kubernetes environments? Download our DevSecGuide to Kubernetes!