Shift Left and GRC
When I first heard about ‘shift left’ a few years ago, I had hoped that after only 20ish years, my assembly programming module at university might finally prove to be useful. That’s a bit of an ‘in joke’ because only those with exposure to low level programming will know shift left in that case means something quite different to its current cybersecurity incarnation - multiplying by the power of 2. That meaning is not, however, the cause of the varied interpretations I see in the cybersecurity industry which I shall address in this article. In addition, ‘shift left’ seems to be an approach that GRC practitioners are sometimes unfamiliar with. Associated terms like CI/CD (Continuous Integration / Continuous Deployment), pipelines, pull requests and policy-as-code may leave some GRC folks feeling like they’ve walked into a sprint meeting they weren’t invited to. In this article I’d like to give an overview of the key concepts to help explain this important area of risk.
Before I get into the details, if you’re already familiar with DevSecOps practices and terminology I’d ask that you bear with me. I think there are many GRC practitioners who are familiar but some who aren’t and I’d like to ensure we’re all on the same page before moving forward by briefly stating what ‘Shift Left’ is and what it’s not.
Shift Left – What it is and isn’t
The term has been around since 2001, however from the mid 2010s it started to gain increased prominence due to the adoption of agile development methodologies and the DevOps movement – a cultural shift where development and operational teams collaborated more closely to enable faster release cycles. As a result of this change Dev teams started incorporating automated testing (primarily for quality and functionality) into the deployment process. The case for also embedding security testing early in the development lifecycle was also clear – fixing bugs / vulnerabilities after deployment is more insecure and costly than fixing them during development and hence DevSecOps came to fruition. So the term ‘shift left’ is very closely related to DevSecOps with ‘shift left’ being the principle of testing code early and DevSecOps being the cultural security shift which aims to deliver on that principle.
Now something ‘shift left’ is not – it’s not defining security requirements earlier in the SDLC. The principle of defining security requirements during ‘Planning’ or ‘Design’ phases of the SDLC predates DevOps and DevSecOps. Although this principle is sound and should still happen, it is not the same thing as incorporating automatic code testing during development and deployment.
Shift Left in Practice
One of the technological advancements which facilitated DevOps / DevSecOps culture is the use of CI/CD pipelines. If you’re unfamiliar with CI/CD pipelines, they’re automated workflows which define code testing, integration and deployment steps triggered by events like code updates – they directly enable the rapid code release associated with DevOps. Typical features include workflow automation, secrets management, role-based access control, integration with third party security tools, artifact storage and audit logging. CI/CD pipelines are often an additional service or module associated with Source Code Management (SCM) vendors like GitHub or GitLab. So in the case of GitHub for example, the CI/CD solution is GitHub Actions. Other solutions in the space include GitLab CI, AWS CodePipeline and Jenkins. For the purpose of illustration I’m going to be using GitHub Actions.
So CI/CD pipelines can be used to test and deploy changes to code and later in this article I’ll be making changes to Terraform (HCL) – an example of Infrastructure-as-code (IaC) which I’ll be using to deploy AWS resources. HCL can also be used to configure cloud infrastructure across the other major cloud platforms and is extremely powerful – enabling the deployment, configuration and destruction of entire computing environments through simple, declarative, intuitive code.
CI/CD Pipeline Risk
For example, the single command ‘Terraform Destroy’ will destroy all cloud resources it has previously deployed. In theory this could include entire computing environments or platforms; networks, servers, applications, containers, databases - everything.
Now there are a number of dependencies which must be met for this to happen. The CI/CD pipeline must authenticate to the cloud account (typically via OIDC) and the pipeline requires credentials assigned the necessary privileges within the cloud account. The power of IaC, is that it can efficiently deploy resources using code improving automation, repeatability, testability and recoverability – hence IaC is generally considered best practice for cloud deployments. However, this power comes with risk – in particular CI/CD credentials ending up in the wrong hands, could be devastating.
So what are the risks associated with the use of CI/CD pipelines? There’s no need for me to reinvent the wheel as OWASP already provides plenty of details, however to summarise pipeline authentication, privilege management, flow control and monitoring should be areas of focus. OWASP also publishes a CI/CD Security Cheat Sheet which goes into far greater detail. I previously wrote an article, Infrastructure as Code and the Upside Down, which highlights the risk and I end the article explaining that what’s most important is a “mindset shift” - GRC teams must focus their attention on CI/CD pipelines and not limit activities to cloud accounts / resources - I repeat that point again here.
Policy-as-Code
Policy-as-code is an approach to securing IT infrastructure which defines rules (policies) and compliance through machine readable code. There are a number of commercial and non-commercial approaches which I’d categorise as follows:
1. General-purpose policy engines
- Open Policy Agent (open source) using Rego (programming language developed for policy evaluation)
2. Pipeline / Platform Integrated testing tools
- HashiCorp Sentinel (Hashicorp)
- Conftest (open source)
3. IaC scanning and validation tools
- Checkov (open source)
- CloudFormation Guard (AWS)
- Terrascan (Tenable)
These categories aren’t exhaustive and others may use different categories, but my categorisation should provide some structure as well as a view of what sits under the policy-as-code umbrella. One thing I would mention is I’ve seen other types of policy or configuration monitoring / enforcement described as policy-as-code. For example, AWS Service Control Policies, AWS Config or Azure Policy. For the purpose of this article, I don’t consider these approaches to be policy-as-code as they don’t ‘shift left’ in the way I described earlier; instead, they enable ‘run-time’ monitoring or enforcement in cloud platforms. Although using such technologies may be important to prevent drift, that’s not ‘shift left’ capability as I’m discussing here. Policy-as-code should enforce the deployment of security controls and secure configuration prior to deployment.
You’ll note from the list above that Open Policy Agent (OPA) is the dominant open source and freely available policy engine. Conftest acts as a wrapper to provide CLI interaction passing structured data (e.g. JSON) files for policy evaluation with the OPA engine. In the section below, I’ll demonstrate the use of Conftest in a GitHub Actions CI/CD pipeline to control the deployment of AWS resources. Before I illustrate policy-as-code in action, I’d like to set out my thoughts on policy-as-code compared to the traditional cybersecurity policy, GRC professionals might be more familiar with.
Commentary on Policy vs Policy-as-Code
As someone who has a fair amount of experience writing and reviewing cybersecurity policies, I would make an important distinction between the traditional understanding of the word ‘policy’ in cybersecurity versus policy in the context of policy-as-code. Cybersecurity policy, in the traditional sense, is meant to be high level and should ideally use principle-based language so that it’s easy for technical and non-technical personnel (including executives) to understand and to affect change – which is the whole purpose. To affect change, it’s imperative policy is approved by executives and should therefore be subject to executive review – which will only be frustrated by technical jargon including platform specific controls. By comparison - and perhaps unsurprisingly - policy-as-code is…. code. This means that it operates at a different level of abstraction to traditional policy and although policy-as-code can be written to support different cloud platforms, it must still be customised for each. You can’t simply code ‘enforce least privilege access’ for example. You have to define what least privilege looks like in the context of AWS IAM Policy or an Azure RBAC Policy for example. In short, policy-as-code is technology (or platform) dependent and I would argue, for this reason, would be better termed ‘standard-as-code’, ‘controls-as-code’ or something similar. Despite forming the acronym CAC, I’d still go with ‘controls-as-code’.
I have further concerns with the term policy-as-code or at least its use. As bureaucratic and uninteresting as traditional policy development may be, it is essential to any security function to agree with the business (and IT) on the baseline security controls which represent the organisation’s minimum risk tolerance and ensure these are documented somewhere. How else are personnel going to know what is or is not allowed? No variety of policy-as-code could possibly support this use case; it’s not designed to. Despite this, I’ve read plenty of commentary which states policy-as-code will replace the need for ‘documents and manual checks’. Policy-as-code can reduce the need for manual checks but it does not replace the need to define what’s acceptable and document it so that people can refer to it. Suggesting otherwise, feeds a technology focused narrative which is divorced from the realities of human behaviour and organisational governance. In my view, legitimate frustrations with traditional policy and its reputation for having little meaningful impact while sat on a legacy SharePoint site, would be best directed toward other areas of governance. More often than not, policy’s ineffectiveness is a result of an unwillingness to measure against it or enforce it… but that’s a discussion for another article.
Policy as Code in Action
I've created a mirror site for the GitHub repo which I'll describe below. It can be accessed here and has the same files (without the access to my AWS accounts).
Below, you can see Terraform files structured to represent different types of resources to be deployed in an AWS Account.
In the file structure on the left, you can see there are directories for two categories of resources – one for compliant resources and one for noncompliant resources. In production, you wouldn’t have this structure however I’m using it to illustrate the capability of Conftest so you can observe the policy tests both failing and passing.
Above on the left, you can see a ‘compliant’ version of an S3 bucket for my-public-website (using AES256 encryption) and one for the ‘noncompliant’ version of the same bucket which uses DSSE encryption (for the sake of this demo I don’t want to allow DSSE).
Within the repo you can also see Rego files which perform a number of checks against the Terraform code. These files could be structured to match a policy or standard to provide easy referencing to that oh so important documentation I mentioned earlier.
Rego is not intuitive and as I left my engineering days behind me some years ago, I’m not best placed to explain code in detail but the principle is that you define rules which check structured data to ensure it meets policy. In this case, Terraform creates a JSON representation of the plan – the intended changes. The JSON file is evaluated against each deny rule and if one evaluates to true, it indicates a policy violation; Conftest then returns a failure and the pipeline is terminated. So far I’ve explained the programmatic logic and how the Terraform HCL is evaluated against Rego but I haven’t yet demonstrated how it’s applied in the pipeline itself.
As described earlier the pipeline is defined in GitHub Actions through a YAML file (above) which defines the sequential steps for testing and deploying the code. In this case the workflow defines the steps for deploying the AWS resources using Terraform. Importantly you can see how the workflow calls Conftest using the Terraform JSON file and Rego policy files as input (under the ‘policy’ directory called with -p). This is where the policy rules are applied in the pipeline – this is ‘shift left’ in action.
In the case of the noncompliant S3 bucket, you can see that the workflow fails illustrating how the deployment will fail (or alert if configured) automatically if changes conflict with policy. Other policy rules also fail in my demonstration e.g. RDS encryption and tagging.
In the case of the compliant resources, Conftest test passes allowing the deployment of a number of AWS resources including the S3 buckets defined in the HCL above.
Of course, the example only includes a small number of basic policy tests but with more time and better programming capability than my own, you can see how an entire policy set could be defined for each cloud platform to enforce technical controls. This demonstrates true ‘shift left’ capability in action where cloud resources and infrastructure are tested against policies / controls prior to deployment.
Conclusion
Earlier in this article when I described CI/CD pipelines, I highlighted the associated risks referencing OWASP. However as complete and beneficial as any list of risk / threats may be, they don’t always translate into a clear understanding. If you’ve had limited exposure to CI/CD pipelines, you’ve hopefully gained a more visceral feel for that risk - CI/CD pipelines really do represent the keys to the kingdom when it comes to cloud. Please do check out the CI/CD Security Cheat Sheet for more details on how to secure your CI/CD pipelines.
In addition, through this article I’ve illustrated what true shift left capability looks like – policy-as-code offers an opportunity for GRC functions to engineer controls directly into pipelines preventing insecure and noncompliant resources from being deployed. I do think that the term ‘policy-as-code’ is somewhat of a misnomer but I’d hate to be one of those GRC folks who spends entire meetings arguing about terms and definitions (too late!)
The important points are: i) policy-as-code offers a programmatic (and therefore flexible and powerful) approach for implementing security requirements / controls / policies directly into the pipeline where computing resources are built. ii) by adopting this technology organisations can reduce their risk by preventing the deployment of less secure (noncompliant) resources rather than waiting for a detective control to identify a misconfiguration and iii) policy-as-code does not replace the need for agreeing risk tolerance with the business, documenting it and communicating it in traditional policies.
I hope this article aids in your understanding of the importance of ‘shift left’, CI/CD pipelines and what GRC functions should be doing in this space.
