Dev is Prod
Everyone’s shifting left
It seems like everybody is shifting to CI/CD. Hopefully, they are involving security (‘shifting left’ on security), and security is engaging.
Discussions around security and CI/CD focus on security of the product under development, rather than details of the development tooling.
DevSecOps focuses on mitigating bugs and security problems early in the development lifecycle at best, and vulnerability scanning container artifacts as part of a deployment pipeline at worst (while still a net positive step). The data shows shifting security left in this fashion is a solidly positive impact for organizations 2.
Prod is also shifting left
As new organizations adopt these processes and technologies, a fundamental change occurs that security teams tend to overlook: the boundary of the ‘production’ system changes.
If a developer can commit code to a git repo, and have that code show up in ‘production’, then the developer and her systems is ‘in prod’.
In organizations with regulations (PCI, ISO, FISMA, etc.), the ‘boundary’ is a crucial thing to define properly. ‘Dev’ historically happens outside of the boundary of ‘production’, with minimal security controls, no monitoring, and none of the invasive ‘security’ controls that ‘production’ has - generally stemming from legacy security operations models.
It’s not a good idea to drag those legacy tools and lockdowns into the development space (they’ve not been there for a reason!), but it is crucial for security and risk management to understand that ‘production’ now includes the entire development environment.
If a developer can commit code to a git repo, and have that code show up in ‘production’, then the developer and her systems is ‘in prod’. This is irrespective of the contents of the pipeline - code review gates and other best practices are critical details for shifting product security left, but, don’t change the fact that the production system boundary is now inclusive of the deployment/delivery pipeline.
An adversary does not draw distinctions between arbitrary network segments or systems - if an adversary can easily access and compromise a development pipeline or CI/CD infrastructure to gain access to sensitive data or systems, just because it’s not marked as ‘production’ on the system diagram doesn’t mean it’s not in scope.
This risk is an aspect of supply chain security, which has come barreling to the forefront in the recent months, with the Solar Winds 1 hack.
Catching SecOps up to this new world
Given the risks involved, the lack of exposure to these CI/CD pipelines and toolsets amongst security professionals is a significant capability gap. As their organizations adopt CI/CD, SecOps teams should be aware of the tooling in use in ‘dev’, and understand its features and security controls, how to handle incident response within the system, and how to monitor and manage the associated risks, without putting an unnecessary burden on the development team.
As an example, the basic features and capabilities SecOps should look at for each system include:
Authentication & Authorization (AuthN/AuthZ)
- RBAC
- RBAC PR/Merge granularity (can control who merged PR)
- Separation of concerns enforcement
- ‘System’ credentials in the system (e.g. to build cloud infra or download git repo changes)
- MFA
- SSO / Federated Identity
Vulnerability Management
- Artifact signing
- SBOM generation/validation
- SBOM film scanning
- Code scanning (quality/bugs/vulns)
- Certificate / crypto material management
Incident Reponse / Forensics (DFIR)
- Alerting
- Logging (security logging - who did what, not build logs)
- Forensic artifact availability (e.g. RAM dumps)
Compliance
- HIPAA BAA
- PCI Attestation 3
- SOC/ISO Cert
- FISMA / FedRAMP / DOD
Meta Security
- Security Guide
- Security Checklist
- Bug Bounty / Responsible Disclosure
- Security POC with secure communication capability
- Length of support contract / version lifetimes