How to Protect Your CI/CD Pipeline

Last fall, the Open Worldwide Application Security Project (OWASP) announced a new project focused on the top 10 security risks for CI/CD pipelines. (CI/CD stands for continuous integration and continuous delivery/deployment, a sort of ‘perpetual motion’ approach to software development.)

The top 10 reflects how widespread CI/CD have become—and how vulnerable its pipelines can be due to technology stack complexity, increased use of automation and infrastructure as code, and more third-party integration. It also highlights another case of traditional cybersecurity practitioners taking on expanded responsibilities as security operations and cloud security continue to converge.

Least privilege, Zero Trust

Based on our assessment, OWASP’s Top 10 CI/CD Security Risks fall into three general categories: access and credential risks; integration and dependency risks; and configuration risks. If there’s a recurring message, it’s that organizations should adopt least-privilege principles wherever possible and embrace the spirit of the Zero Trust approach.

Access and credential risks	Integration and dependency risks	Configuration risks
Inadequate identity and access management Insufficient credential hygiene Insufficient pipeline-based access controls (PBAC)	Dependency chain abuse Poisoned pipeline execution Ungoverned usage of third-party services Improper artifact integrity validation	Insufficient flow control mechanisms Insecure system configuration Insufficient logging and visibility

Access and credential risks in the CI/CD pipeline

Inadequate Identity and Access Management

The open, continuous nature of CI/CD adds up to more people and machines participating in the development ecosystem across its phases and stages. Even one compromised identity has the potential to cause fairly serious damage.

According to OWASP, identities can suffer from any number of weaknesses—from too-broad permissions to lapsing out of date. It recommends continuous mapping of all internal and external identities, removing unnecessary permissions, and not granting blanket permissions to all users or large groups. Stale accounts should have a defined shelf life. Local accounts (ones that aren’t managed centrally) should be prohibited, along with the use of personal or non-enterprise email addresses, self-registration, and shared accounts.

Insufficient credential hygiene

Bad actors prize credentials for the access they give to high-value resources and the opportunities they afford to deploy malicious code and artifacts. They can be exposed any number of ways in the CI/CD pipeline.

If pushed to a software change management (SCM) repository branch, credentials can be read by anyone with access to the repository. They can be left out in the open when used insecurely in build and deployment processes or image layers and may also be printed to console outputs.

OWASP’s recommendations are to rotate credentials regularly, apply the principle of least privilege always, use temporary credentials instead of static ones, and remove any secrets or credentials from artifacts once they are no longer needed.

Insufficient pipeline-based access controls (PBAC)

CI/CD execution nodes access both internal and external systems and resources. Bad actors can use that access to disseminate malicious code, exploiting all the permissions associated with the pipeline stage the code is running in.

To remedy this, OWASP says nodes with different levels of sensitivity or resource requirements should not be shared by multiple pipelines. It also advises a form of least privilege to ensure every pipeline step can access only the secrets it needs, and no others. Once the pipeline is executed, the node should be restored to its “pristine state”, and every node should have fully up-to-date security patches.

Integration and dependency risks in the CI/CD pipeline

Dependency chain abuse

Code in the CI/CD pipeline may depend on other code to function, and those dependencies can be used to import malicious packages into the development process.

OWASP has identified four common dependency chain attacks: dependency confusion (naming malicious packages the same as legitimate packages); dependency hijacking (replacing legitimate packages with malicious ones); ‘typosquatting’ (naming malicious packages variations of common misspellings of popular legitimate packages); and ‘brandjacking’ (camouflaging malicious packages as trusted brands).

OWASP says package-fetching from the internet or untrusted sources should be prohibited, and all private packages should be verified as being within the organization’s scope. Any installation scripts should run without access to secrets or sensitive resources, and the names of internal projects should never be published to public repositories.

Poisoned pipeline execution

The whole CI/CD pipeline can also be ‘poisoned’ with malicious code if an attacker gains access to source control systems. Pipelines using unreviewed code are especially susceptible.

Organizations should restrict pipelines that run unreviewed code to isolated nodes, and all CI configuration files should be reviewed before the pipeline runs. OWASP recommends limiting the pipelines that can be triggered on public repositories and protecting sensitive pipelines with branch-protection rules in the SCM. And if users don’t need certain permissions in the SCM repository, they shouldn’t have them.

Ungoverned usage of third-party services

Third-party access to an organization’s internal resources is built into the CI/CD model, and a compromised third party can be used to penetrate the entire ecosystem. To illustrate the point, OWASP gives the example of a third party with write permissions being exploited to push code to a repository that then triggers a build and infects the build system.

Recommended remedies include:

• Imposing governance controls on third-party services throughout the lifecycle
• Vetting third parties before granting access to ecosystem resources—and assigning least-privilege permissions
• Maintaining continuous visibility of all integrated third parties including how they are integrated, the permissions they have (and what they actually use), and their behaviors
• Having a quick and easy way to offboard third parties and deprovision their access if it is no longer needed

Improper artifact integrity validation

Artifacts in the CI/CD pipeline are pieces of code or data used in the build process that get saved or remain accessible to the system. If they aren’t validated, they can be used to push malicious code or artifacts into the pipeline.

Here again, OWASP has a few recommendations: validate the integrity of all resources from development to production; incorporate secure code-signing technology; deploy artifact verification software; and monitor for ‘configuration drift’ in CI/CD assets. These apply equally to internal and external/third-party artifacts.

Configuration risks in the CI/CD pipeline

Insufficient flow control mechanisms

Many of the risks summarized so far are exacerbated by the fact that once an attacker has access to some part of the CI/CD process, they have virtually free rein to push malicious code wherever they want due to a lack of flow control mechanisms.

OWASP’s recommendation is to ensure that no person, machine, or program can “ship sensitive code and artifacts through the pipeline without external verification or validation” by:

• Setting protection rules for branches that host sensitive or production-related code
• Limiting the use of auto-merge rules
• Preventing pipelines from being triggered without approval or review where possible
• Verifying artifacts and managing how they flow through the pipeline
• Detecting and preventing code drift and inconsistency

Insecure system configuration

Sub-optimal configurations can be a significant source of risk in CI/CD pipelines. Attackers can take advantage of self-managed assets that are out of date, unpatched, or have admin privileges installed in the operating system. They can also exploit systems with excessive access permissions, inadequate credential hygiene, or insecure configurations related to authorization, logging, and other functions.

Organizations should address configuration vulnerabilities by keeping a complete, current inventory of systems and versions, and checking regularly for known vulnerabilities. OWASP also recommends least-privilege access and permissions along with periodic configuration reviews.

Insufficient logging and visibility

As OWASP puts it: “The existence of strong logging and visibility capabilities is essential for an organization’s ability to prepare for, detect and investigate a security related incident.” IT cybersecurity tends to do a good job of this, but engineering systems and processes don’t have the same protections.

This can be addressed by maintaining a current map of the environment, setting up appropriate human and programmatic log sources, aggregating and correlating logs from different systems for intelligence-gathering, and using alerts to flag anomalies and potential malicious activity.

These kinds of actions can be taken with extended detection and response (XDR) capabilities, centralizing logs and telemetry across cloud security and AppSec tools. Break down cloud security siloes and leverage information from those tools as part of a broader XDR approach to hunt, investigate, and respond to theats in a single platform.

Protect your CI/CD pipeline

OWASP’s top 10 lists are invaluable resources for cybersecurity teams, especially those coming from the traditional security operations center (SOC) world and are now finding themselves on the hook for application security, DevOps protection, and cloud defenses.

The CI/CD pipeline top 10 is a welcome addition to the set, reinforcing unequivocally that strong checks and balances, tightly restricted permissions, and an overall Zero Trust mindset are crucial to keeping enterprise assets and data safe while capitalizing on the open and iterative nature of the CI/CD model.

These measures align with Trend Micro’s position that a Zero Trust approach is critical in distributed development environments. Organizations need Zero Trust Secure Access and continuous assessments of identity and device-related risks to provide a secure foundation for fast-moving DevOps teams. Trend Vision One helps drive secure development by including identity logs and user behavior as part of investigations, scanning artifacts for vulnerabilities and malware, and more. It’s all part of our commitment to help security teams take on new responsibilities in the software-created enterprise and support businesses’ strategic use of the cloud.

Next steps

For more Trend Micro insights into CI/CD pipeline security, check out these resources:

Read More HERE