How to Resolve Permission Issues in CI/CD Pipelines

DevOps teams are familiar with the ways security concerns and process issues can stall CI/CD operations. Operational hurdles that lead to miscommunication between team members and the broader organization are all too common in DevOps pipelines. One of the leading operational issues DevOps teams encounter are permission issues.

Permission issues are a seemingly small, yet significant, roadblock to smooth CI/CD pipelines. If you fail to address them, the result is a lack of cohesion between development and organizational objectives.

Here’s how to streamline these processes, boost security integration within the broader CI/CD framework, and maintain robust security postures.

Review Pipeline Tools

The DevOps cycle contains several tools with different access needs and permissions. Jeremy Hess, head of developer relations at secrets management platform Akeyless, calls this a “secrets sprawl.”

“The combination of proliferation and decentralization of secrets creates an operational burden, if not a nightmare,” Hess says. “For organizations that operate in both a cloud-native environment and classic IT infrastructure, a duplication issue is created due to having their own secrets managed with different tools and cloud-native solutions.”

There is also the risk of these tools exposing user credentials and permissions to malicious actors. For instance, configuration tools like Jenkins use plugins to determine access and artifact deployment. Thanks to communicating with other pipeline tools, credential details can be present in configuration details.

Developer passwords are not visible on the front end but are accessible from the system. Any user with “configure” permissions can request a credential and inject them into agents. The result is that AWS keys, git credentials, and passwords are at risk.

What to Do:

• The first step is to delete hardcoded secrets from CI/CD tool files.
• Distributing secrets between multiple tool config files also reduces the possibility of attack while easing developer and engineer access.
• Password managers are also a good choice, but validate them for security before implementing a solution.

Practice Least-Privilege Access

Access issues often create a lot of frustration among DevOps teams as they are forced to assign blanket access to the majority irrespective of the member’s role or job function. While this situation encourages rapid development, it creates massive security issues.

Balancing security with CI/CD needs is tough to get right. That’s where the principle of least privilege comes in. Team members receive access to secrets on a need-to-know basis. Note that this principle applies to everything from apps to systems and connected devices.

While most teams put this principle into practice, they leave their process intact. The lack of access audits, not the level of access, creates DevOps frustration.

What to Do:

• CISOs should regularly involve DevOps teams when reviewing access to mitigate issues quickly. Embedding a security role within every delivery team will mitigate access-related risks quickly. The security team member will have insights into risk-based access needs and can quickly approve or reject requests.
• Creating an access management repository will also remove any confusion related to role-based access. In addition, record time-based and task-based access permissions in the repository. The result is every DevOps team member will understand their access paths before projects get started. It allows them time to offer feedback and request one-off access to sensitive secrets.
• Review segmentation rules within your systems when assigning role-based access. Often, these rules will have to change depending on delivery timelines. Involving all stakeholders in these discussions is good practice and prevents frustration down the road.

Implementing one-time passwords (OTPs) and other authentication factors is also a good idea when validating user access to secrets.

Review OSS Projects

Open source projects are essential to industry growth but might pose security risks if access is mismanaged. Zan Markan, developer advocate at CI platform CircleCI, summarizes the problem aptly.

“Often the company that initiated and owns a popular OSS project continues to employ the core contributors,” Markan writes. “They will probably be joined by other regular contributors and maintainers that are not part of that company. And then there is everyone else — anyone who occasionally might contribute a fix or a feature.”

As user access grows, security concerns grow exponentially. Enforcing rigid user-based access is unrealistic and detrimental to an OSS project.

What to Do:

• CISOs or other security-focused managers must review whether sensitive secrets are being passed during builds for pull requests. Monitoring who can place requests and the roles that review them will ensure a good level of security.
• Establishing machine identity is also critical, given the degree of non-human access pipelines require. Authentication can be based on verifying whether client runtime container attributes match the characteristics of the valid container. Once authenticated, role-based access can take over, limiting access to secrets.
• It’s also a good policy to destroy containers and virtual machines (VMs) after they’ve been used.

Streamlining DevOps Operations Is a Top Priority

DevOps is critical to every organization’s success. Access and permission-related issues are common occurrences that are easily avoided. Reviewing access and establishing a balance between delivery and operational needs is essential to maintaining a competitive edge.

Read More HERE