Search
What Is Ungoverned Usage of Third-Party Services?
5 min. read
Ungoverned usage of third-party services — an OWASP Top 10 CI/CD Security Risk — involves the unregulated integration of external services into an organization's CI/CD systems. The simplicity of third-party integrations often bypasses the need for special permissions or approvals. A resulting lack of oversight and visibility hinders the implementation of role-based access control (RBAC) within CI/CD systems, exposing the organization to a host of security vulnerabilities. To counteract risk, AppSec and DevSecOps teams must implement governance controls throughout the lifecycle of third-party service usage — from approval and integration to ongoing usage monitoring and eventual deprovisioning.
CICD-SEC-8: Ungoverned Usage of Third-Party Services Explained
Ungoverned usage of third-party services in the context of continuous integration and continuous deployment (CI/CD) refers to the unchecked and unregulated use of external services granted access to an organization's CI/CD systems and resources.
DevOps teams frequently leverage third-party services for tasks such as code analysis, artifact storage, or monitoring. The simplicity of these integrations and the immediate value they provide make third parties essential to daily engineering operations. But as the diversity of integration methods increases and their associated complexities decrease, organizations face the challenge of maintaining control.
Challenges in Controlling Third-Party Integrations
Consider GitHub SaaS, a common source code management (SCM) system, for example. Third-party applications can connect to GitHub through several methods — GitHub applications, OAuth applications, access tokens, SSH keys, webhook events. Each method requires only seconds to minutes to implement and can grant third parties a wide range of capabilities, which can extend from reading code in a single repository to administering the GitHub organization.
Integration of Third Parties in Build Systems
Build systems also facilitate easy integration of third parties. Adding a few lines of code to the pipeline configuration file or installing a plugin from the build system’s marketplace can integrate third-party functionality. The build process then imports and executes the functionality, providing full access to the resources available at the pipeline stage.
But the lack of governance and visibility around third-party implementations can prevent organizations from maintaining role-based access control (RBAC) within their CI/CD systems. The permissive nature of third parties means that an organization's security is only as strong as the security of its third parties. Minimal governance and diligence around third-party implementations, coupled with insufficient implementation of RBAC and least privilege, can significantly increase an organization’s attack surface.
Third-Party Services Defined
Third-party services streamline complex tasks, enabling DevOps and AppSec teams to focus on core competencies. Source code repositories like GitHub or Bitbucket offer version control, while Jenkins or CircleCI automate builds and tests. Container orchestration via Kubernetes or Dockers ensures scalability, and cloud platforms like AWS or Azure provide infrastructure. The significance of these services lies in their contribution to productivity, enabling faster deployment times and encouraging resource optimization.
The selection of external services hinges on factors such as functionality, compatibility, cost, and vendor reputation. Teams often lean toward services with extensive community support and comprehensive documentation. In a healthy CI/CD pipeline, third-party services interoperate seamlessly, each contributing to a cohesive, efficient, and secure software delivery process. Regular updates, maintenance, and monitoring ensure these services perform optimally.
Components of Effective Governance of Third-Party Services
Effective governance of third-party services involves policy formulation, oversight, risk management, and performance monitoring to ensure that the use of services adheres to the organization's security standards. It entails defining acceptable use, tracking compliance, and maintaining an inventory of third-party services. But several influences can challenge effective governance. Examples include:
- Lack of visibility into third-party service use
- Ineffective policies
- Inattention to service terms
- Lack of understanding of the associated risks
- Insufficient internal resources for management
- Rapid adoption of new technologies
How CICD-SEC-8 Happens
Let's imagine a scenario that illustrates how an attacker exploits an insecure system configuration — and underscores the importance of effective governance controls.
Identifying the Target
The attacker identifies an organization that uses a third-party static analysis platform within its CI/CD pipeline. This platform is granted an access token, allowing it to read and write code in the organization's repositories.
Exploiting the Vulnerability
The attacker discovers a vulnerability in the third-party platform's security, specifically in its token management system. The platform doesn’t adequately monitor or limit the usage of its access tokens. The attacker launches a phishing attack, targeting an employee who has the ability to generate or manage access tokens for the third-party service. At the same time, the attacker uses a brute-force method to predict a poorly generated access token.
Gaining Unauthorized Access
With the stolen access token, the attacker impersonates the third-party service and gains the same level of access to the organization's repositories. The CI/CD systems don’t have monitoring in place to detect the unusual activity, which allows the attacker to operate undetected.
Lateral Movement
Using the permissions of the compromised third-party service, the attacker moves laterally within the organization's systems. They target a repository that contains the organization's pipeline configuration files. Within these files, they find credentials for the build system, stored in plaintext. The attacker uses these to gain access to the build system where they can modify the build scripts and pipeline configuration directly, bypassing the need for the third-party service.
Injecting Malicious Code
The attacker writes malicious code that opens a backdoor in the system, giving them remote access even after the initial access token isn’t valid. The malicious code is committed to a repository, triggering a build in the CI/CD pipeline. The build system, unaware of the malicious code, executes it as part of the build process, which gives the attacker control over the build system.
Data Exfiltration
The attacker uses their control over the build system to initiate a data leakage. They modify the build scripts to include commands that dump the production database and send it to an external server under their control. This server is set up to receive and store the sensitive customer information extracted from the database.
Maintaining Persistence
The attacker ensures their continued access by embedding a backdoor within the system. They modify the CI/CD pipeline to automatically include this backdoor in every build, ensuring their access persists even if the initial backdoor is discovered and removed. Using the stolen build system credentials, they compromise another third-party service connected to the CI/CD systems, further expanding their reach within the organization's infrastructure.
Importance of Governing Third-Party Services in CI/CD
Third-party services in a CI/CD pipeline introduce an external variable into a controlled environment. While these services empower development, they can extend a potential attack vector to an adversary. Given the highly interconnected dynamics of the CI/CD system and environment, compromise of a single third-party can ripple well beyond the scope of the system the third party is connected to.
Risks Associated with Ungoverned Usage of Third-Party Services
Ungoverned usage can lead to several risks, including data breaches, compliance issues, and exposure to vulnerabilities within the third-party service. Moreover, it can cause system instability, service unavailability, or unauthorized access. The overall risk is a diminished trust in the organization's ability to safeguard its digital assets.
Case Study 1: From Compromised DeepSource Account to GitHub Application Permissions
In July 2020, GitHub's security team alerted DeepSource about potential malicious activity related to the DeepSource GitHub application. The activity, which began in mid-June, involved a large number of requests from unusual IP addresses for various DeepSource users. DeepSource promptly rotated all user tokens, client secrets, and private keys. Further investigation revealed that an employee's GitHub account had been compromised by the Sawfish phishing campaign, granting the attacker access to DeepSource GitHub app's credentials.
Case Study 2: Unauthorized Use of GitHub OAuth Token Behind Waydev Breach
In July 2020, Waydev, a software analytics company, faced a security incident involving unauthorized use of a GitHub OAuth token. The attack was reported by a user in their trial environment. Waydev's security team, in collaboration with an independent company, Bit Sentinel, identified multiple attacks, exploratory activities, and automated scanners. The company promptly fixed the issues, revoked all GitHub OAuth tokens, and took additional security measures.
Case Study 3: Codecov’s Docker Image Creation Process Allows Credential Extract
In April 2021, Codecov, a leading provider of code coverage solutions, discovered unauthorized alterations to their Bash Uploader script. The alterations, which began in January 2021, allowed a third party to export information from users' continuous integration environments to an external server. The breach was due to an error in Codecov's Docker image creation process. Upon discovery, Codecov secured the affected script, initiated an investigation, and advised affected users to reroll all credentials, tokens, or keys used in their CI processes.
Preventing Ungoverned Usage of Third-Party Services
AppSec teams with an understanding of the risks associated with ungoverned usage of third-party services are better equipped to manage risks through governance controls at every stage in the third-party usage lifecycle. Primary recommendations include:
Approval
Establish vetting procedures to ensure third parties granted access to resources are pre-approved and granted only the level of permission that align with the principle of least privilege.
Integration
Introduce controls and procedures to maintain continuous visibility of all third parties integrated to CI/CD systems, including:
- Method of integration. Make sure all methods of integration for each system are covered (marketplace apps, plugins, OAuth applications, programmatic access tokens, etc.).
- Level of permission granted to the third party.
- Level of permission in use by the third party.
Visibility over Ongoing Usage
Ensure each third party’s access is limited and scoped to the specific resources required and remove unused or redundant permissions. Third-parties services integrated as part of the build process should run inside a scoped context with limited access to secrets and code and with strict ingress and egress filters.
Deprovisioning
Periodically review your inventory of integrated third parties and remove those no longer in use.
Industry Standards for Governing Third-Party Services
Standards such as ISO/IEC 27001 offer guidelines for third-party service governance, including implementing service level agreements (SLAs) and in some cases, business associate agreements. Additionally, standards like OWASP ASVS define controls for using third-party services within application security.
Policy Formulation for Acceptable Use
Craft clear policies to prevent ungoverned usage of third-party services. Policies should define acceptable use and establish requirements for adopting new technologies, as well as covering aspects like data privacy, security standards, and compliance requirements. Policies must also outline the process for evaluating and integrating new third-party services, including security assessments, compatibility checks, and performance evaluations.
Oversight and Resource Allocation for Third-Party Service Management
Effective management of third-party services requires dedicated oversight and sufficient resources. Organizations should assign a designated team or individual the responsibility of managing these services, ensuring compliance with policies and addressing issues that arise. The oversight team should possess the necessary skills and resources to effectively manage the integration and use of third-party services, including understanding their functionality, monitoring their performance, and addressing any security or compliance issues.
Performance Monitoring and Compliance Tracking
Maintain an inventory of all third-party services and regularly monitor their performance, promptly addressing issues. Regular audits should be conducted to identify noncompliance issues, according to the organization's policies and relevant regulations.
Tools Supporting Governance of Third-Party Services
Several tools assist with the governance of third-party services and help organizations by automating the process of checking third parties for compliance with policies and standards.
Software composition analysis (SCA) tools, for example, can scan the codebase for third-party components, identifying their versions, licenses, and any known vulnerabilities, as well as creating and maintaining a software bill of materials (SBOM).
Static application security testing (SAST) tools can analyze source code for security vulnerabilities, including those introduced by third-party services. Dynamic application security testing (DAST) tools, like OWASP ZAP or Nessus, can identify security vulnerabilities in running applications, including those resulting from third-party services.
For containerized environments, a container security solution can scan container images for vulnerabilities, ensuring third-party services within containers meet security standards. As well, configuration management tools like Ansible or Puppet can enforce consistent configurations across third-party services, reducing the risk of misconfigurations that could lead to security vulnerabilities.
A cloud-native application protection platform (CNAPP) offers comprehensive security for cloud-native applications, seamlessly integrating many of the above tools into its platform. In addition to providing visibility into application components — including third-party services and their interactions — CNAPPs can detect anomalies, flagging potential security threats or policy violations. They also offer automated remediation, reducing the time to respond to security incidents.
By integrating a CNAPP into the CI/CD pipeline, organizations can continuously monitor third-party services for compliance with security policies, identify and address vulnerabilities, and ensure secure configurations. The CNAPP's ability to provide real-time security insights and enforce policies across the application lifecycle makes it a powerful tool for governing third-party services.
Ungoverned Usage of Third-Party Services FAQs
Indicators of compromise (IoCs) are bits of forensic data, such as data found in system log entries or files, that identify potentially malicious activity on a system or network. By detecting and analyzing IoCs, organizations can discover attacks and promptly respond to prevent or mitigate damage. Examples of IoCs include:
- Unusual outbound network traffic
- Anomalies in privileged user account activity
- Sudden increase in database read volume
- Unexpected system file changes
- Unusual DNS requests
- Mismatched port-application traffic
A software development kit is a collection of software tools and libraries provided by third-party services that developers use to create applications for specific software packages or frameworks.
In the context of cybersecurity, a brute-force attack refers to a trial-and-error method used by attackers to gain access to information. An attacker systematically checks all possible combinations of passwords or encryption keys until stumbling on the correct one.
If, for example, an attacker wants to crack a four-digit PIN, they might start at 0000 and go all the way to 9999, trying each possible combination until they find the one that works. Because passwords are typically far more complex, brute-force attacks can require computational power and time.
DevSecOps teams can mitigate brute-force attacks with measures that include account lockouts after a certain number of failed attempts, requiring complex passwords, and using two-factor authentication.
OAuth is an open standard for access delegation, commonly used as a way for internet users to grant websites or applications access to their information on other websites but without giving them the passwords.
An API key is a code passed in by computer programs calling an API to identify the calling program, its developer, or its user to the API.
A webhook is a method of augmenting or altering the behavior of a webpage or web application with custom callbacks. Webhooks are often used for triggering CI/CD pipelines based on events in third-party systems.
Serverless computing is a cloud computing execution model in which the cloud provider runs the server and dynamically manages the allocation of machine resources. Providers of serverless computing services are often third-party companies.
A container registry is a repository for storing container images. Third-party services such as Docker Hub or Google Container Registry provide remote, hosted container registries.
A content delivery network is a geographically distributed network of proxy servers and their data centers. The goal is to provide high availability and performance by distributing the service spatially relative to end users.
Risk management plays a crucial role in preventing ungoverned usage of third-party services in cloud environments. By continuously assessing, identifying, and mitigating potential risks, organizations can maintain control over their data and infrastructure.
To implement effective risk management, prioritize asset classification and develop a comprehensive risk assessment framework. Regularly evaluate and update security controls, incorporating continuous monitoring and automated auditing processes. Establish strict access control policies, ensuring least privilege access to critical resources. Employ encryption and data loss prevention techniques to protect sensitive information.
You’ll also want to collaborate with third-party vendors to define clear security requirements and responsibilities, formalized in service level agreements (SLAs). Implement an incident response plan to swiftly detect, contain, and remediate threats. Above all, integrating these practices into your DevOps, DevSecOps, and AppSec processes.
A service-level agreement (SLA) is a contract between a service provider and client that outlines the level of service expected from the service provider. In cloud computing, SLAs define performance metrics — such as availability, uptime, latency, and data throughput — along with penalties for noncompliance. SLAs also specify responsibilities for data security, backup, and disaster recovery, ensuring accountability of involved parties.
A business associate agreement (BAA) is a legal contract between a healthcare provider or organization (covered entity) and a third-party service provider (business associate) that deals with protected health information (PHI). The BAA upholds accountability, ensuring that both parties comply with Health Insurance Portability and Accountability Act (HIPAA) regulations and safeguard the privacy and security of PHI. The agreement outlines the responsibilities of each party in handling PHI, the permitted uses and disclosures of the information, and the security measures required to protect the data.
