Open source software has its perks, but supply chain risks can’t be ignored

February 22, 2023 TH Author

Analysis Open source components play an increasingly central role in the software development scene, proving to be a boon in a time of continuous integration and deployment, DevOps, and daily software updates.

In a report last year, silicon design automation outfit Synopsys found that 97 percent of codebases in 2021 contained open source, and that in four of 17 industries studied – computer hardware and chips, cybersecurity, energy and clean tech, and the Internet of Things (IoT) – open source software (OSS) was in 100 percent of audited codebases. The other verticals had open source in at least 93 percent of theirs.

It can help drive efficiency, cost savings, and developer productivity.

“Open source really is everywhere,” Fred Bals, senior technical writer at Synopsys, wrote in a blog post about the report.

That said, the increasing use of open source packages in application development also creates a path for threat groups that want to use the software supply chain as a backdoor to myriad targets that depend on it.

The broad use of OSS packaging in development means that often enterprises don’t know exactly what’s in their software. Having a lot of different hands involved increases complexity, and it’s hard to know what’s going on in the software supply chain. A report last year from VMware found that concerns about OSS included having to rely on a community to patch vulnerabilities, and the security risks that come with that.

Varun Badhwar, co-founder and CEO of Endor Labs – a startup working to secure OSS in app development – called it “the backbone of our critical infrastructure.” But he added that developers and executives are often surprised by how much of their applications’ code comes from OSS.

Badhwar noted that 95 percent of all vulnerabilities are found in “transitive dependencies” – open source code packages that are indirectly pulled into projects rather than selected by developers.

“This is a huge arena, yet it’s been largely overlooked,” he warned.

Growing awareness of the threat

The trend toward using OSS packages isn’t new. Developers have been doing it for a dozen years or more, according to Brian Fox, co-founder and CTO at software supply chain management vendor Sonatype and a member of the OpenSSF (Open Source Security Foundation) governing board.

Developers pull the source components together and add business logic, Fox told The Register. This way, open source becomes the foundation of the software.

What’s changed in recent years is the general awareness of it – not only among well-meaning developers that are creating the software from these disparate parts.

“The attackers have figured this out as well,” he said. “A big notable change over the last five or so years has been the rise of intentional malware attacks on the supply chain.”

That came to the fore with the SolarWinds breach in 2020, in which miscreants linked to Russia broke into the firm’s software system and slipped in malicious code. Customers who unknowingly downloaded and installed the code during the update process were then compromised. Similar attacks followed – including Kaseya and, most notably, Log4j.

Getting the picture through Log4j

The Java-based logging tool is an example of the massive consolidation of risk that comes with the broad use of popular components in software, Fox argued.

“It’s a simple component way down [in the software] and it was so popular you can basically stipulate it exists in every Java application – and you would be right 99.99 percent of the time,” he said. “As an attacker … you’re going to focus on those types of things. If you can figure out how to exploit it, it makes it possible to ‘spray and pray’ across the internet – as opposed to in the ’90s, when you had to sit down and figure out how to break each bespoke web application because they all had custom code.”

Enterprises have “effectively outsourced 90 percent of your development to people you don’t know and can’t trust. When I put it that way, it sounds scary, but that’s what’s been happening for ten years. We’re just now grappling with the implications of it.”

Log4j also highlighted another issue within the software supply chain and woke many up to how dependent they are on OSS. Even so, an estimated 29 percent of downloads of Log4j are still of the vulnerable versions.

According to analysis by Sonatype, the majority of the time that a company uses a vulnerable version of any component, a fixed version of the component is available – but they’re not using it. That points to a need for more education, according to Fox. “96 percent of the problem is people keep taking the tainted food off the shelf instead of taking a cleaned-up one.”

Targeting the repositories

There is another rising threat related to OSS: the injection of malware into package repositories like GitHub, Python Package Index (PyPI), and NPM. Cybercriminals are creating malicious versions of popular code via dependency confusion and other techniques to trick developers into putting the code into their software.

They may use an underscore instead of a dash in their code, in hopes of confusing developers into grabbing the wrong component.

“The challenge with this is that the attack happens as soon as the developer downloads that component and these downloads happen by the tools,” Fox said. “It’s not like they’re literally going to a browser and downloading it like the old days, but they’re putting it into their tool and it happens behind the scenes and it might execute this malware.

“The sophistication of the attacks is low and these malware components don’t even often pretend to be a legitimate component. They don’t compile. They’re not going to run the test. All they do is deliver the payload. It’s like a smash-and-grab.”

Defenses are going up

Despite the security risks inherent in OSS, there are advantages to using it. It’s more visible and transparent than commercial software, Fox argued. He pointed to the response to the Log4j vulnerabilities: the team working on Log4j turned around a fix within a few days – something commercial organizations would likely not have been able to do.

Mike Parkin, senior technical engineer at Vulcan Cyber, agreed that the open source model of having more eyes on the code can help mitigate cyber threats, but it also makes it easier for potential attackers.

That said, “historically the tradeoff has usually favored the open source developers,” Parkin told The Register.

The SolarWinds attack put a lot of focus on software supply chain security. Building on US president Biden’s 2021 Cybersecurity Executive Order, the White House in September 2022 ordered [PDF] federal agencies to follow NIST guidelines when using third-party software – including self-attestation and software bills of materials (SBOMs) by the software makers.

There is a broad array of efforts in train by vendors looking to harden the security of the software supply chain. These include the rise of multi-vendor frameworks like the Open Software Supply Chain Attack Reference, tools like the Vulnerability Exploitability Exchange (VEX), and other products being developed by cybersecurity vendors.

Still, there are other steps Sonatype’s Fox would like to see – like requiring software makers to recall defective software components. Right now, they’re made to work up an SBOM. Fox compared that to car manufacturers only having to give buyers a list of vehicle parts, which can then be stuck into a glove box and forgotten, without a responsibility to recall the car if any of those parts are defective.

“What we really need is something to basically mandate that they can do a recall, because that implies that they know all the parts and where they ship them and which versions of the applications have which open source dependencies, but it also means they’re actually managing it and looking out for that,” he said. “That drives you towards that proper behavior.”

Fox wants the focus on actually maintaining the OSS packages. There is some movement by governments in that direction, he said, noting that the EU’s Cyber Resilience Act talks about the need for recalls, even if it doesn’t use the exact words. Fox said the Biden administration may be starting to warm up to the idea.

He also is broaching the idea of component-level firewalls that work in ways similar to packet-level firewalls, which can inspect network traffic and block malicious traffic before an attack can begin. Likewise, a component-level firewall could stop malicious code before it compromises the software.

“If you don’t even know what’s in your software to start with, you probably have no visibility into what’s going on with the malware, which is almost a worse problem because it’s not just the vulnerability that’s latent, waiting for somebody to exploit,” he said. “It’s causing harm the moment you touch it. Not enough people are really getting their head around that part of the problem either.”

Sonatype built that capability into its platform with the Nexus Firewall, which Fox said was modeled after credit card fraud protection. The firewall understands what normal behavior looks like and then, using artificial intelligence and machine learning techniques, can detect abnormal behavior. In 2022, the firewall flagged more than 108,000 malicious attack attempts.

“So many organizations don’t even know that this is a problem,” he said. “It’s where the game is happening right now and the attackers are kind of having a field day, unfortunately.”

A combination of SBOM and firewall-like capabilities is needed.

“Yes, you need to know where all those parts are, so when the next Log4j happens, you can remediate it immediately and not have to start triaging thousands of applications,” Fox argued. “But that’s not going to stop these malicious attacks. You also need to be perfect protecting the factory.” ®