Security vendor SolarWinds says product updates were subverted by nation-state, Fireye says exploit is rampant

UPDATE Security vendor SolarWinds’ “Orion” IT monitoring platform has been compromised, and speculation is swirling that it was used in attacks on major US government agencies that could also be linked to last week’s revalation that security vendor FireEye’s top hacking tools have been accessed.

A statement from Kevin Thompson, SolarWinds president and CEO says the company is “aware of a potential vulnerability which if present is currently believed to be related to updates which were released between March and June 2020 to our Orion monitoring products.”

“We believe that this vulnerability is the result of a highly-sophisticated, targeted and manual supply chain attack by a nation state. We are acting in close coordination with FireEye, the Federal Bureau of Investigation, the intelligence community, and other law enforcement to investigate these matters. As such, we are limited as to what we can share at this time.”

As we report in our update to this story below, FireEye says it found the flaw in a compromised .dll file that was posted to the downloads section of SolarWinds’ site.

The Register has asked SolarWinds for further detail, but evidence of updates in the relevant timeframe is not hard to find: here’s a June 2020 patch to the company’s remote monitoring agent for Windows.

If you’re a SolarWinds customer, assume compromise and immediately activate your incident response team.

News of the SolarWinds hack was broken by newswire Reuters, which also reports that US government agencies, among them Treasury and the Department of Commerce, have been hit with a hack so serious that the National Security Council met to discuss it on Saturday.

The Washington Post has reported that the government hacks were made possible by flaws in SolarWinds products and that the attack was perpetrated by Russian hacking group APT29, aka Cozy Bear. US government officials have acknowledged the incidents, but have not offered further details.

This situation is properly scary because a supply chain attack that poisons product updates issued by a major security vendor suggests that Cozy Bear could be deep inside all sorts of systems and vendors. If that doesn’t scare you, maybe SolarWinds’ customer list will, as it mentions the following organisations are users.

• More than 425 of the US Fortune 500
• All ten of the top ten US telecommunications companies
• All five branches of the US Military
• The US Pentagon, State Department, NASA, NSA, Postal Service, NOAA, Department of Justice, and the Office of the President of the United States
• All five of the top five US accounting firms

While the prospect of Cozy Bear rummaging around inside the abovementioned organisations is scary, security experts aren’t panicking.

Security analyst Jake Williams has posted a Twitter thread pointing out that products like Orion are a fine jumping-off point for an attack but points out that many such products are implemented to observe IT infrastructure performance rather than actively change configurations. He therefore urges readers not to assume the attack automatically translates to an ability to control systems.

Former US Cybersecurity and Infrastructure Security Agency head Chris Krebs suggested the attack has likely been under way for months, but that it should be possible to contain.

“If you’re a SolarWinds customer & use the below product, assume compromise and immediately activate your incident response team,” he advised. “Odds are you’re not affected, as this may be a resource intensive hack. Focus on your Crown Jewels. You can manage this.”

As news breaks about what looks to be a pretty large-scale hack, I have the utmost confidence in the @CISAgov team and other Federal partners. I’m sorry I’m not there with them, but they know how to do this. This thing is still early, I suspect. Let’s let the pros work it.

— Chris Krebs (@C_C_Krebs) December 13, 2020

Hopefully, Krebs and Williams are correct. But even if they are, the fact remains that two big security vendors – FireEye and SolarWinds – have been revealed to be cracked and something appears to have taken a bite out of the US government. And all of these organisations boast of having strong defences against such attacks. ®

UPDATE: 03:50 UTC, Monday December 14th.

FireEye has posted an analysis of the flaw in SolarWinds code that says the problem is present in a file called SolarWinds.Orion.Core.BusinessLayer.dll that it describes as a “digitally-signed component of the Orion software framework that contains a backdoor that communicates via HTTP to third party servers.”

FireEye says that once the .dll reaches a machine it remains dormant for up to two weeks, but then comes to life and “retrieves and executes commands, called ‘Jobs’, that include the ability to transfer files, execute files, profile the system, reboot the machine, and disable system services.”

“The malware masquerades its network traffic as the Orion Improvement Program (OIP) protocol and stores reconnaissance results within legitimate plugin configuration files allowing it to blend in with legitimate SolarWinds activity. The backdoor uses multiple obfuscated blocklists to identify forensic and anti-virus tools running as processes, services, and drivers.”

FireEye continues: “The trojanized update file is a standard Windows Installer Patch file that includes compressed resources associated with the update, including the trojanized SolarWinds.Orion.Core.BusinessLayer.dll component. Once the update is installed, the malicious DLL will be loaded by the legitimate SolarWinds executable SolarWind.BusinessLayerHost.exe or SolarWindws.BusinessLayerHostx64.exe (depending on system configuration).”

The malware then goes dormant for another fortnight before attempting to resolve a subdomain of avsvmcloud[.]com. “The DNS response will return a CNAME record that points to a Command and Control (C2) domain. The C2 traffic to the malicious domains is designed to mimic normal SolarWinds API communications.”

FireEye says it has “detected this activity at multiple entities worldwide.”

“The victims have included government, consulting, technology, telecom and extractive entities in North America, Europe, Asia and the Middle East. We anticipate there are additional victims in other countries and verticals.”

Long story short, this is a bad one and made worse by the fact that SolarWinds offers infrastructure monitoring but appears not to have been able to keep its own website and APIs clean.

READ MORE HERE