‘This collaboration is absolutely critical going forward’… One positive thing about Meltdown CPU hole? At least it put aside tech rivalries…

A panel of eggheads from Intel, the US government, and academia held court this week to figure how they can keep the likes of El Reg from spoiling their next major bug reveal.

The group met at the Churchill Club in San Francisco to reflect on 2018’s big security story – the Spectre-Meltdown CPU flaws – and ponder how it could be better handled going forward. Although chip designers were alerted to the vulnerabilities around June 2017, and operating system developers soon after, an action plan for disclosure was still being formulated the week before they hoped to public on Tuesday, January 9, 2018. The Reg blew the lid off it on January 2, after hearing no response from vendors, forcing timetables to be torn up.

Among the board of brains were Intel government and policy director Audrey Plonk, Semiconductor Industry Association CEO John Neuffer, UC Berkeley Law Prof Deidre Mulligan, and White House NSC bod turned Venable cybersec director Ari Schwartz.

The talk centered on the CPU speculative execution holes that sent chip designers back to the drawing board, and kernel and toolchain programmers back to their IDEs, to solve and come up with mitigations. Now one year past the big reveal, the panel pondered how they could have done things differently.

For Schwartz, the saga reaches back to 2014’s Heartbleed, the data-leaking OpenSSL bug that was Meltdown before Meltdown. At the time, he was working in the White House, and had to actually play up the risk of the bug until it got the right attention.

“When we looked at it we know this was very big,” Schwartz recounted. “The chief of staff to the President walked into our office, and said: I want to know everything about this.”

The crisis of Heartbleed seemingly trained the tech giants on how to handle mass disclosure and patching of major security holes that affect the entire industry. Companies would learn how to cooperate with one another and set aside competitive differences for the greater good.

Fast forward three years to late 2017, and researchers dotted around the world uncovered fundamental flaws in the way modern CPUs predicted which data or code would be needed next, flaws that could be exploited by malware to read memory that should be out of bounds – kernel memory or that of another application – and potentially steal passwords and other secrets.

Fixing the flaws would require the hardware and software vendors coming together and not only addressing the security shortcomings, but also coming to terms with various performance hits – some large, some small, some unnoticeable – that would result from the changes. Companies, some dealing with public open-source projects such as C/C++ compilers and the Linux kernel, would need to discreetly share intelligence and code to close the holes while minimizing performance hits and keeping the entire affair quiet.

It had to stay under wraps to prevent the development and distribution of malware that exploits the design weaknesses before folks had a chance to patch. In the end, no known software nasties targeting Meltdown and Spectre were spotted, perhaps because so much fuss was made over addressing the holes, and perhaps because the bugs are difficult to exploit compared to asking someone to open a booby-trapped webpage or document on Windows.

We also note that not everyone was included in the private Meltdown-Spectre discussions. While the secret party featured the expected faces of Intel, AMD, Arm, Red Hat, Microsoft, Amazon, Google, and other large companies, folks like the BSD crowd, smaller Linux distros, and some cloud providers were a tad hurt they were left out, or brought in little notice. Intel et al hadn’t even warned US-CERT by the time we broke the news.

For Neuffer, a man whose job entails wrangling billionaire CEOs intent on destroying one another, the Meltdown-Spectre crisis brought out the best in companies that would normally be at each others’ throats.

“They are fiercely competitive, amazingly competitive, yet this was an example where they found a common cause,” Neumann explained. “This kind of collaboration is absolutely critical going forward, and I’m sure as we attack more complicated causes in the future this collaboration will continue going forward.”

Yet, despite this hush-hush effort, El Reg couldn’t help but notice efforts to rewrite chunks of the memory management portions of the Linux and Windows kernel, lines of code that are so sensitive they shouldn’t be touched once they’re known to work. That indicated something big was afoot, something that had to be mitigated in part in software because the hardware was not enforcing its security as hoped. It started with changes made to the open-source Linux kernel.

“It wasn’t so much that the open source community did something wrong, people were doing their job, and people did their job and pieced parts together and found it out,” said Plonk. “That’s always a risk we take, but we have to work with the open source community.”

It did, however, give cause to reflect on how the big companies should handle confidentiality and non-disclosure agreements, specifically what to do when someone breaks them.

“If you have to work with that person, you have to find a way to make them have some skin in the process,” Schwartz suggested. “Maybe a donation to a charity that focuses on security.”

We can only hope the spirit of collaboration holds as the battle between AMD and Intel heats up.

After our January 2018 report, Intel management was incandescent with anger that an AMD engineer, in a patch quietly submitted to the Linux kernel mailing list in late December 2017, had revealed that there was a flaw only in Intel x86 CPUs and that it involved speculative reads from kernel memory by applications – aka Meltdown. However, that patch was in response to Intel’s private attempts to lump AMD in with its own vulnerabilities; AMD’s patch sought to make clear only Intel’s CPU cores were hit by Meltdown, and thus the kernel’s Meltdown mitigations, and associated slowdowns, shouldn’t be applied to AMD components. When the next big bug hits, we hope the pair of chip designers can put this feud aside.

There is also the question of patching. The panelists agreed that all of the awareness in the world will not help if admins and users are not applying the necessary updates to flaws and closing the vast majority of bugs attackers use to spread malware.

“One of the things that is important is to figure out why people don’t patch,” said Mulligan.

“The level of knowledge of people that are managing machines within businesses varies widely, so you want to think about where you’re positioning responsibility, and if there is liability, where it gets positioned.” ®

READ MORE HERE