After brutal critiques, Microsoft Recall will get these major privacy and security changes

June 7, 2024 TH Author

Last month, with great fanfare, Microsoft announced its new line of Copilot+ PCs, with a signature AI-powered feature called Microsoft Recall. That feature is intended to make life easier for hundreds of millions of customers worldwide. But the company forgot to factor in the privacy risks associated with that product’s data collection practices or the ingenuity of hackers and security professionals, ethical or otherwise.

Also: How Microsoft’s new AI Copilot features could transform teamwork and projects

The result was a flood of criticism about the feature and its risks. Security professionals who were able to try the feature in advance of its release put together detailed critiques of its design, with Kevin Beaumont delivering probably the most brutal assessment:

I think it’s an interesting entirely, really optional feature with a niche initial user base that would require incredibly careful communication, cybersecurity, engineering and implementation. Copilot+ Recall doesn’t have these. The work hasn’t been done properly to package it together, clearly.

[…]

I think they are probably going to set fire to the entire Copilot brand due to how poorly this has been implemented and rolled out. It’s an act of self harm at Microsoft in the name of AI, and by proxy real customer harm.

Apparently, that critique triggered a five-alarm fire drill in Microsoft, with the company now announcing significant changes to the feature. In a blog post blandly titled “Update on the Recall preview feature for Copilot+ PCs,” Microsoft Corporate VP Pavan Davuluri, who runs the Windows + Devices division, acknowledged the criticism: “[W]e have heard a clear signal that we can make it easier for people to choose to enable Recall on their Copilot+ PC and improve privacy and security safeguards. With that in mind, we are announcing updates that will go into effect before Recall (preview) ships to customers on June 18.”

According to the blog post, the following changes will be implemented in the released version of the feature:

The setup experience of Copilot+ PCs will offer “a clearer choice to opt-in to saving snapshots using Recall.” The feature will be off by default and will be enabled only if the user chooses to enable it. (In my earlier post, I had suggested this as a crucial change.)
Enabling the feature will require Windows Hello enrollment, with secure proof of the user’s identity, typically via biometrics. In addition, Microsoft said, “proof of presence” also will be required to view the Recall timeline and search its contents.
The database itself will get an extra layer of data protection, including “just in time” decryption, which Microsoft says will be protected by Windows Hello Enhanced Sign-in Security (ESS). The search index database will be encrypted as well.

That last change is probably the most interesting one and should prevent some of the more dire scenarios that security researchers have warned about.

Also: I just ordered the cheapest Surface Pro option – why I (probably) won’t regret it

The “just in time” decryption feature means that Recall snapshots will be protected by a second layer of encryption, using Windows Hello Enhanced Sign-in Security (ESS). Even if an attacker is able to access the user’s database, they won’t be able to decrypt and access its contents unless they can provide secure authentication on a device that’s enabled for that user with Windows Hello.

In combination, all of these changes should make things significantly more difficult for someone trying to exfiltrate the Recall database and access its contents. Info-stealer malware that succeeds in retrieving the Recall database will find its contents encrypted and unreadable. Even an administrator on a Windows 11 PC will be unable to access the contents of another user’s database without their consent, because they won’t be able to provide the required biometric proof.

The irony here is that Redmond is making these changes to a product that hasn’t even been released in a preview version. All the reverse engineering that researchers have done so far is based on unlocking access to the feature using a preview build of Windows and OEM packages that have to be installed using the developer equivalent of voodoo. (There’s a good write-up of the process over at Tom’s Hardware, courtesy of a developer who chose not to use their real name, for obvious reasons.)

Also: Microsoft’s latest Windows 11 security features aim to make it ‘more secure out of the box’

And even when the Copilot+ PCs ship on June 18, the feature will still be available only as a preview, with the official release date not yet scheduled. Presumably, that will allow Microsoft developers to process more feedback from the tiny population of bleeding-edge Windows users who’ve bought a first-generation Copilot+ PC.

Personally, I’m looking forward to trying out this feature when my new Snapdragon X-equipped Surface arrives later this month. But you can bet I’ll be paying careful attention to the setup process.