Microsoft’s AI ‘Recall’ feature raises security, privacy concerns

Microsoft revealed its AI-optimized Copilot+ PCs on Monday, including a new feature that has raised concern among some security experts.

The Copilot+ PCs will ship with a preview version of a feature called “Recall,” which Microsoft said is designed to feel like having a “photographic memory” of everything you’ve viewed on your PC.

Recall takes “snapshots” of the user’s active screen every few seconds and then enables the user to review their activity in a timeline or by search in order to locate webpages, apps or files they previously viewed.

Microsoft’s blog stated the feature will help users quickly find something they previously viewed without needing to dig through websites, files or “hundreds of emails” to locate it.

A demonstration of the feature during a Wall Street Journal interview of Microsoft CEO Satya Nadella showed how the AI models built into the PCs can be used to search for content such as photos using natural language queries.

Microsoft’s FAQ about the Recall feature notes it “does not perform content moderation” and “will not hide information such as passwords or financial account numbers,” raising concern by many that the sensitive data made readily available through snapshots could fall into the hands of threat actors.

The feature has also been compared with “spyware” due to its constant monitoring of the user’s computer activity.

“Microsoft’s Recall feature raises a few alarms, including security risks of potentially capturing and store detailed and sensitive information, as well as concerns surrounding invasion of privacy. The potential of sensitive information being stored without proper security protocols, puts your cybersecurity and even your identity risk,” Patrick Tiquet, vice president of security & architecture at zero-trust cybersecurity provider Keeper Security told SC Media.

Critics of the feature have been vocal on social media, including prominent voices in the cybersecurity sphere.

“Thanks, Microsoft, for your service to enabling malicious hackers,” Kevin Beaumont, a security researcher and former senior threat intelligence analyst at Microsoft wrote on X, saying the feature sets the stage for future “CoPilot Recall malware, where it steals everything you’ve ever typed or viewed as it’s in an already assembled database.”

Cybersecurity giant Malwarebytes also chimed in on the controversy, making comments on X such as “Built-in keylogger is a hell of a feature,” and, “Who needs privacy when you can have AI instead?”

Security, privacy experts wary of ‘invasive’ AI Recall feature

The snapshots taken by Recall are stored locally on the PC’s hard disk and are protected with data encryption, according to Microsoft’s FAQ, but this won’t necessarily block them from view of a threat actor with privileged remote access.

Microsoft also enables users to completely disable the Recall feature or block it from taking snapshots of certain websites or applications, and does not take snapshots of InPrivate browsing sessions on Microsoft Edge or digital rights management (DRM) protected material, according to the FAQ.

Narayana Pappu, CEO of data security and privacy compliance company Zendata, told SC Media that storing snapshots locally instead of in the cloud does not guarantee safety, nor does the option to opt out of the feature.

“Endpoints, like PCs, historically have higher ransomware risk than cloud environments. In fact, a survey by Absolute found that 42% of endpoints were unprotected at any given moment. Second, most users don’t even opt out of diagnostic/telemetry data that Microsoft gets from PC users” Pappu said. “So, as exciting as this development is, the risk mitigation would depend on automatic data retention standards, auto-enforcement of security/encryption before turning on Recall, and finally, the type of information stored (actual data vs. metadata).”

Omri Weinberg, co-founder and CRO of automated SaaS security company DoControl, also told SC Media that while AI features like Recall can have benefits, cybersecurity may struggle to keep up with these advancements.

“Microsoft’s new Recall feature is a major step forward in helping users with advanced context, but it also brings up some serious security and privacy concerns. First off, constantly taking screenshots of a user’s PC creates a treasure trove of sensitive information, like financial and personal data,” Omri Weinberg, co-founder and CRO of automated SaaS security company DoControl, told SC Media. “This is a goldmine for cybercriminals and also raises big questions about compliance with global data protection regulations like GDPR and CCPA.”

Weinberg added that there’s significant risk if Recall is unable to differentiate between general information and sensitive details, as suggested by its lack of “content moderation.”

“The biggest issue I see is user awareness. Microsoft does offer opt-out options, but how effective are they if users don’t fully understand the extent of the data being collected or what it means if they don’t opt out? Until these issues are sorted out, I’ll be recommending that people steer clear of using the system,” Weinberg said.

Gal Ringel, co-founder and CEO at global data privacy management firm Mine, told SC Media that Microsoft’s Recall is an “affront to user privacy and an assault on best security and privacy practices.”

“Beyond its extraordinarily invasive nature, the fact that there are no parameters in place to censor or conceal sensitive information like credit card numbers, personal identifiable information, or company trade secrets is a severe misstep in product design that presents risks far beyond hackers,” Ringel said.

For enterprises that store protected data of employees, users and customers, many of which do not have the resources to securely store large amounts of unstructured data, using a system that collects up to millions of screenshots is “an accident waiting to happen,” Ringel added.

“Opt-outs for features like this are nowhere near enough to protect user safety. Anything that tracks individuals this closely must be opt-in, with clear notices presented to the public explaining what the feature does and why it is necessary,” Ringel concluded.

SC Media reached out to Microsoft to ask about its response to the criticism, how data collected through Recall will be protected and used, and whether Recall is activated by default on CoPilot+ PCs, and did not receive a response.