Apple Intelligence: How the iPhone’s on-device and cloud-based AI will work

Apple is expected to have one of its most groundbreaking Worldwide Developers Conference (WWDC) keynotes on Monday, as the company plans to add major artificial intelligence (AI) features to its operating systems. 

But rather than unveiling a slew of flashy generative AI features to knock your socks off, Apple is expected to focus on incorporating AI into its apps to simplify users’ daily tasks. And it’ll categorize such features under the name “Apple Intelligence.” 

Also: The best Apple deals of June 2024: iPhones, Apple Watches, iPads, and more

According to Bloomberg, Apple is branding its AI features under Apple Intelligence — and we didn’t miss the snarky word play. Apple Intelligence will include the latest AI features coming to its operating systems, including iOS, iPadOS, MacOS, and WatchOS. 

Apple Intelligence focuses on broad-appeal AI features rather than advanced image and video generation. To do this, the company developed in-house AI models and partnered with OpenAI to power a chatbot that will work similarly to ChatGPT. 

Also: Apple to give Control Center its most useful customization feature ever with iOS 18

Some of the biggest AI features we’re expecting with Apple Intelligence include:

• Improved photo editing, like object removal, with AI in Photos.
• Greater Siri control over apps and actions, including asking Siri to delete emails or edit photos. 
• AI generation of custom emojis based on text prompts.
• Generating quick recaps of notes, text messaging threads, emails, and more texts.
• Automatically suggesting responses for emails and messages.
• An improved Mail app that can categorize emails and generate messages. 
• Automatic transcription of voice memos.
• AI enhancements for Xcode to auto-complete code.

Aside from these AI features, Bloomberg reports that iOS 18 will include new customizable icons and interface updates for Control Center, Settings, and Messages. Apple is also expected to launch a new Passwords app to replace the iCloud Keychain and give users a more user-friendly option, similar to 1Password and LastPass.

Also: Apple to unveil ‘Passwords’ manager app at WWDC 2024: What it is and how it works

On-device vs. cloud-based

Though Apple was rumored to work on different ways to keep its AI running strictly on-device for security and privacy, Apple Intelligence is expected to rely on the cloud for at least some tasks. This will depend on device complexity, resource availability, data privacy considerations, and latency requirements. 

Essentially, if a task is simple enough to be processed locally, leveraging the device’s processing power and battery life, and requires immediate results, it is more likely to be handled on-device. Tasks involving sensitive data could also prioritize on-device processing, as Apple tries to prioritize data privacy. 

Also: ChatGPT privacy tips: Two important ways to limit the data you share with OpenAI

In turn, cloud-based AI processing requires sending data from the device to remote servers that can handle complex or computationally heavy tasks. In Apple’s case, tasks requiring processing large amounts of data or updated models could include advanced natural language processing (NLP), intricate analysis, and complex image and video generation.

Depending on its complexity and system requirements, an algorithm will determine whether a task requiring AI should be processed on-device or offloaded to the cloud. Simpler tasks like a Siri request and other basic NLP tasks can be processed on-device. More complex tasks, like generating a detailed summary of a large document, will be sent to the cloud, where more robust processing can occur. 

Technical requirements for Apple Intelligence

According to Bloomberg, Apple’s new AI features will be compatible with the latest Apple devices, including iPhone 15 Pro or newer models, which run on an A17 Pro chip, and iPads and Macs with an M1 chip or newer. While these AI features may help drive sales of new iPhones and Macs, as a current iPhone 14 Pro Max owner, I hope that at least some will trickle down to older iPhone models. We’ll know what the official compatibility list is come WWDC on Monday.

Also: Apple Photos app is getting an AI-powered editing feature to wipe out photobombers

During WWDC, Apple is expected to highlight new security measures for running AI tasks, including chip-based security in data centers for cloud-based processing. It will also reiterate its commitment not to build user profiles based on consumer data. 

Perhaps most importantly, users can opt-in for Apple Intelligence features, which will be introduced as beta versions as Apple works to improve its AI capabilities over time.