Apple unveils generative AI ‘Apple Intelligence’ coming to iPhone and Mac

Apple Inc. today unveiled its long-awaited artificial intelligence strategy at its Worldwide Developer Conference with its “Apple Intelligence” technology that will be integrated deeply into all its devices and apps, including its AI assistant Siri.

The company also announced a partnership with OpenAI to bring ChatGPT to its devices, including access to GPT-4o, the generative AI company’s flagship AI model that will allow Siri to send questions to the chatbot to answer questions.

At the forefront, Siri is getting smarter and more conversational with a bit of a makeover. What was once a personal assistant on the phone is now a livelier version with a deeper understanding of human language that can follow along even if a person stumbles over words when asking a question or pauses to think when presenting a command.

The assistant also understands conversational context, so if a user is already beginning to input information about going to a restaurant, and calls up Siri, it’s not necessary to say the name of the restaurant. Instead the person could just ask, “How do I get there?” and the AI will answer.

Siri will also be able to take actions on a user’s behalf at their request on command by triggering events in apps through voice command, such as, “Show me all the photos of Mom, Olivia and me.” Or, a user can ask the assistant to bring up a picture of a particular friend and then have it adjust the colors so that it’s more vivid.

It would also be capable of understanding even more context about a user’s personal life to help answer questions about schedules, such as when an airplane was arriving, or planned outings. All of this information, according to Apple, would be gleaned from a user’s “personal context,” including messages, emails, calendar and other information stored on their devices.

“With Apple Intelligence and its conversational context, Siri transforms from a simple voice assistant into a proactive, intelligent tool that manages tasks and applications with unparalleled ease,” Tim Miner, founder and chief executice of By the People Technology, an AI consultancy. “This integration of Siri with various applications means users can rely on their iPhones to handle complex tasks, reducing the need for manual input and making everyday activities more convenient and efficient.”

For example, say a user is picking someone up from the airport and doesn’t quite recall when the plane is arriving. The person can say, “Siri, when is my friend’s flight arriving?” It’s not necessary to say which friend because the AI assistant will cross-reference messages, emails and other context from the phone to determine flight times and other information to figure that out and then display relevant information using Apple Intelligence and share the real-time flight plan. With a follow-up question such as “How long will it take me to get to the airport?” a map can be viewed including traffic and a route.

Language capabilities for understanding and creation

Apple Intelligence will be available on devices with new systemwide Writing Tools built into the upcoming operating systems iOS 18, iPadOS 18 and macOS Sequoia coming later this year. It will allow users to adjust their writing quickly in ways they’ve gotten used to with generative AI writing tools, such as rewriting, proofreading and summarizing text across any app where they have a text field.

Rewrite allows users to pick between various styles of what’s already been written, allowing them to alter tone, correct their approach, be more creative, more humorous or maybe tone down their flamboyance if they’re trying to talk to their professor or boss.

Proofread is all about making sure those pesky grammar errors don’t sneak through. It also checks for word choice, and sentence structure and also suggests edits – it also provides explanations as to why those edits are needed. Meanwhile, Summarize provides shorter forms of the original text in a paragraph, bullet points, a table or a list.

The same intelligence is being rolled out across other apps so that users get a better experience such as in Mail, where the inbox no longer simply shows the first few lines of each email. Now it displays a summary of the important points.

Priority messages also float to the top, surfacing the most urgent emails like same-day reservations and boarding passes. This level of understanding also extends to Notifications, prioritizing the important ones and putting key details front and center.

Image Playground for image creation

If users are feeling creative, they can use Image Playground, a new feature that allows users to quickly generate new images in moments by choosing between three different styles: Animation, Illustration and Sketch. It’s built directly into apps including Messages to make it easy to call up.

The feature provides a set of suggestions that can be flipped through and even allows users to describe what they want with a prompt, or use one of their contacts to help form the picture. To help users build their image it offers a range of themes, costumes, accessories and places.

In the Notes app, users can use “Image Wand,” to change a rough sketch and the surrounding context into a vivid image. Or users can simply circle empty space and the Image Playground will use surrounding context, such as words already written in the Notes app as a prompt and suggest potential images that can spice up the presentation.

ChatGPT integration comes to Apple’s platforms

Apple said that it’s integrating ChatGPT into iOS 18, iPad 18 and macOS Sequoia. Users will be able to call it up within a multitude of apps for writing or question-and-answer assistance without needing to call it up directly.

It will work much like an ordinary context menu with the Apple Intelligence symbol, but users will be informed first before anything is sent to ChatGPT. Users do not need a login to use ChatGPT and those with a premium subscription can take advantage of their subscription should they choose, including advanced image generation capabilities. Siri can also tap into ChatGPT’s expertise, of course, Siri will ask first before sending anything along.

“The introduction of Apple Intelligence is not just about upgrading Siri; it’s about enhancing the overall user experience and ensuring Apple remains at the forefront of technological innovation,” said Miner. “Aligning with OpenAI addresses Siri’s shortcomings and positions Apple as a forward-thinking company prioritizing innovation and user satisfaction.”

Apple said it plans to offer more models after ChatGPT in the future but did not detail which ones were in the works.

Apple Intelligence and generative AI privacy

To allay privacy concerns, Craig Federighi, Apple’s senior vice president of software engineering explained, most of the AI processing for Apple Intelligence happens on-device using Apple’s M-family of chips. With this processing power on-device, he said, it’s also possible to verify that users are in control of their own data and it’s not being misused.

However, when greater processing power is needed and it must reach out into the cloud, Apple has created what the company calls “Private Cloud Compute.” Federighi said it “allows Apple Intelligence to flex and scale its computational capability and draw on even larger server-based models for even more complex requests while protecting your privacy.”

He added that the models run on servers specifically created on Apple silicon designed based on the same technology as the iPhone and similar software that can be audited. Independent experts can verify the promise of privacy, he said.

During his commentary, Federighi took a shot at AI companies and their cloud services, saying that their privacy models were “unverifiable.” The claim that the Private Cloud Compute service is verifiable is monumental but untested, so that may be something to look forward to in Apple’s future.

“Your iPhone, iPad and Mac will refuse to talk to a server unless its software has been publicly logged for inspection,” Federighi said. “This sets a brand-new standard for privacy in AI.”

Images: Apple