Apple Offers Peek at Its AI Language Model as iOS 18 Looms
Apple hasn’t said much publicly about its plans to join the many companies offering generative AI products, but this week it did open up a window into its behind-the-scenes work on the kind of system that powers AI chatbots and image generators.
On Monday, it released OpenELM, which it calls a “state-of-the-art open language model.” Language models are the massive sets of information that tools like ChatGPT, Gemini, Perplexity and Dall-E draw on to respond to the prompts you type when you want an AI to whip up an email, write computer code or create a fanciful image.
So it’s not yet the Apple AI product we’ve all been waiting for, but it is a logical step in that direction — and potentially hints at the AI capabilities Apple might offer in its upcoming iOS 18 software for iPhones.
OpenELM’s release comes just weeks ahead of Apple’s WWDC event in early June, where the company traditionally talks about its next wave of software offerings.
Apple did not respond to a request for comment.
But during a quarterly earnings call in February, CEO Tim Cook hinted that Apple would reveal its plans for generative AI at some point in 2024. Also around that time, Apple reportedly shuttered its long-running electric car project to focus on generative AI and the Apple Vision Pro, the wearable that went on sale that same month and that CNET reviewer Scott Stein calls “Apple’s wildest and strangest device.”
It’s not clear yet how OpenELM fits into these plans. However, in a research paper posted in March, Apple discussed multimodal large language models, or those that can generate a variety of content formats.
While Apple has been holding fire, most tech giants and a rash of startups have already rushed out one or more generations of gen AI products. Adobe, Anthropic, Google and OpenAI are in a race to release increasingly capable models that not only understand a wider variety of queries, but produce more realistic images and videos. They’re even keen to highlight internal research projects in fields like video games and music composition to tease what they may be able to offer to capture your attention and lock in your potential loyalty as users of their AI products. (For all the latest AI news, plus product reviews, explainers, tips and more, check out our new AI Atlas guide.)
Some takeaways on OpenELM
A paper posted to Apple’s Machine Learning Research site notes that the OpenELM release includes everything needed to train and evaluate the language model on publicly available datasets, including training logs, checkpoints and pretraining configurations. (The ELM part of the name stands for “efficient language model.”)
On LinkedIn, Maxwell Horton, a machine learning researcher from Apple, wrote that OpenELM is part of a new open-source repository called CoreNet, which is available via GitHub. This resource also includes code to convert models to an array framework for machine learning research on Apple chips, which allows for inference and fine-tuning on Apple devices, as well as vision and language models with training recipes and code release for 11 Apple research publications.
The OpenELM paper said the goal is to pave the way for future open research endeavors.
“The reproducibility and transparency of large language models are crucial for advancing open research, ensuring the trustworthiness of results and enabling investigations into data and model biases, as well as potential risks,” the paper added.
Large language models are measured in what is known as parameters, or the number of variables in a mathematical calculation used to produce an output from a given input. As the number of parameters increase, so does the model’s ability to make sense of language. Apple’s pretrained and instruction-tuned models have 270 million, 450 million, 1.1 billion and 3 billion parameters.
By way of comparison, ChatGPT 3.5 has 175 billion parameters. The latest version of Meta’s Llama 3 model has 70 billion parameters.
Reports emerged last July that Apple was working on an AI chatbot called Apple GPT and a large language model called Ajax, but the company has not commented.
“We have a lot of work going on internally, as I’ve alluded to before,” Cook said of generative AI during the February earnings call. “Our M.O., if you will, has always been to do work and then talk about work, and not to get out in front of ourselves, so we’re going to hold that to this as well.”
Editors’ note: CNET used an AI engine to help create several dozen stories, which are labeled accordingly. The note you’re reading is attached to articles that deal substantively with the topic of AI but are created entirely by our expert editors and writers. For more, see our AI policy.
