Apple WWDC 2024 recap: Apple Intelligence, iOS 18, Siri, OpenAI, and more

Today's the day! WWDC 2024 kicks off in just a few hours with the keynote presentation starting at 10:00 am PST/1:00 pm EST. We're expecting some exciting announcements about Apple's operating systems and its first big leap into AI. 

Today's keynote will be a pivotal moment for Apple as it attempts to prove it can compete with AI industry leaders like Microsoft and Google while also maintaining its reputation as a leader in user privacy. 

Over the next few hours leading up to the keynote, we'll be recapping all the highlights on the road to WWDC 2024, so stay tuned all day for updates and insights. 

 Apple is expected to take a big leap into AI today, reportedly with help from OpenAI, whose employees recently raised concerns about OpenAI’s lack of safety protocols. 

What would Steve Jobs think of this partnership and the impact of AI on privacy? He summed up his view of Apple’s approach to privacy in an interview at the 2010 All Things Digital Conference: 

“We’ve always had a very different view of privacy than some of our colleagues in the Valley. We take privacy extremely seriously… That’s one of the reasons we have the curated app store. We have rejected a lot of apps that want to take a lot of your personal data and suck it up into the cloud… Privacy means people know what they’re signing up for – in plain English and repeatedly.”  

All of Apple’s platforms are expected to get updates today, but the iPad could have the most to gain – or lose. Apple unveiled the powerful new iPad Pro M4 last month alongside the iPad Air M2. 

Both devices have impressive hardware, but the clunky design of iPadOS is holding them back. Will Apple finally give iPadOS a meaningful makeover? I hope so – or, at the very least, I hope they add the calculator app. 

That’s right: the Siri AI is getting a big overhaul, but the iPad still doesn’t have a calculator. 

It’s been almost 13 years since Apple originally launched Siri, which is expected to get a major update today. However, some of the comments initially surrounding Siri sound eerily similar to today’s AI chatter. 

People took to Twitter back in 2011 to compare Siri to menacing AIs from science fiction, including Skynet (Terminator), HAL 9000 (2001: A Space Odyssey), and GLaDOS (Portal). Sound familiar? 

iOS 18 will be center stage at WWDC today. Most iPhone users will get the update, but some older models are losing software support this year. 

Apple analyst Nikias Molina leaked the list of iPhones getting iOS 18 in an X post on Sunday. It includes everything from the iPhone XR onward, but unfortunately, the iPhone X, iPhone 8, and iPhone 8 Plus will probably lose support starting with iOS 18. 

There’s another twist, too: some iPhones might not be compatible with the rumored on-device AI features Apple is expected to launch today. Due to high processing power requirements, newer iPhones could be the only models with support for “Apple Intelligence.” 

Data privacy is a major concern when it comes to AI and Apple is no exception. Some of the rumored “Apple Intelligence” features will require cloud computing, which may leave some users hesitant to try it out. We’re expecting Apple to go into detail about how it will handle security with Apple Intelligence, but Apple’s current iCloud security protocols offer a hint. 

Right now, there are two levels of iCloud security: standard and advanced. Standard is the default, but anyone can opt to use advanced data protection, which includes end-to-end encryption for a wider range of data. 

The drawback is that Apple may not be able to help you recover encrypted data if you lose access to your Apple ID account. If more people opt for advanced data protection with Apple Intelligence, will Apple need to take steps to minimize the risk of data loss if a user’s account is compromised? We’ll have to wait and see what Apple announces today. 

The last Apple event was the “Let Loose” virtual event on May 7, where Apple unveiled the iPad Pro M4, iPad Air M2, Apple Pencil Pro, and a redesigned Magic Keyboard for the M4 iPads. It was exciting getting new iPads after a whole year without an update to the line-up, but the event resulted in some controversy over the “Crush” ad Apple created to promote the iPad Pro M4. 

The ad sparked outrage among creatives due to the implication that tech, specifically AI, can replace creatives and the tools of their craft. Apple responded by canceling its plans to air the ad on TV. 

Will Apple adjust its messaging around AI at WWDC to potentially make amends with the millions of creatives who use Apple products? We’ll have to wait and see.  

Last year’s WWDC included the unveiling of the long-rumored Apple Vision Pro headset, which was released in February. It boasts some seriously impressive hardware, but got off to a rocky start due to mixed reviews citing issues like the headset’s weight and lack of native apps. 

Luckily, Apple is expected to release a big visionOS update today that’s rumored to include more native apps (meaning, apps designed for visionOS rather than ports of existing iPad apps). Hopefully, the update is significant enough to revive interest in the Vision Pro and resolve some of those initial concerns about visionOS. 

However, improvements to the physical design (and weight) of the Vision Pro will have to wait until the next generation model, which will likely also be less expensive. 

Will Apple announce any new hardware today? It’s not impossible, but it does seem unlikely. Apple will probably want to keep the focus on its first big wave of AI features. However, we could see an announcement for the M3 Ultra chip. M4 Macs are unlikely to make an appearance until the end of the year, though. 

Last month, Apple announced the M4 chip, but it came to the iPad before the Mac. Wondering why? The answer may have something to do with the Qualcomm Snapdragon X Elite chip that appeared in a slew of AI-powered laptops from Apple’s competitors at Microsoft Build and Computex. 

The first benchmark tests revealed that the Snapdragon X Elite performs similarly to the M3 chip in most tests, although it appears to have a significant edge in AI performance. 

So, Apple may have chosen to put the M4 chip in the iPad Pro before the Mac because the M4 Macs need more time for R&D but the M4 chip could help Apple retain its position at the top of the mobile computing market. The iPad was the quickest way to stay ahead of the competition and allows more time for Apple to sell M3 Macs, which haven’t been out a full year yet. Plus, the M3 Ultra chip is still missing. Maybe we'll see it at WWDC this week. 

If you don't want to wait for the M4 chip to pick up a new Mac, you can get an M3 MacBook Air on sale. 

There have been a lot of rumors about iOS 18 and the AI features it will include. Most of the leaked features are things Android phones have been able to do for a while now, such as AI photo editing. 

However, one unique feature rumored for iOS 18 is AI-generated emojis. Siri will create these custom emojis based on the text you’re typing. It’s still unclear how exactly AI-generated emojis will work, though. Will they be specially generated text characters or emoji-style images? Will they only work with other iPhone users or will Android users also be able to receive them? 

Apple’s plans to bring RCS to iPhone may play a role since RCS messaging allows for sending higher-quality images than SMS. It would make it easier to send AI-generated emojis to Android users and improve the quality, features, and security of text conversations between Android phones and iPhones. 

Apple just took its online store offline ahead of the WWDC keynote! Could this mean there's something big in the keynote? Maybe even "one more thing" we aren't expecting? We'll find out soon. 

The keynote presentation is set to kick off in just 15 minutes. Here’s a final overview of the top announcements it might include: 

• Apple Intelligence
• iOS 18
• iPadOS 18
• macOS 15
• watchOS 11
• visionOS 2
• M3 Ultra chip

The keynote is starting now! It opens with a video of the Apple team skydiving over Apple HQ (including one member wearing a Vision Pro instead of goggles). Then we cut to Tim Cook standing on the roof of Apple Park, naturally. 

Cook kicks things off with AppleTV+, which is celebrating its 5th year. There are a slew of new originals coming this year, including Dark Matter, Fly Me to the Moon, Silo, and a new season of Severance. 

Time for the platform updates. visionOS is up first. Mike Rockwell, Vice President of the Vision Products Group, steps up to show us what's new for Apple's headset. 

"New apps, including some from the world's biggest names in entertainment, productivity, and gaming, are arriving on the [visionOS] app store every day," Rockwell notes. 

"There's already over 2,000 apps created specifically for Apple Vision Pro... All of these amazing apps and experiences are made possible by visionOS. It's been just four months since we launched Vision Pro and visionOS and today we're already announcing our first major update." 

Rockwell turns things over to Haley Allen, Director of visionOS Program Management, for more details. 

The photos app on visionOS can now add depth to regular photos from your other Apple devices, turning them into spatial photos like those taken on the Vision Pro. It uses machine learning to replicate the natural depth so you can view the photo from different angles in three dimensions. The Photos app also now includes SharePlay, which allows you to view photos and videos in visionOS with other Vision Pro users. 

There are also new hand gestures that allow for shortcuts to the visionOS home screen and Control Center. You can also make larger virtual displays now, including an ultrawide display equivalent to two 4K monitors. 

Mike Rockwell is now covering some new APIs and frameworks that will make it easier for developers to create apps for visionOS including Volumetric APIs, TabletopKit, and Enterprise APIs. 

It's also going to be easier for filmmakers to create spatial videos for the Vision Pro thanks to a new spatial video lens made by Canon. The lens will be specifically for the Canon EOS R7 digital camera. Plus, Final Cut Pro will soon support editing for spatial videos. visionOS is also getting the Vimeo app, which could be Apple's attempt to mitigate the lack of a YouTube app for the Vision Pro. 

Apple is also partnering with BlackMagic Design to make immersive videos more accessible for filmmakers. A new slate of immersive videos is coming later this year including an extreme sports series, music experiences from The Weeknd and other artists, and Submerged, the first scripted immersive short film for visionOS. 

Tim Cook is back to announce that the Vision Pro is coming to several new regions around the world later this year, including China, Japan, Singapore, Australia, Canada, France, Germany, and the United Kingdom.  

Craig Federighi, Senior Vice President of Software Engineering, is up to talk about iOS 18. 

He starts with some big news: you can now place app icons anywhere on the home screen! At long last, iOS is catching up to Android on customization. Plus, the iPhone is getting a dark mode (which I, for one, will definitely be using). "Customization sheets" allow you to recolor all of your app icons, too. 

The Control Center is getting an update, too. There are now different control categories, which you can customize in the "Controls Gallery", which now supports controls from third-party apps. 

The controls on the lock screen will be customizable now, as well. You can swap out the flashlight and camera icons for any other control shortcut you want, including controls from third-party apps. 

Federighi is now talking about new security options. iOS 18 will allow you to lock or hide specific apps for extra security. If you hide an app, it will go into a "hidden apps" folder that's also locked. Plus, when you allow an app to view your contacts, you can now select specific contacts to restrict it to. 

The Messages and Mail apps are up next! Tapback reactions in the Messages app now include all emojis (rather than just a handful). iOS 18 will also allow you to schedule texts and format messages with animated text effects. 

iPhone 14 and later can also now use Messages via Satellite to stay connected when you're out of range of your usual wireless signal, similar to the existing Emergency SOS via Satellite feature. It even includes end-to-end encryption. 

The Maps app will now include topographic information, which shows variations in elevation. It's a useful feature for activities like hiking where elevation changes are important to know ahead of time. Speaking of hiking, the Maps app will also soon include hiking maps for all 63 U.S. National Parks. 

The Wallet app is getting tap-to-cash, which allows you to exchange Apple Cash without sharing your phone number. Event tickets stored in the Wallet app are also getting redesigned with more event details. 

The Journal and Gaming apps are getting updates, too. The Journal app is getting a mood-tracking feature and journaling insights dashboard, which should be popular among bullet journaling enthusiasts. iOS 18 will also include a "Game Mode" for iPhone to improve frame rates and connections to Bluetooth accessories like controllers. 

The Photos app is getting a big redesign! There's now a single view that has your library at the top of the screen and your albums below. There are new filters to make it easier to find photos in your library (i.e. filtering out screenshots). 

There are also now "Collections" in the Photos app, which automatically organizes photos based on their content or the date they were taken. You will be able to share entire Collections if you want. 

You can customize the order of Collections and albums in the Photos app as well as a new carousel that highlights specific photos. 

Audio and Home tech is up next. Ron Huang, Vice President of Sensing and Connectivity, steps up to tell us about it. 

AirPods can now detect when you nod or shake your head so you can respond to prompts from Siri without speaking out loud. It sounds like a cool feature for using Siri in crowded or quiet places. 

AirPods Pro are also getting voice isolation to help your voice sound better even when there's a lot of background noise around you. Apple has also built a Spatial Audio API to support personalized spatial audio on AirPods. 

AppleTV+ is getting "Insight," a feature that resembles the trivia feature in Prime Video. It will give you info about the show or movie you're watching, the actors in it, and more fun facts. Apple is also using machine learning to enhance audio quality on AppleTV+ and improve subtitles. 

Time for watchOS 11 with David Clark, Senior Director of watchOS Engineering! 

The first new feature for watchOS 11 is "Training Load", which will use a new algorithm to calculate an "effort rating" for workouts. By tracking workout intensity, Training Load can help you monitor the impact of your workouts on your body over time, helping to prevent injuries and over-exertion.  

You can also now customize the summary tab in the Fitness app on iPhone. You can adjust or pause your activity rings now, too, a helpful feature if you have an injury or just want a day off.

Dr. Sumbul Ahmad Desai, Vice President of Health, is up to talk about new health features for the Watch. Apple Watch is getting a "Vitals" app for viewing health metrics in greater detail. It can even notify you if your vitals data goes outside normal levels. 

The Cycle Tracking feature in the Health app is getting a helpful update, as well. It will now include features for monitoring pregnancies and prompt users to update certain health benchmarks that may change during pregnancy, such as heart rate. 

New widgets are coming to the Smart Stack in watchOS 11, including automated widgets like the precipitation widget, which can warn you when rain is coming. The Translate app on Apple Watch is getting a widget, as well. It will allow you to use machine learning to dictate phrases to translate directly from your Apple Watch. 

The Apple Watch is also getting a helpful personal safety feature called Check In, which allows you to notify someone in your contacts when your watch detects that you made it home safely. 

watchOS 11 is getting a redesigned Photos watch face with an algorithm that will help you find the best photos to use as watch faces. It can also automate adding filters, numerals, and framing. 

Craig Federighi is back to talk about iPadOS. 

iPadOS 18 is getting the same home screen and control center customization features and Photos app updates as iOS 18. Apps are also getting a customizable floating tab bar and improved animations. 

SharePlay now allows you to tap and draw on screen or even remotely take control of someone else's iPad. This feature could come in handy for helping friends and family with iPad tech difficulties. 

It's finally here! That's right: Apple is finally giving the iPad a Calculator app. It includes everything the iPhone Calculator app has plus a special "Math Notes" feature that can solve handwritten math for you. 

It includes support for variables, too, so you can use it to solve more complex math, such as physics problems. It can even create graphs for you. The regular Notes app will have access to Math Notes features, as well. Sounds like a great homework helper tool for students! 

The Notes app on iPadOS is getting a "smart script" feature that can learn and replicate your handwriting. This allows the app to clean up your handwriting or even convert typed text into handwritten text. 

Time for macOS 15 (after a brief cutscene of Craig Federighi doing parkour). 

macOS 15 is officially named "Sequoia" after the California National Park. 

macOS Sequoia will include many of the same features coming to iOS and iPadOS, including the new Messages features, a typed version of Math Notes, and the trail features in Maps. 

The Continuity app is getting improvements to make it more convenient. "iPhone Mirroring" will let you remotely access your iPhone from your Mac without needing to physically take out your phone. Clicking notifications on your Mac will open them in iPhone Mirroring mode, too. You can also use iPhone Mirroring to more easily move files between your phone and your Mac. Don't worry, though. Your iPhone will stay locked while you're mirroring it. 

macOS 15 will include a neat "Tiles" feature, which automatically sizes windows when you place them around your screen so they are laid out in organized tiles. There will also be a "Presenter Preview" feature for previewing which windows you're sharing on video calls. There will also be background effects for video calls. Both features have been around on other platforms for a while now, but they're still useful additions to macOS. 

The Apple ecosystem is getting a Passwords app, as well. It will definitely be helpful, but Apple is pretty late to the party with this feature, too, which has been around on competitors' devices and browsers for years. 

Safari is getting some much-needed improvements. The new AI Highlights feature will make it easier to quickly find important info on web pages and summarize content as you're browsing. There will also be a distraction-free reading mode. Plus, you can now use picture-in-picture video with just a tap. 

Craig Federighi is now covering gaming on the Apple ecosystem. He highlighted Baldur's Gate 3, which runs on Mac. Apple is releasing Game Porting Toolkit 2, an updated version of the toolkit many developers have been using to bring their games to Mac. Hopefully, this updated version brings even more games like Baldur's Gate 3 to Mac from other platforms and brings more Mac games to iPhone and iPad. 

Speaking of new games on Mac, Federighi mentions several coming to the Apple ecosystem soon including Valheim, Frostpunk 2, and Control. 

Federighi turns things over to Marc-Alexis Cote, Vice President and Executive Producer of Assassin's Creed, to talk about more Ubisoft games coming to Mac, including Assassin's Creed: Shadows and Prince of Persia: The Lost Crown. Assassin's Creed: Shadows will be on Mac at launch, just like PC, and it's going to be available on iPad like Assassin's Creed: Mirage. Is this the start of an exciting new era for gaming in the Apple ecosystem? I sure hope so! 

Craig Federighi closes out the OS updates by announcing they will all be available to users this fall. Apple usually releases a developer beta in July followed by the full versions of OS updates in September, so mark your calendars. Now back to Tim Cook. 

Looks like it's time to finally hear about Apple's AI plans! 

"Recent developments in generative intelligence and large language models offer powerful capabilities that provide the opportunity to take the experience of using Apple products to new heights," Tim Cook notes. 

"As we look to build in these incredible new capabilities, we want to make sure the outcome reflects the core of our products." 

Cook highlights five core traits Apple's AI platform will center around: powerful, intuitive, integrated, personal, and private. 

Just as we reported last week, Apple's AI is called Apple Intelligence. Tim Cook turns things back over to Craig Federighi to tell us more. 

Apple seems to be focusing heavily on personalization in its AI platform, but Federighi and Cook are both emphasizing Apple's commitment to privacy. 

The first feature announced is notification summaries, which was rumored earlier this year. It can also summarize content in other apps, like Mail, Notes, Pages, and Safari. 

Apple Intelligence can generate images of people in your contacts in different styles. You can use its image generation tools in Messages as well as Keynote, Notes, Pages, and others. 

Apple Intelligence will be able to perform actions for you, as well. It will be able to understand context for individual users so it can do things like remember a podcast you listened to recently or tell if you're busy based on info from a variety of apps and messages. 

Federighi is now covering security for Apple Intelligence. On-device processing is front and center. It looks like it will work on the A17 Pro chip as well as all the M-series chips. Most of the models for the "semantic index" for Apple Intelligence will run on-device. 

For models that do need cloud computing, Apple Intelligence will use Private Cloud Compute, a new type of cloud computing running on Apple Silicon with additional security features. An algorithm on your Apple device will determine when a task requires using Private Cloud Compute. Only the absolute minimum data needed will go through the Cloud and no data will be stored there. 

Kelsey Peterson, Director of Machine Learning and AI, is up to talk about Siri. As expected, Siri is getting a big makeover, starting with a new app icon design. 

When you activate Siri, a multicolored frame will now appear on-screen. Siri can also understand natural language more accurately, a much-needed improvement. You can tell Siri follow-up commands or ask follow-up questions, too, since Siri can now remember context in conversations. 

Apple Intelligence is helping out with Siri, too. Siri will soon get on-screen awareness, allowing it to recognize additional context on-screen, such as an address or phone number shared in a text message. 

Siri is getting a Photos app integration, too. It will soon be able to help you edit photos. 

Siri will use Apple Intelligence to access personal context for individual users. This will allow it to understand a wider range of questions and remember things you recently did on your phone. To demonstrate, Peterson asks Siri about plans she made with her mom. Siri identifies a lunch plan from a text conversation. It can do the same thing with content from across your device and apps. 

Apple Intelligence is coming to the Mail app as well as third-party apps. It will be able to do things like summarize content in apps, generate content like cover letter text or emails, and help you find things in apps. For example, it can identify "priority" notifications in the Mail app. 

Apple Intelligence will power a "Reduced Interruptions" focus mode, as well, which will reduce and summarize notifications without muting them entirely. 

The rumored AI-generated emojis feature is official! "Genmoji" can generate custom emojis using text prompts. It can even create emojis of people in your contacts. 

Similarly, Apple is also introducing "Image Playground," which looks like an AI image generator. It runs on-device and won't include a limit on how many images you can generate. There will be three art styles: Animation, Illustration, and Sketch. Rather than text prompts, it generates images based on a few concepts you pick, such as a "party" theme or someone in your contacts. 

Image Playground will be available in Messages, Keynote, Freeform, and third-party apps. It will also have its own dedicated app. 

The Photos app is getting a new "Cleanup" tool, which is basically Apple's version of Google's Magic Eraser. It will use Apple Intelligence to identify and remove background objects in photos. 

Apple Intelligence will also be able to understand the content of videos in your library so you can search for them using text. It can identify photos and videos to feature in "Memory Movies" for you, as well. 

Craig Federighi is now covering Apple's collaboration with OpenAI's ChatGPT. It will be integrated into Siri and Apple Intelligence, but it will only come into play when you specifically allow Siri to share your question with ChatGPT. You will also be able to use ChatGPT to compose text and generate images if you prefer it over Apple's image generation tool. 

Federighi is also highlighting that users' data won't be shared with ChatGPT and they won't have to pay extra to use it or create an account with OpenAI. It looks like the feature effectively makes it easier to access ChatGPT from Apple devices without embedding ChatGPT into Siri itself. 

Users can use Siri and Apple Intelligence but opt not to use ChatGPT if they want. It's an important clarification for privacy and security-minded users, especially considering recent controversy surrounding OpenAI. 

Apple Intelligence will roll out as a beta on iPhone, iPad, and Mac starting this fall, with more features to follow later in the year. 

Now back to Tim Cook to wrap things up. It looks like there won't be any "one more thing" announcement this year.  

That's all for the keynote, but stay tuned for more updates today as we unpack everything Apple announced in the keynote presentation as well as more info from this afternoon's Platforms State of the Union. 

Thanks for joining us for the WWDC 2024 keynote presentation! A lot of our predictions for this year's updates to iOS, iPadOS, and macOS panned out, so it will be exciting to see how these new features work when they launch. 

We will continue covering the latest WWDC news this week and dive into everything announced today in more depth. So, make sure you stick around for more updates on Apple, its new AI platform, and more.