Announcements from WWDC 2024

Today Apple kicked off the 35th Worldwide Developers Conference (WWDC) with their traditional keynote event. While this event is primarily for software developers attending workshops to get the latest features into their apps, the keynote event is of interest to anyone who likes to keep up with the latest Apple news. This is the event where they unveil their annual software updates. Software developers receive prerelease "beta" versions today so they can prepare their apps, and the general public gets these updates in the fall, typically in September.

Apple is releasing a public beta in July to solicit feedback and crash reports, but I strongly advise against using the beta unless you know what you are doing. Take regular backups and don't install it on a device you can't live without in case things go sideways!

If you want to watch the full nearly two-hour keynote event, here it is.

https://www.youtube.com/watch?v=RXeOiIDNNek

As I normally do, here is my concise recap of today's announcements.

Today's announcements were huge! Apple is playing catch-up in the AI world and if the final product is anywhere as great as their demo I'll be greatly impressed. But there are a lot of other huge features as well. The single feature that I am most excited about is iPhone Mirroring, which will allow you to use your iPhone from your Mac.

The list of features below are nowhere near comprehensive. There are always dozens of minor changes and tweaks and sometimes even a few big features that Apple doesn't bother to mention. Some features are saved for a later version of iOS 18. And some features may be restricted to newer devices.

Several of the features are pretty much universal across all devices:

• Messages gets a huge upgrade. RCS support (the successor to SMS), tapbacks lets you reply with any emoji, scheduled send, text formatting and effects, and the ability to send and receive messages via satellite when you have no celluar service.
• Mail gets message categorization (later this year). Digest view provides an overview of all messages from one sender.
• Maps gets topographical maps with custom and offline hiking trails as well as turn-by-turn navigation for hikes.
• Passwords app makes managing your iCloud Keychain even easier. Even available on Windows! It allows you to easily view and modify saved website passwords, passkeys, authentication codes, and WiFi passwords! Previously this was buried in Settings, so I'm glad to see it get more attention.
• Safari analyzes web pages, calling out relevent information like addresses, music, and people. It gets a new Reader Mode with an AI summary and table of contents. And a new video Viewer mode that does for videos what Reader Mode does for articles.
• Math Notes auto calculates things for you in the Notes app just by typing it out.

macOS 15 Sequoia

It's unclear whether macOS 15 is named for Sequoia National Park, or the sequoia redwood trees. The update this year sounds exciting. macOS 15 Sequoia is only dropping support for MacBook Air 2018 and 2019.

• The biggest new feature is iPhone Mirroring, which allows you to control your iPhone from your Mac when it is nearby, all while keeping your iPhone locked. You can drag and drop files across devices. Notifications from iPhone appear on your Mac, and clicking them initiates the remote control and opens the app. This is going to be huge for me since most of my appointments are remote and I'll be able to remotely access your iPhone when we are on a call rather than asking you to tap things yourself. It's unclear if this feature is also coming to iPad.
• Window tiling is easier with dragging and dropping or with keyboard shortcuts.
• Presenter Preview allows you to see what will be shared before sharing starts.
• Background Replacement for any video call app.
• Apple Pay comes to third party browsers.

iOS 18

iOS 18 is a really big update, bringing many customization features people have been begging for for years. iOS 18 runs on all devices that run iOS 17.

• Photos get a huge redesign with a larger focus on AI-generated albums and memories. As well as the ability to easily filter out images you don't want to see. And an enormously improved search ("show me photos of Jess skateboarding in a tie-die shirt").
• Home Screen personalization allows you to freely rearrange icons, tint the color of all icons, and adds support for Dark Mode.
• Controls Gallery allows you to add buttons for various apps to the Control Center, Lock Screen, and Action Button. You can now even change your camera and flashlight button to something else if you prefer.
• Control Center is now customizable with third-party Controls.
• Lock apps with Face ID, preventing others from opening them or from their content appearing in other places like Search and Notifications. Or apps can be hidden so they don't show up unless you access the hidden folder.
• Phone calls can be recorded with a transcription and AI summary.
• Wallet gets a few big upgrades. Tap two iPhones together to transfer Apple Cash without exchanging contact info. Event guides with details like a venue map and weather in your ticket. And bank rewards tracking.
• Emergency SOS live video allows you to share videos and photos with 911.
• SharePlay adds the ability to remote control an iPad or iPhone in FaceTime screen sharing.
• Journal gets several small new features including search.
• Game Mode improves performance with games.

iPadOS 18

iPadOS is basically just a modified version of iOS. Therefore, iPadOS 18 includes the features of iOS 18 above plus the following. iPadOS 18 is dropping support for 10.5-in iPad Pro (1st generation), 12.9-in iPad Pro (2nd generation) and iPad (6th generation).

• Calculator is redesigned with history and unit conversion. And Math Notes lets you draw expressions with Apple Pencil, either in the Calculator app or in the Notes app.
• Notes gets Smart Script, which improves your penmanship by learning your handwriting and rewriting it to make it more legible and consistent. You can also paste content into your notes which gets transcribed into your own handwriting. Or scratch out content to remove it. Or write a formuala and have it solved. Or sketch something that ges automatically converted into an image using AI.

watchOS 11

WatchOS gets several new features this year. Little is groundbreaking but they are all nice quality of life improvements. This year watchOS 11 drops support fro the Series 4, Series 5, and first-gen SE.

• Training Load tracks intensity of your worksouts.
• Vitals app can see when there are warning signs of a larger health issues and promts you if you should investigate.
• Pause your Rings when you need to recover.
• Pregnancy features can help you keep better track of your health.
• Smart Stack is getting more intelligent and available to third parties
• Check In feature is coming to workouts
• New Photos watch face is better at finding good photos.

Audio & Home

This covers Apple TV, Apple TV+, HomePod, Apple Home/HomeKit, and AirPods. tvOS 18 is compatible with Apple TV HD and later, same as tvOS 17. It appears that all HomePod models are getting this update.

• InSight in Apple TV+ allows you to easily see what actors and music are in a certain scene.
• Enhanced Dialogue uses AI to make dialogue easier to hear.
• Subtitles are briefly enable when you skip back or mute
• 21:9 projector support
• New screensavers with portraits, scenes from TV+ shows, Snoopy.
• HomeKit gets support for robot vacuums and guest access, can get energy data from your power company.
• AirPods (unspecified models) can answer Siri by shaking your head yes or no. Voice Isolation for AirPods Pro; spatial audio for gaming

visionOS 2

Apple released visionOS 1.2 today for Apple Vision Pro with some minor updates, but they also launched visionOS 2 beta.

• Mac Virtual Display can be extended into an ultrawide display, similar to dual 4k screens.
• More Immersive Video content is coming to Apple TV app.
• Final Cut and Vimeo are getting Spatial Video support.
• Canon EOS R7 is getting a Spatial Video lens.
• Immersive Video tools are coming to Davinci Resolve Studio, Blackmagic.
• 2D photos can be shown in 3D using AI.
• SharePlay will allow for better sharing of content with others, such as Photos.
• New gestures make navigating easier.
• Travel Mode works better on trains.
• AirPlay Receiver, though not discussed I assume means you can AirPlay any content from your iPhone, iPad, or Mac to a virtual display.
• Mouse support.
• Launching soon in China, Japan, Singapore, Australia, Canada, France, Germany, and the UK.

Apple Intelligence

The big announcement today is Apple Intelligence, Apple's AI. Below is a short 3-minute video that runs through some of the most useful features. These features are not in the beta currently but are expected to enter beta this fall. It probably means it will be available to the public around the end of the year or the beginning of 2025. Because processing is done mostly on-device It will be available only for iPhone 15 Pro and iPad/Mac with an M chip.

https://youtu.be/LtIWE4CCX5M

Apple Intelligence has some pretty amazing power if it works as well as they claim.

• Text manipulation and generation. For example, you can ask it to write a document, summarize text, summarize a recorded call, rewrite text, check grammar and style, or generate a reply to an email.
• Image manipulation and generation. Create custom emoji ("A t-rex in a tutu on a surfboard"). Image Playground lets you generate images from words, ideas, and people in your Photos app. The Photos app can remove distractions from your photos, create custom slideshows, and adjust color. Sketches in Notes can be converted into an image.
• Auto summarization of many things such as notifications, emails, web pages. Preview text on email messages gives you a summary rather than the first few words.
• Prioritizing of notifications can put your most important messages at the top or allow important messages to break through Focus.
• Siri will get far more capable by being able to sift through the data in your documents, messages, and emails. And it remembers context. You can ask "when does my mom's flight arrive" then "and what were we planning to do for lunch?". Siri can even interact with apps in new ways. You can ask "Siri, summarize my meeting notes and email them to my boss." Siri is also capable of understanding if you backtrack or stumble over your words.
• ChatGPT support. If a question would benefit from off-device AI you may be promted to send your request to ChatGPT, with other AI chatbots being added in the future. Requests sent to ChatGPT are free, unlogged, and sent via private relay.